├── .gitattributes ├── .gitignore ├── COPYING ├── MANIFEST.in ├── README.rst ├── demo ├── embedded.py ├── empty.py ├── encoding.py ├── exceptions.py ├── iso_8859_1.py ├── module_globals.py ├── recurse.py ├── recurse2.py ├── recurse3.py ├── recurse4.py ├── the_main.py ├── the_other.py ├── threads.py ├── twocalls.py ├── twocalls2.py ├── utf_8.py └── utf_8_bom.py ├── pprofile ├── __init__.py ├── __main__.py ├── _version.py └── zope.py ├── setup.cfg ├── setup.py ├── test.sh ├── versioneer.py └── zpprofile.py /.gitattributes: -------------------------------------------------------------------------------- 1 | pprofile/_version.py export-subst 2 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | dist/ 2 | build/ 3 | pprofile.egg-info/ 4 | *.pyc 5 | -------------------------------------------------------------------------------- /COPYING: -------------------------------------------------------------------------------- 1 | GNU GENERAL PUBLIC LICENSE 2 | Version 2, June 1991 3 | 4 | Copyright (C) 1989, 1991 Free Software Foundation, Inc., 5 | 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA 6 | Everyone is permitted to copy and distribute verbatim copies 7 | of this license document, but changing it is not allowed. 8 | 9 | Preamble 10 | 11 | The licenses for most software are designed to take away your 12 | freedom to share and change it. By contrast, the GNU General Public 13 | License is intended to guarantee your freedom to share and change free 14 | software--to make sure the software is free for all its users. This 15 | General Public License applies to most of the Free Software 16 | Foundation's software and to any other program whose authors commit to 17 | using it. (Some other Free Software Foundation software is covered by 18 | the GNU Lesser General Public License instead.) You can apply it to 19 | your programs, too. 20 | 21 | When we speak of free software, we are referring to freedom, not 22 | price. Our General Public Licenses are designed to make sure that you 23 | have the freedom to distribute copies of free software (and charge for 24 | this service if you wish), that you receive source code or can get it 25 | if you want it, that you can change the software or use pieces of it 26 | in new free programs; and that you know you can do these things. 27 | 28 | To protect your rights, we need to make restrictions that forbid 29 | anyone to deny you these rights or to ask you to surrender the rights. 30 | These restrictions translate to certain responsibilities for you if you 31 | distribute copies of the software, or if you modify it. 32 | 33 | For example, if you distribute copies of such a program, whether 34 | gratis or for a fee, you must give the recipients all the rights that 35 | you have. You must make sure that they, too, receive or can get the 36 | source code. And you must show them these terms so they know their 37 | rights. 38 | 39 | We protect your rights with two steps: (1) copyright the software, and 40 | (2) offer you this license which gives you legal permission to copy, 41 | distribute and/or modify the software. 42 | 43 | Also, for each author's protection and ours, we want to make certain 44 | that everyone understands that there is no warranty for this free 45 | software. If the software is modified by someone else and passed on, we 46 | want its recipients to know that what they have is not the original, so 47 | that any problems introduced by others will not reflect on the original 48 | authors' reputations. 49 | 50 | Finally, any free program is threatened constantly by software 51 | patents. We wish to avoid the danger that redistributors of a free 52 | program will individually obtain patent licenses, in effect making the 53 | program proprietary. To prevent this, we have made it clear that any 54 | patent must be licensed for everyone's free use or not licensed at all. 55 | 56 | The precise terms and conditions for copying, distribution and 57 | modification follow. 58 | 59 | GNU GENERAL PUBLIC LICENSE 60 | TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION 61 | 62 | 0. This License applies to any program or other work which contains 63 | a notice placed by the copyright holder saying it may be distributed 64 | under the terms of this General Public License. The "Program", below, 65 | refers to any such program or work, and a "work based on the Program" 66 | means either the Program or any derivative work under copyright law: 67 | that is to say, a work containing the Program or a portion of it, 68 | either verbatim or with modifications and/or translated into another 69 | language. (Hereinafter, translation is included without limitation in 70 | the term "modification".) Each licensee is addressed as "you". 71 | 72 | Activities other than copying, distribution and modification are not 73 | covered by this License; they are outside its scope. The act of 74 | running the Program is not restricted, and the output from the Program 75 | is covered only if its contents constitute a work based on the 76 | Program (independent of having been made by running the Program). 77 | Whether that is true depends on what the Program does. 78 | 79 | 1. You may copy and distribute verbatim copies of the Program's 80 | source code as you receive it, in any medium, provided that you 81 | conspicuously and appropriately publish on each copy an appropriate 82 | copyright notice and disclaimer of warranty; keep intact all the 83 | notices that refer to this License and to the absence of any warranty; 84 | and give any other recipients of the Program a copy of this License 85 | along with the Program. 86 | 87 | You may charge a fee for the physical act of transferring a copy, and 88 | you may at your option offer warranty protection in exchange for a fee. 89 | 90 | 2. You may modify your copy or copies of the Program or any portion 91 | of it, thus forming a work based on the Program, and copy and 92 | distribute such modifications or work under the terms of Section 1 93 | above, provided that you also meet all of these conditions: 94 | 95 | a) You must cause the modified files to carry prominent notices 96 | stating that you changed the files and the date of any change. 97 | 98 | b) You must cause any work that you distribute or publish, that in 99 | whole or in part contains or is derived from the Program or any 100 | part thereof, to be licensed as a whole at no charge to all third 101 | parties under the terms of this License. 102 | 103 | c) If the modified program normally reads commands interactively 104 | when run, you must cause it, when started running for such 105 | interactive use in the most ordinary way, to print or display an 106 | announcement including an appropriate copyright notice and a 107 | notice that there is no warranty (or else, saying that you provide 108 | a warranty) and that users may redistribute the program under 109 | these conditions, and telling the user how to view a copy of this 110 | License. (Exception: if the Program itself is interactive but 111 | does not normally print such an announcement, your work based on 112 | the Program is not required to print an announcement.) 113 | 114 | These requirements apply to the modified work as a whole. If 115 | identifiable sections of that work are not derived from the Program, 116 | and can be reasonably considered independent and separate works in 117 | themselves, then this License, and its terms, do not apply to those 118 | sections when you distribute them as separate works. But when you 119 | distribute the same sections as part of a whole which is a work based 120 | on the Program, the distribution of the whole must be on the terms of 121 | this License, whose permissions for other licensees extend to the 122 | entire whole, and thus to each and every part regardless of who wrote it. 123 | 124 | Thus, it is not the intent of this section to claim rights or contest 125 | your rights to work written entirely by you; rather, the intent is to 126 | exercise the right to control the distribution of derivative or 127 | collective works based on the Program. 128 | 129 | In addition, mere aggregation of another work not based on the Program 130 | with the Program (or with a work based on the Program) on a volume of 131 | a storage or distribution medium does not bring the other work under 132 | the scope of this License. 133 | 134 | 3. You may copy and distribute the Program (or a work based on it, 135 | under Section 2) in object code or executable form under the terms of 136 | Sections 1 and 2 above provided that you also do one of the following: 137 | 138 | a) Accompany it with the complete corresponding machine-readable 139 | source code, which must be distributed under the terms of Sections 140 | 1 and 2 above on a medium customarily used for software interchange; or, 141 | 142 | b) Accompany it with a written offer, valid for at least three 143 | years, to give any third party, for a charge no more than your 144 | cost of physically performing source distribution, a complete 145 | machine-readable copy of the corresponding source code, to be 146 | distributed under the terms of Sections 1 and 2 above on a medium 147 | customarily used for software interchange; or, 148 | 149 | c) Accompany it with the information you received as to the offer 150 | to distribute corresponding source code. (This alternative is 151 | allowed only for noncommercial distribution and only if you 152 | received the program in object code or executable form with such 153 | an offer, in accord with Subsection b above.) 154 | 155 | The source code for a work means the preferred form of the work for 156 | making modifications to it. For an executable work, complete source 157 | code means all the source code for all modules it contains, plus any 158 | associated interface definition files, plus the scripts used to 159 | control compilation and installation of the executable. However, as a 160 | special exception, the source code distributed need not include 161 | anything that is normally distributed (in either source or binary 162 | form) with the major components (compiler, kernel, and so on) of the 163 | operating system on which the executable runs, unless that component 164 | itself accompanies the executable. 165 | 166 | If distribution of executable or object code is made by offering 167 | access to copy from a designated place, then offering equivalent 168 | access to copy the source code from the same place counts as 169 | distribution of the source code, even though third parties are not 170 | compelled to copy the source along with the object code. 171 | 172 | 4. You may not copy, modify, sublicense, or distribute the Program 173 | except as expressly provided under this License. Any attempt 174 | otherwise to copy, modify, sublicense or distribute the Program is 175 | void, and will automatically terminate your rights under this License. 176 | However, parties who have received copies, or rights, from you under 177 | this License will not have their licenses terminated so long as such 178 | parties remain in full compliance. 179 | 180 | 5. You are not required to accept this License, since you have not 181 | signed it. However, nothing else grants you permission to modify or 182 | distribute the Program or its derivative works. These actions are 183 | prohibited by law if you do not accept this License. Therefore, by 184 | modifying or distributing the Program (or any work based on the 185 | Program), you indicate your acceptance of this License to do so, and 186 | all its terms and conditions for copying, distributing or modifying 187 | the Program or works based on it. 188 | 189 | 6. Each time you redistribute the Program (or any work based on the 190 | Program), the recipient automatically receives a license from the 191 | original licensor to copy, distribute or modify the Program subject to 192 | these terms and conditions. You may not impose any further 193 | restrictions on the recipients' exercise of the rights granted herein. 194 | You are not responsible for enforcing compliance by third parties to 195 | this License. 196 | 197 | 7. If, as a consequence of a court judgment or allegation of patent 198 | infringement or for any other reason (not limited to patent issues), 199 | conditions are imposed on you (whether by court order, agreement or 200 | otherwise) that contradict the conditions of this License, they do not 201 | excuse you from the conditions of this License. If you cannot 202 | distribute so as to satisfy simultaneously your obligations under this 203 | License and any other pertinent obligations, then as a consequence you 204 | may not distribute the Program at all. For example, if a patent 205 | license would not permit royalty-free redistribution of the Program by 206 | all those who receive copies directly or indirectly through you, then 207 | the only way you could satisfy both it and this License would be to 208 | refrain entirely from distribution of the Program. 209 | 210 | If any portion of this section is held invalid or unenforceable under 211 | any particular circumstance, the balance of the section is intended to 212 | apply and the section as a whole is intended to apply in other 213 | circumstances. 214 | 215 | It is not the purpose of this section to induce you to infringe any 216 | patents or other property right claims or to contest validity of any 217 | such claims; this section has the sole purpose of protecting the 218 | integrity of the free software distribution system, which is 219 | implemented by public license practices. Many people have made 220 | generous contributions to the wide range of software distributed 221 | through that system in reliance on consistent application of that 222 | system; it is up to the author/donor to decide if he or she is willing 223 | to distribute software through any other system and a licensee cannot 224 | impose that choice. 225 | 226 | This section is intended to make thoroughly clear what is believed to 227 | be a consequence of the rest of this License. 228 | 229 | 8. If the distribution and/or use of the Program is restricted in 230 | certain countries either by patents or by copyrighted interfaces, the 231 | original copyright holder who places the Program under this License 232 | may add an explicit geographical distribution limitation excluding 233 | those countries, so that distribution is permitted only in or among 234 | countries not thus excluded. In such case, this License incorporates 235 | the limitation as if written in the body of this License. 236 | 237 | 9. The Free Software Foundation may publish revised and/or new versions 238 | of the General Public License from time to time. Such new versions will 239 | be similar in spirit to the present version, but may differ in detail to 240 | address new problems or concerns. 241 | 242 | Each version is given a distinguishing version number. If the Program 243 | specifies a version number of this License which applies to it and "any 244 | later version", you have the option of following the terms and conditions 245 | either of that version or of any later version published by the Free 246 | Software Foundation. If the Program does not specify a version number of 247 | this License, you may choose any version ever published by the Free Software 248 | Foundation. 249 | 250 | 10. If you wish to incorporate parts of the Program into other free 251 | programs whose distribution conditions are different, write to the author 252 | to ask for permission. For software which is copyrighted by the Free 253 | Software Foundation, write to the Free Software Foundation; we sometimes 254 | make exceptions for this. Our decision will be guided by the two goals 255 | of preserving the free status of all derivatives of our free software and 256 | of promoting the sharing and reuse of software generally. 257 | 258 | NO WARRANTY 259 | 260 | 11. BECAUSE THE PROGRAM IS LICENSED FREE OF CHARGE, THERE IS NO WARRANTY 261 | FOR THE PROGRAM, TO THE EXTENT PERMITTED BY APPLICABLE LAW. EXCEPT WHEN 262 | OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR OTHER PARTIES 263 | PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED 264 | OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF 265 | MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. THE ENTIRE RISK AS 266 | TO THE QUALITY AND PERFORMANCE OF THE PROGRAM IS WITH YOU. SHOULD THE 267 | PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF ALL NECESSARY SERVICING, 268 | REPAIR OR CORRECTION. 269 | 270 | 12. IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING 271 | WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MAY MODIFY AND/OR 272 | REDISTRIBUTE THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, 273 | INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING 274 | OUT OF THE USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED 275 | TO LOSS OF DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY 276 | YOU OR THIRD PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER 277 | PROGRAMS), EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE 278 | POSSIBILITY OF SUCH DAMAGES. 279 | 280 | END OF TERMS AND CONDITIONS 281 | 282 | How to Apply These Terms to Your New Programs 283 | 284 | If you develop a new program, and you want it to be of the greatest 285 | possible use to the public, the best way to achieve this is to make it 286 | free software which everyone can redistribute and change under these terms. 287 | 288 | To do so, attach the following notices to the program. It is safest 289 | to attach them to the start of each source file to most effectively 290 | convey the exclusion of warranty; and each file should have at least 291 | the "copyright" line and a pointer to where the full notice is found. 292 | 293 | 294 | Copyright (C) 295 | 296 | This program is free software; you can redistribute it and/or modify 297 | it under the terms of the GNU General Public License as published by 298 | the Free Software Foundation; either version 2 of the License, or 299 | (at your option) any later version. 300 | 301 | This program is distributed in the hope that it will be useful, 302 | but WITHOUT ANY WARRANTY; without even the implied warranty of 303 | MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 304 | GNU General Public License for more details. 305 | 306 | You should have received a copy of the GNU General Public License along 307 | with this program; if not, write to the Free Software Foundation, Inc., 308 | 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA. 309 | 310 | Also add information on how to contact you by electronic and paper mail. 311 | 312 | If the program is interactive, make it output a short notice like this 313 | when it starts in an interactive mode: 314 | 315 | Gnomovision version 69, Copyright (C) year name of author 316 | Gnomovision comes with ABSOLUTELY NO WARRANTY; for details type `show w'. 317 | This is free software, and you are welcome to redistribute it 318 | under certain conditions; type `show c' for details. 319 | 320 | The hypothetical commands `show w' and `show c' should show the appropriate 321 | parts of the General Public License. Of course, the commands you use may 322 | be called something other than `show w' and `show c'; they could even be 323 | mouse-clicks or menu items--whatever suits your program. 324 | 325 | You should also get your employer (if you work as a programmer) or your 326 | school, if any, to sign a "copyright disclaimer" for the program, if 327 | necessary. Here is a sample; alter the names: 328 | 329 | Yoyodyne, Inc., hereby disclaims all copyright interest in the program 330 | `Gnomovision' (which makes passes at compilers) written by James Hacker. 331 | 332 | , 1 April 1989 333 | Ty Coon, President of Vice 334 | 335 | This General Public License does not permit incorporating your program into 336 | proprietary programs. If your program is a subroutine library, you may 337 | consider it more useful to permit linking proprietary applications with the 338 | library. If this is what you want to do, use the GNU Lesser General 339 | Public License instead of this License. 340 | -------------------------------------------------------------------------------- /MANIFEST.in: -------------------------------------------------------------------------------- 1 | include COPYING 2 | include README.rst 3 | include versioneer.py 4 | include pprofile/_version.py 5 | -------------------------------------------------------------------------------- /README.rst: -------------------------------------------------------------------------------- 1 | Line-granularity, thread-aware deterministic and statistic pure-python profiler 2 | 3 | Inspired from Robert Kern's line_profiler_ . 4 | 5 | Usage 6 | ===== 7 | 8 | As a command:: 9 | 10 | $ pprofile some_python_executable arg1 ... 11 | 12 | Once `some_python_executable` returns, prints annotated code of each file 13 | involved in the execution. 14 | 15 | As a command, ignoring any files from default `sys.path` (ie, python modules 16 | themselves), for shorter output:: 17 | 18 | $ pprofile --exclude-syspath some_python_executable arg1 ... 19 | 20 | Executing a module, like :code:`python -m`. `--exclude-syspath` is not 21 | recommended in this mode, as it will likely hide what you intend to profile. 22 | Also, explicitly ending pprofile arguments with `--` will prevent accidentally 23 | stealing command's arguments:: 24 | 25 | $ pprofile -m some_python_module -- arg1 ... 26 | 27 | As a module: 28 | 29 | .. code:: python 30 | 31 | import pprofile 32 | 33 | def someHotSpotCallable(): 34 | # Deterministic profiler 35 | prof = pprofile.Profile() 36 | with prof(): 37 | # Code to profile 38 | prof.print_stats() 39 | 40 | def someOtherHotSpotCallable(): 41 | # Statistic profiler 42 | prof = pprofile.StatisticalProfile() 43 | with prof( 44 | period=0.001, # Sample every 1ms 45 | single=True, # Only sample current thread 46 | ): 47 | # Code to profile 48 | prof.print_stats() 49 | 50 | For advanced usage, see :code:`pprofile --help` and :code:`pydoc pprofile`. 51 | 52 | Profiling overhead 53 | ------------------ 54 | 55 | pprofile default mode (`Deterministic profiling`_) has a large overhead. 56 | Part of the reason being that it is written to be as portable as possible 57 | (so no C extension). This large overhead can be an issue, which can be 58 | avoided by using `Statistic profiling`_ at the cost of some result 59 | readability decrease. 60 | 61 | Rule of thumb: 62 | 63 | +-----------------------------+----------------------------+------------------------+ 64 | | Code to profile runs for... | `Deterministic profiling`_ | `Statistic profiling`_ | 65 | +=============================+============================+========================+ 66 | | a few seconds | Yes | No [#]_ | 67 | +-----------------------------+----------------------------+------------------------+ 68 | | a few minutes | Maybe | Yes | 69 | +-----------------------------+----------------------------+------------------------+ 70 | | more (ex: daemon) | No | Yes [#]_ | 71 | +-----------------------------+----------------------------+------------------------+ 72 | 73 | Once you identified the hot spot and you decide you need finer-grained 74 | profiling to understand what needs fixing, you should try to make to-profile 75 | code run for shorter time so you can reasonably use deterministic profiling: 76 | use a smaller data set triggering the same code path, modify the code to only 77 | enable profiling on small pieces of code... 78 | 79 | .. [#] Statistic profiling will not have time to collect 80 | enough samples to produce usable output. 81 | 82 | .. [#] You may want to consider triggering pprofile from 83 | a signal handler or other IPC mechanism to profile 84 | a shorter subset. See `zpprofile.py` for how it can 85 | be used to profile code inside a running (zope) 86 | service (in which case the IPC mechanism is just 87 | Zope normal URL handling). 88 | 89 | Output 90 | ====== 91 | 92 | Supported output formats. 93 | 94 | Callgrind 95 | --------- 96 | 97 | The most useful output mode of pprofile is `Callgrind Profile Format`_, allows 98 | browsing profiling results with kcachegrind_ (or qcachegrind_ on Windows). 99 | 100 | :: 101 | 102 | $ pprofile --format callgrind --out cachegrind.out.threads demo/threads.py 103 | 104 | Callgrind format is implicitly enabled if ``--out`` basename starts with 105 | ``cachegrind.out.``, so above command can be simplified as:: 106 | 107 | $ pprofile --out cachegrind.out.threads demo/threads.py 108 | 109 | If you are analyzing callgrind traces on a different machine, you may want to 110 | use the ``--zipfile`` option to generate a zip file containing all files:: 111 | 112 | $ pprofile --out cachegrind.out.threads --zipfile threads_source.zip demo/threads.py 113 | 114 | Generated files will use relative paths, so you can extract generated archive 115 | in the same path as profiling result, and kcachegrind will load them - and not 116 | your system-wide files, which may differ. 117 | 118 | Annotated code 119 | -------------- 120 | 121 | Human-readable output, but can become difficult to use with large programs. 122 | 123 | :: 124 | 125 | $ pprofile demo/threads.py 126 | 127 | Profiling modes 128 | =============== 129 | 130 | Deterministic profiling 131 | ----------------------- 132 | 133 | In deterministic profiling mode, pprofile gets notified of each executed line. 134 | This mode generates very detailed reports, but at the cost of a large overhead. 135 | Also, profiling hooks being per-thread, either profiling must be enable before 136 | spawning threads (if you want to profile more than just the current thread), 137 | or profiled application must provide ways of enabling profiling afterwards 138 | - which is not very convenient. 139 | 140 | :: 141 | 142 | $ pprofile --threads 0 demo/threads.py 143 | Command line: ['demo/threads.py'] 144 | Total duration: 1.00573s 145 | File: demo/threads.py 146 | File duration: 1.00168s (99.60%) 147 | Line #| Hits| Time| Time per hit| %|Source code 148 | ------+----------+-------------+-------------+-------+----------- 149 | 1| 2| 3.21865e-05| 1.60933e-05| 0.00%|import threading 150 | 2| 1| 5.96046e-06| 5.96046e-06| 0.00%|import time 151 | 3| 0| 0| 0| 0.00%| 152 | 4| 2| 1.5974e-05| 7.98702e-06| 0.00%|def func(): 153 | 5| 1| 1.00111| 1.00111| 99.54%| time.sleep(1) 154 | 6| 0| 0| 0| 0.00%| 155 | 7| 2| 2.00272e-05| 1.00136e-05| 0.00%|def func2(): 156 | 8| 1| 1.69277e-05| 1.69277e-05| 0.00%| pass 157 | 9| 0| 0| 0| 0.00%| 158 | 10| 1| 1.81198e-05| 1.81198e-05| 0.00%|t1 = threading.Thread(target=func) 159 | (call)| 1| 0.000610828| 0.000610828| 0.06%|# /usr/lib/python2.7/threading.py:436 __init__ 160 | 11| 1| 1.52588e-05| 1.52588e-05| 0.00%|t2 = threading.Thread(target=func) 161 | (call)| 1| 0.000438929| 0.000438929| 0.04%|# /usr/lib/python2.7/threading.py:436 __init__ 162 | 12| 1| 4.79221e-05| 4.79221e-05| 0.00%|t1.start() 163 | (call)| 1| 0.000843048| 0.000843048| 0.08%|# /usr/lib/python2.7/threading.py:485 start 164 | 13| 1| 6.48499e-05| 6.48499e-05| 0.01%|t2.start() 165 | (call)| 1| 0.00115609| 0.00115609| 0.11%|# /usr/lib/python2.7/threading.py:485 start 166 | 14| 1| 0.000205994| 0.000205994| 0.02%|(func(), func2()) 167 | (call)| 1| 1.00112| 1.00112| 99.54%|# demo/threads.py:4 func 168 | (call)| 1| 3.09944e-05| 3.09944e-05| 0.00%|# demo/threads.py:7 func2 169 | 15| 1| 7.62939e-05| 7.62939e-05| 0.01%|t1.join() 170 | (call)| 1| 0.000423908| 0.000423908| 0.04%|# /usr/lib/python2.7/threading.py:653 join 171 | 16| 1| 5.26905e-05| 5.26905e-05| 0.01%|t2.join() 172 | (call)| 1| 0.000320196| 0.000320196| 0.03%|# /usr/lib/python2.7/threading.py:653 join 173 | 174 | Note that time.sleep call is not counted as such. For some reason, python is 175 | not generating c_call/c_return/c_exception events (which are ignored by current 176 | code, as a result). 177 | 178 | Statistic profiling 179 | ------------------- 180 | 181 | In statistic profiling mode, pprofile periodically snapshots the current 182 | callstack(s) of current process to see what is being executed. 183 | As a result, profiler overhead can be dramatically reduced, making it possible 184 | to profile real workloads. Also, as statistic profiling acts at the 185 | whole-process level, it can be toggled independently of profiled code. 186 | 187 | The downside of statistic profiling is that output lacks timing information, 188 | which makes it harder to understand. 189 | 190 | :: 191 | 192 | $ pprofile --statistic .01 demo/threads.py 193 | Command line: ['demo/threads.py'] 194 | Total duration: 1.0026s 195 | File: demo/threads.py 196 | File duration: 0s (0.00%) 197 | Line #| Hits| Time| Time per hit| %|Source code 198 | ------+----------+-------------+-------------+-------+----------- 199 | 1| 0| 0| 0| 0.00%|import threading 200 | 2| 0| 0| 0| 0.00%|import time 201 | 3| 0| 0| 0| 0.00%| 202 | 4| 0| 0| 0| 0.00%|def func(): 203 | 5| 288| 0| 0| 0.00%| time.sleep(1) 204 | 6| 0| 0| 0| 0.00%| 205 | 7| 0| 0| 0| 0.00%|def func2(): 206 | 8| 0| 0| 0| 0.00%| pass 207 | 9| 0| 0| 0| 0.00%| 208 | 10| 0| 0| 0| 0.00%|t1 = threading.Thread(target=func) 209 | 11| 0| 0| 0| 0.00%|t2 = threading.Thread(target=func) 210 | 12| 0| 0| 0| 0.00%|t1.start() 211 | 13| 0| 0| 0| 0.00%|t2.start() 212 | 14| 0| 0| 0| 0.00%|(func(), func2()) 213 | (call)| 96| 0| 0| 0.00%|# demo/threads.py:4 func 214 | 15| 0| 0| 0| 0.00%|t1.join() 215 | 16| 0| 0| 0| 0.00%|t2.join() 216 | File: /usr/lib/python2.7/threading.py 217 | File duration: 0s (0.00%) 218 | Line #| Hits| Time| Time per hit| %|Source code 219 | ------+----------+-------------+-------------+-------+----------- 220 | [...] 221 | 308| 0| 0| 0| 0.00%| def wait(self, timeout=None): 222 | [...] 223 | 338| 0| 0| 0| 0.00%| if timeout is None: 224 | 339| 1| 0| 0| 0.00%| waiter.acquire() 225 | 340| 0| 0| 0| 0.00%| if __debug__: 226 | [...] 227 | 600| 0| 0| 0| 0.00%| def wait(self, timeout=None): 228 | [...] 229 | 617| 0| 0| 0| 0.00%| if not self.__flag: 230 | 618| 0| 0| 0| 0.00%| self.__cond.wait(timeout) 231 | (call)| 1| 0| 0| 0.00%|# /usr/lib/python2.7/threading.py:308 wait 232 | [...] 233 | 724| 0| 0| 0| 0.00%| def start(self): 234 | [...] 235 | 748| 0| 0| 0| 0.00%| self.__started.wait() 236 | (call)| 1| 0| 0| 0.00%|# /usr/lib/python2.7/threading.py:600 wait 237 | 749| 0| 0| 0| 0.00%| 238 | 750| 0| 0| 0| 0.00%| def run(self): 239 | [...] 240 | 760| 0| 0| 0| 0.00%| if self.__target: 241 | 761| 0| 0| 0| 0.00%| self.__target(*self.__args, **self.__kwargs) 242 | (call)| 192| 0| 0| 0.00%|# demo/threads.py:4 func 243 | 762| 0| 0| 0| 0.00%| finally: 244 | [...] 245 | 767| 0| 0| 0| 0.00%| def __bootstrap(self): 246 | [...] 247 | 780| 0| 0| 0| 0.00%| try: 248 | 781| 0| 0| 0| 0.00%| self.__bootstrap_inner() 249 | (call)| 192| 0| 0| 0.00%|# /usr/lib/python2.7/threading.py:790 __bootstrap_inner 250 | [...] 251 | 790| 0| 0| 0| 0.00%| def __bootstrap_inner(self): 252 | [...] 253 | 807| 0| 0| 0| 0.00%| try: 254 | 808| 0| 0| 0| 0.00%| self.run() 255 | (call)| 192| 0| 0| 0.00%|# /usr/lib/python2.7/threading.py:750 run 256 | 257 | Some details are lost (not all executed lines have a non-null hit-count), but 258 | the hot spot is still easily identifiable in this trivial example, and its call 259 | stack is still visible. 260 | 261 | Thread-aware profiling 262 | ====================== 263 | 264 | ``ThreadProfile`` class provides the same features as ``Profile``, but uses 265 | ``threading.settrace`` to propagate tracing to ``threading.Thread`` threads 266 | started after profiling is enabled. 267 | 268 | Limitations 269 | ----------- 270 | 271 | The time spent in another thread is not discounted from interrupted line. 272 | On the long run, it should not be a problem if switches are evenly distributed 273 | among lines, but threads executing fewer lines will appear as eating more CPU 274 | time than they really do. 275 | 276 | This is not specific to simultaneous multi-thread profiling: profiling a single 277 | thread of a multi-threaded application will also be polluted by time spent in 278 | other threads. 279 | 280 | Example (lines are reported as taking longer to execute when profiled along 281 | with another thread - although the other thread is not profiled):: 282 | 283 | $ demo/embedded.py 284 | Total duration: 1.00013s 285 | File: demo/embedded.py 286 | File duration: 1.00003s (99.99%) 287 | Line #| Hits| Time| Time per hit| %|Source code 288 | ------+----------+-------------+-------------+-------+----------- 289 | 1| 0| 0| 0| 0.00%|#!/usr/bin/env python 290 | 2| 0| 0| 0| 0.00%|import threading 291 | 3| 0| 0| 0| 0.00%|import pprofile 292 | 4| 0| 0| 0| 0.00%|import time 293 | 5| 0| 0| 0| 0.00%|import sys 294 | 6| 0| 0| 0| 0.00%| 295 | 7| 1| 1.5974e-05| 1.5974e-05| 0.00%|def func(): 296 | 8| 0| 0| 0| 0.00%| # Busy loop, so context switches happen 297 | 9| 1| 1.40667e-05| 1.40667e-05| 0.00%| end = time.time() + 1 298 | 10| 146604| 0.511392| 3.48826e-06| 51.13%| while time.time() < end: 299 | 11| 146603| 0.48861| 3.33288e-06| 48.85%| pass 300 | 12| 0| 0| 0| 0.00%| 301 | 13| 0| 0| 0| 0.00%|# Single-treaded run 302 | 14| 0| 0| 0| 0.00%|prof = pprofile.Profile() 303 | 15| 0| 0| 0| 0.00%|with prof: 304 | 16| 0| 0| 0| 0.00%| func() 305 | (call)| 1| 1.00003| 1.00003| 99.99%|# ./demo/embedded.py:7 func 306 | 17| 0| 0| 0| 0.00%|prof.annotate(sys.stdout, __file__) 307 | 18| 0| 0| 0| 0.00%| 308 | 19| 0| 0| 0| 0.00%|# Dual-threaded run 309 | 20| 0| 0| 0| 0.00%|t1 = threading.Thread(target=func) 310 | 21| 0| 0| 0| 0.00%|prof = pprofile.Profile() 311 | 22| 0| 0| 0| 0.00%|with prof: 312 | 23| 0| 0| 0| 0.00%| t1.start() 313 | 24| 0| 0| 0| 0.00%| func() 314 | 25| 0| 0| 0| 0.00%| t1.join() 315 | 26| 0| 0| 0| 0.00%|prof.annotate(sys.stdout, __file__) 316 | Total duration: 1.00129s 317 | File: demo/embedded.py 318 | File duration: 1.00004s (99.88%) 319 | Line #| Hits| Time| Time per hit| %|Source code 320 | ------+----------+-------------+-------------+-------+----------- 321 | [...] 322 | 7| 1| 1.50204e-05| 1.50204e-05| 0.00%|def func(): 323 | 8| 0| 0| 0| 0.00%| # Busy loop, so context switches happen 324 | 9| 1| 2.38419e-05| 2.38419e-05| 0.00%| end = time.time() + 1 325 | 10| 64598| 0.538571| 8.33728e-06| 53.79%| while time.time() < end: 326 | 11| 64597| 0.461432| 7.14324e-06| 46.08%| pass 327 | [...] 328 | 329 | This also means that the sum of the percentage of all lines can exceed 100%. It 330 | can reach the number of concurrent threads (200% with 2 threads being busy for 331 | the whole profiled execution time, etc). 332 | 333 | Example with 3 threads:: 334 | 335 | $ pprofile demo/threads.py 336 | Command line: ['demo/threads.py'] 337 | Total duration: 1.00798s 338 | File: demo/threads.py 339 | File duration: 3.00604s (298.22%) 340 | Line #| Hits| Time| Time per hit| %|Source code 341 | ------+----------+-------------+-------------+-------+----------- 342 | 1| 2| 3.21865e-05| 1.60933e-05| 0.00%|import threading 343 | 2| 1| 6.91414e-06| 6.91414e-06| 0.00%|import time 344 | 3| 0| 0| 0| 0.00%| 345 | 4| 4| 3.91006e-05| 9.77516e-06| 0.00%|def func(): 346 | 5| 3| 3.00539| 1.0018|298.16%| time.sleep(1) 347 | 6| 0| 0| 0| 0.00%| 348 | 7| 2| 2.31266e-05| 1.15633e-05| 0.00%|def func2(): 349 | 8| 1| 2.38419e-05| 2.38419e-05| 0.00%| pass 350 | 9| 0| 0| 0| 0.00%| 351 | 10| 1| 1.81198e-05| 1.81198e-05| 0.00%|t1 = threading.Thread(target=func) 352 | (call)| 1| 0.000612974| 0.000612974| 0.06%|# /usr/lib/python2.7/threading.py:436 __init__ 353 | 11| 1| 1.57356e-05| 1.57356e-05| 0.00%|t2 = threading.Thread(target=func) 354 | (call)| 1| 0.000438213| 0.000438213| 0.04%|# /usr/lib/python2.7/threading.py:436 __init__ 355 | 12| 1| 6.60419e-05| 6.60419e-05| 0.01%|t1.start() 356 | (call)| 1| 0.000913858| 0.000913858| 0.09%|# /usr/lib/python2.7/threading.py:485 start 357 | 13| 1| 6.8903e-05| 6.8903e-05| 0.01%|t2.start() 358 | (call)| 1| 0.00167513| 0.00167513| 0.17%|# /usr/lib/python2.7/threading.py:485 start 359 | 14| 1| 0.000200272| 0.000200272| 0.02%|(func(), func2()) 360 | (call)| 1| 1.00274| 1.00274| 99.48%|# demo/threads.py:4 func 361 | (call)| 1| 4.19617e-05| 4.19617e-05| 0.00%|# demo/threads.py:7 func2 362 | 15| 1| 9.58443e-05| 9.58443e-05| 0.01%|t1.join() 363 | (call)| 1| 0.000411987| 0.000411987| 0.04%|# /usr/lib/python2.7/threading.py:653 join 364 | 16| 1| 5.29289e-05| 5.29289e-05| 0.01%|t2.join() 365 | (call)| 1| 0.000316143| 0.000316143| 0.03%|# /usr/lib/python2.7/threading.py:653 join 366 | 367 | Note that the call time is not added to file total: it's already accounted 368 | for inside "func". 369 | 370 | Why another profiler ? 371 | ====================== 372 | 373 | Python's standard profiling tools have a callable-level granularity, which 374 | means it is only possible to tell which function is a hot-spot, not which 375 | lines in that function. 376 | 377 | Robert Kern's line_profiler_ is a very nice alternative providing line-level 378 | profiling granularity, but in my opinion it has a few drawbacks which (in 379 | addition to the attractive technical challenge) made me start pprofile: 380 | 381 | - It is not pure-python. This choice makes sense for performance 382 | but makes usage with pypy difficult and requires installation (I value 383 | execution straight from checkout). 384 | 385 | - It requires source code modification to select what should be profiled. 386 | I prefer to have the option to do an in-depth, non-intrusive profiling. 387 | 388 | - As an effect of previous point, it does not have a notion above individual 389 | callable, annotating functions but not whole files - preventing module 390 | import profiling. 391 | 392 | - Profiling recursive code provides unexpected results (recursion cost is 393 | accumulated on callable's first line) because it doesn't track call stack. 394 | This may be unintended, and may be fixed at some point in line_profiler. 395 | 396 | .. _line_profiler: https://github.com/rkern/line_profiler 397 | .. _`Callgrind Profile Format`: http://valgrind.org/docs/manual/cl-format.html 398 | .. _kcachegrind: http://kcachegrind.sourceforge.net 399 | .. _qcachegrind: http://sourceforge.net/projects/qcachegrindwin/ 400 | -------------------------------------------------------------------------------- /demo/embedded.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | import threading 3 | import pprofile 4 | import time 5 | import sys 6 | 7 | def func(): 8 | # Busy loop, so context switches happen 9 | end = time.time() + 1 10 | while time.time() < end: 11 | pass 12 | 13 | # Single-treaded run 14 | prof = pprofile.Profile() 15 | with prof: 16 | func() 17 | prof.annotate(sys.stdout, __file__) 18 | 19 | # Dual-threaded run 20 | t1 = threading.Thread(target=func) 21 | prof = pprofile.Profile() 22 | with prof: 23 | t1.start() 24 | func() 25 | t1.join() 26 | prof.annotate(sys.stdout, __file__) 27 | -------------------------------------------------------------------------------- /demo/empty.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/vpelletier/pprofile/14ef3a8cb2670451d09fc45c61f72ac35620a8ee/demo/empty.py -------------------------------------------------------------------------------- /demo/encoding.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | import iso_8859_1 3 | import utf_8 4 | import utf_8_bom 5 | -------------------------------------------------------------------------------- /demo/exceptions.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | 3 | def trigger(): 4 | raise Exception 5 | 6 | def indirect(): 7 | trigger() 8 | 9 | # Caught exception 10 | try: 11 | raise Exception 12 | except Exception: 13 | pass 14 | 15 | # Caught exception, from function 16 | try: 17 | trigger() 18 | except Exception: 19 | pass 20 | 21 | # Caught exception, from deeper function 22 | try: 23 | indirect() 24 | except Exception: 25 | pass 26 | 27 | # Uncaught exception, from function 28 | try: 29 | trigger() 30 | finally: 31 | pass 32 | 33 | print 'Never reached' 34 | -------------------------------------------------------------------------------- /demo/iso_8859_1.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/vpelletier/pprofile/14ef3a8cb2670451d09fc45c61f72ac35620a8ee/demo/iso_8859_1.py -------------------------------------------------------------------------------- /demo/module_globals.py: -------------------------------------------------------------------------------- 1 | # All these globals must be defined. 2 | __builtins__ 3 | __file__ 4 | __name__ 5 | __package__ 6 | -------------------------------------------------------------------------------- /demo/recurse.py: -------------------------------------------------------------------------------- 1 | from time import sleep 2 | MAX_LEVEL = 10 3 | def foo(level=0): 4 | if level < MAX_LEVEL: 5 | foo(level + 1) 6 | sleep(0.01) 7 | foo() 8 | -------------------------------------------------------------------------------- /demo/recurse2.py: -------------------------------------------------------------------------------- 1 | from time import sleep 2 | MAX_LEVEL = 10 3 | def boo(level=0): 4 | if level < MAX_LEVEL: 5 | baz(level + 1) 6 | sleep(0.01) 7 | def baz(level): 8 | boo(level) 9 | boo() 10 | -------------------------------------------------------------------------------- /demo/recurse3.py: -------------------------------------------------------------------------------- 1 | from time import sleep 2 | MAX_LEVEL = 5 3 | def bar(level=0): 4 | if level < MAX_LEVEL: 5 | bar(level + 1) 6 | bar(level + 1) 7 | sleep(0.01) 8 | bar() 9 | -------------------------------------------------------------------------------- /demo/recurse4.py: -------------------------------------------------------------------------------- 1 | from time import sleep 2 | MAX_LEVEL = 5 3 | def bar(level=0): 4 | if level < MAX_LEVEL: 5 | bar(level + 1) 6 | bar(level + 1) 7 | bar(level + 1) 8 | sleep(0.01) 9 | bar() 10 | -------------------------------------------------------------------------------- /demo/the_main.py: -------------------------------------------------------------------------------- 1 | """ 2 | some docstring 3 | """ 4 | a_global_from_main = 'foo' 5 | import the_other 6 | -------------------------------------------------------------------------------- /demo/the_other.py: -------------------------------------------------------------------------------- 1 | from __main__ import * 2 | # Must not raise 3 | print(a_global_from_main) 4 | -------------------------------------------------------------------------------- /demo/threads.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | import threading 3 | import time 4 | 5 | def func(): 6 | time.sleep(1) 7 | 8 | def func2(): 9 | pass 10 | 11 | t1 = threading.Thread(target=func) 12 | t2 = threading.Thread(target=func) 13 | t1.start() 14 | t2.start() 15 | (func(), func2()) 16 | t1.join() 17 | t2.join() 18 | -------------------------------------------------------------------------------- /demo/twocalls.py: -------------------------------------------------------------------------------- 1 | from time import sleep 2 | def bar(): 3 | sleep(0.1) 4 | def baz(): 5 | sleep(0.1) 6 | def foo(): 7 | bar() 8 | baz() 9 | foo() 10 | -------------------------------------------------------------------------------- /demo/twocalls2.py: -------------------------------------------------------------------------------- 1 | from time import sleep 2 | def bar(): 3 | sleep(0.1) 4 | def foo(): 5 | bar() 6 | bar() 7 | foo() 8 | -------------------------------------------------------------------------------- /demo/utf_8.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | # -*- coding: UTF-8 -*- 3 | # This is an utf-8 "e" with acute accent: é 4 | -------------------------------------------------------------------------------- /demo/utf_8_bom.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | # This is an utf-8 "e" with acute accent: é 3 | -------------------------------------------------------------------------------- /pprofile/__init__.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | # Copyright (C) 2013-2024 Vincent Pelletier 3 | # 4 | # This program is free software; you can redistribute it and/or 5 | # modify it under the terms of the GNU General Public License 6 | # as published by the Free Software Foundation; either version 2 7 | # of the License, or (at your option) any later version. 8 | # 9 | # This program is distributed in the hope that it will be useful, 10 | # but WITHOUT ANY WARRANTY; without even the implied warranty of 11 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 12 | # GNU General Public License for more details. 13 | # 14 | # You should have received a copy of the GNU General Public License 15 | # along with this program; if not, write to the Free Software 16 | # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA. 17 | """ 18 | pprofile - Line-granularity, thread-aware deterministic and statistic 19 | pure-python profiler 20 | 21 | Usage as a command line: 22 | $ pprofile --exclude-syspath some_python_executable arg1 ... 23 | $ pprofile --exclude-syspath -m some_python_module -- arg1 ... 24 | $ python -m pprofile --exclude-syspath some_python_executable arg1 ... 25 | $ python -m pprofile -m some_python_module -- arg1 ... 26 | See --help for all options. 27 | 28 | Usage as a python module: 29 | 30 | Deterministic profiling: 31 | >>> prof = pprofile.Profile() 32 | >>> with prof(): 33 | >>> # Code to profile 34 | >>> prof.print_stats() 35 | 36 | Statistic profiling: 37 | >>> prof = StatisticalProfile() 38 | >>> with prof(): 39 | >>> # Code to profile 40 | >>> prof.print_stats() 41 | """ 42 | from __future__ import print_function, division, absolute_import 43 | from collections import defaultdict, deque 44 | from email.mime.multipart import MIMEMultipart 45 | from email.mime.text import MIMEText 46 | from email.mime.application import MIMEApplication 47 | from email.encoders import encode_quopri 48 | from functools import partial, wraps 49 | # Note: use time, not clock. 50 | # Clock, at least on linux, ignores time not spent executing code 51 | # (ex: time.sleep()). The goal of pprofile is not to profile python code 52 | # execution as such (ie, to improve python interpreter), but to profile a 53 | # possibly complex application, with its (IO) waits, sleeps, (...) so a 54 | # developper can understand what is slow rather than what keeps the cpu busy. 55 | # So using the wall-clock as a way to measure time spent is more meaningful. 56 | # XXX: This said, if time() lacks precision, a better but likely 57 | # platform-dependent wall-clock time source must be identified and used. 58 | from time import time 59 | from warnings import warn 60 | import argparse 61 | import io 62 | import inspect 63 | from itertools import count 64 | import linecache 65 | import os 66 | # not caught by 2to3, likely because pipes.quote is not documented in python 2 67 | try: 68 | from pipes import quote as shlex_quote # Python 2 69 | except ImportError: 70 | from shlex import quote as shlex_quote # Python 3 71 | import platform 72 | import re 73 | import runpy 74 | import shlex 75 | from subprocess import list2cmdline as windows_list2cmdline 76 | import sys 77 | import threading 78 | import zipfile 79 | try: 80 | from IPython.core.magic import register_line_cell_magic 81 | except ImportError: 82 | register_line_cell_magic = lambda x: x 83 | 84 | __all__ = ( 85 | 'ProfileBase', 'ProfileRunnerBase', 'Profile', 'ThreadProfile', 86 | 'StatisticProfile', 'StatisticThread', 'run', 'runctx', 'runfile', 87 | 'runpath', 88 | ) 89 | class BaseLineIterator(object): 90 | def __init__(self, getline, filename, global_dict): 91 | self._getline = getline 92 | self._filename = filename 93 | self._global_dict = global_dict 94 | self._lineno = 1 95 | 96 | def __iter__(self): 97 | return self 98 | 99 | def __next__(self): 100 | lineno = self._lineno 101 | self._lineno += 1 102 | return lineno, self._getline(self._filename, lineno, self._global_dict) 103 | 104 | next = __next__ # BBB 2.7 105 | 106 | if sys.version_info < (3, ): 107 | import codecs 108 | # Find coding specification (see PEP-0263) 109 | _matchCoding = re.compile( 110 | r'^[ \t\f]*#.*?coding[:=][ \t]*([-_.a-zA-Z0-9]+)', 111 | ).match 112 | class LineIterator(BaseLineIterator): 113 | _encoding = None 114 | 115 | def __init__(self, *args, **kw): 116 | super(LineIterator, self).__init__(*args, **kw) 117 | # Identify encoding. 118 | first_line = self._getline(self._filename, 1, self._global_dict) 119 | if isinstance(first_line, bytes): 120 | # BOM - python2 only detects the (discouraged) UTF-8 BOM 121 | if first_line.startswith(codecs.BOM_UTF8): 122 | self._encoding = 'utf-8' 123 | else: 124 | # PEP-0263: "the first or second line must match [_matchCoding]" 125 | match = _matchCoding(first_line) 126 | if match is None: 127 | match = _matchCoding( 128 | self._getline(self._filename, 2, self._global_dict), 129 | ) 130 | if match is None: 131 | self._encoding = 'ascii' 132 | else: 133 | self._encoding = match.group(1) 134 | # else, first line is unicode. 135 | 136 | def __next__(self): 137 | lineno, line = super(LineIterator, self).__next__() 138 | if self._encoding: 139 | line = line.decode(self._encoding, errors='replace') 140 | return lineno, line 141 | 142 | next = __next__ # BBB 143 | 144 | def iterframes(current_frames): 145 | return current_frames().iteritems() 146 | else: 147 | # getline returns unicode objects, nothing to do 148 | LineIterator = BaseLineIterator 149 | unicode = basestring = str 150 | 151 | def iterframes(current_frames): 152 | return current_frames().items() 153 | 154 | if platform.system() == 'Windows': 155 | quoteCommandline = windows_list2cmdline 156 | else: 157 | def quoteCommandline(commandline): 158 | return ' '.join(shlex_quote(x) for x in commandline) 159 | 160 | class EncodeOrReplaceWriter(object): 161 | """ 162 | Write-only file-ish object which replaces unsupported chars when 163 | underlying file rejects them. 164 | """ 165 | def __init__(self, out): 166 | self._encoding = getattr(out, 'encoding', None) or 'ascii' 167 | self._write = out.write 168 | 169 | def write(self, data): 170 | try: 171 | self._write(data) 172 | except UnicodeEncodeError: 173 | self._write( 174 | data.encode( 175 | self._encoding, 176 | errors='replace', 177 | ).decode(self._encoding), 178 | ) 179 | 180 | def _isCallgrindName(filepath): 181 | return os.path.basename(filepath).startswith('cachegrind.out.') 182 | 183 | class _FileTiming(object): 184 | """ 185 | Accumulation of profiling statistics (line and call durations) for a given 186 | source "file" (unique global dict). 187 | 188 | Subclasses should be aware that: 189 | - this classes uses __slots__, mainly for cpu efficiency (property lookup 190 | is in a list instead of a dict) 191 | - it can access the BaseProfile instance which created any instace using 192 | the "profiler" property, should they share some state across source 193 | files. 194 | - methods on this class are profiling choke-point - keep customisations 195 | as cheap in CPU as you can ! 196 | """ 197 | __slots__ = ('line_dict', 'call_dict', 'filename', 'global_dict', 198 | 'profiler') 199 | def __init__(self, filename, global_dict, profiler): 200 | self.filename = filename 201 | self.global_dict = global_dict 202 | self.line_dict = defaultdict(lambda: defaultdict(lambda: [0, 0])) 203 | self.call_dict = {} 204 | # Note: not used in this implementation, may be used by subclasses. 205 | self.profiler = profiler 206 | 207 | def hit(self, code, line, duration): 208 | """ 209 | A line has finished executing. 210 | 211 | code (code) 212 | container function's code object 213 | line (int) 214 | line number of just executed line 215 | duration (float) 216 | duration of the line, in seconds 217 | """ 218 | entry = self.line_dict[line][code] 219 | entry[0] += 1 220 | entry[1] += duration 221 | 222 | def call(self, code, line, callee_file_timing, callee, duration, frame): 223 | """ 224 | A call originating from this file returned. 225 | 226 | code (code) 227 | caller's code object 228 | line (int) 229 | caller's line number 230 | callee_file_timing (FileTiming) 231 | callee's FileTiming 232 | callee (code) 233 | callee's code object 234 | duration (float) 235 | duration of the call, in seconds 236 | frame (frame) 237 | calle's entire frame as of its return 238 | """ 239 | try: 240 | entry = self.call_dict[(code, line, callee)] 241 | except KeyError: 242 | self.call_dict[(code, line, callee)] = [callee_file_timing, 1, duration] 243 | else: 244 | entry[1] += 1 245 | entry[2] += duration 246 | 247 | def getHitStatsFor(self, line): 248 | total_hits = total_duration = 0 249 | for hits, duration in self.line_dict.get(line, {}).values(): 250 | total_hits += hits 251 | total_duration += duration 252 | return total_hits, total_duration 253 | 254 | def getLastLine(self): 255 | return max( 256 | max(self.line_dict) if self.line_dict else 0, 257 | max(x for _, x, _ in self.call_dict) if self.call_dict else 0, 258 | ) 259 | 260 | def iterHits(self): 261 | for line, code_dict in self.line_dict.items(): 262 | for code, (hits, duration) in code_dict.items(): 263 | yield line, code, hits, duration 264 | 265 | def iterCalls(self): 266 | for (code, line, callee), (callee_file_timing, hit, duration) in \ 267 | self.call_dict.items(): 268 | yield ( 269 | line, 270 | code, 271 | hit, duration, 272 | callee_file_timing.filename, callee, 273 | ) 274 | 275 | def getCallListByLine(self): 276 | result = defaultdict(list) 277 | for line, code, hit, duration, callee_filename, callee in self.iterCalls(): 278 | result[line].append(( 279 | code, 280 | hit, duration, 281 | callee_filename, callee, 282 | )) 283 | return result 284 | 285 | def getTotalTime(self): 286 | return sum( 287 | y[1] 288 | for x in self.line_dict.values() 289 | for y in x.values() 290 | ) 291 | 292 | def getTotalHitCount(self): 293 | return sum( 294 | y[0] 295 | for x in self.line_dict.values() 296 | for y in x.values() 297 | ) 298 | 299 | def getSortKey(self): 300 | # total duration first, then total hit count for statistical profiling 301 | result = [0, 0] 302 | for entry in self.line_dict.values(): 303 | for hit, duration in entry.values(): 304 | result[0] += duration 305 | result[1] += hit 306 | return result 307 | 308 | FileTiming = _FileTiming 309 | 310 | class LocalDescriptor(threading.local): 311 | """ 312 | Implementation of descriptor API for thread-local properties. 313 | """ 314 | def __init__(self, func=None): 315 | """ 316 | func (callable) 317 | If provided, called when a missing property is accessed 318 | (ex: accessing thread never initialised that property). 319 | If None, AttributeError is raised. 320 | """ 321 | super(LocalDescriptor, self).__init__() 322 | if func is not None: 323 | self.func = func 324 | 325 | def __get__(self, instance, owner): 326 | try: 327 | return getattr(self, str(id(instance))) 328 | except AttributeError: 329 | # Raises AttributeError if func was not provided. 330 | value = self.func() 331 | setattr(self, str(id(instance)), value) 332 | return value 333 | 334 | def __set__(self, instance, value): 335 | setattr(self, str(id(instance)), value) 336 | 337 | def __delete__(self, instance): 338 | try: 339 | delattr(self, str(id(instance))) 340 | except AttributeError: 341 | pass 342 | 343 | _ANNOTATE_HEADER = \ 344 | u'%6s|%10s|' \ 345 | u'%13s|%13s|%7s|' \ 346 | u'Source code' % ( 347 | u'Line #', u'Hits', 348 | u'Time', u'Time per hit', u'%', 349 | ) 350 | _ANNOTATE_HORIZONTAL_LINE = u''.join(x == u'|' and u'+' or u'-' 351 | for x in _ANNOTATE_HEADER) 352 | _ANNOTATE_FORMAT = \ 353 | u'%(lineno)6i|%(hits)10i|' \ 354 | u'%(time)13g|%(time_per_hit)13g|%(percent)6.2f%%|' \ 355 | u'%(line)s' 356 | _ANNOTATE_CALL_FORMAT = \ 357 | u'(call)|%(hits)10i|' \ 358 | u'%(time)13g|%(time_per_hit)13g|%(percent)6.2f%%|' \ 359 | u'# %(callee_file)s:%(callee_line)s %(callee_name)s' 360 | 361 | def _initStack(): 362 | # frame_time: when current frame execution started/resumed last 363 | # frame_discount: time discounted from current frame, because it appeared 364 | # lower in the call stack from the same callsite 365 | # lineno: latest line which execution started 366 | # line_time: time at which latest line started being executed 367 | # line_duration: total time spent in current line up to last resume 368 | now = time() 369 | return (deque([[now, 0, None, now, 0]]), defaultdict(deque)) 370 | 371 | class ProfileBase(object): 372 | """ 373 | Methods common to deterministic and statistic profiling. 374 | 375 | Subclasses can override the "FileTiming" property to use a different class. 376 | """ 377 | __slots__ = ( 378 | 'file_dict', 379 | 'global_dict', 380 | 'total_time', 381 | '__dict__', 382 | '__weakref__', 383 | 'merged_file_dict', 384 | ) 385 | FileTiming = _FileTiming 386 | 387 | def __init__(self): 388 | self.file_dict = {} 389 | self.merged_file_dict = {} 390 | self.global_dict = {} 391 | self.total_time = 0 392 | 393 | def _getFileTiming(self, frame): 394 | try: 395 | return self.global_dict[id(frame.f_globals)] 396 | except KeyError: 397 | f_globals = frame.f_globals 398 | name = self._getFilename(frame) 399 | self.global_dict[id(f_globals)] = file_timing = self.FileTiming( 400 | name, 401 | f_globals, 402 | self, 403 | ) 404 | # file_dict modifications must be thread-safe to not lose measures. 405 | # setdefault is atomic, append is atomic. 406 | self.file_dict.setdefault(name, []).append(file_timing) 407 | return file_timing 408 | 409 | @staticmethod 410 | def _getFilename(frame): 411 | """ 412 | Overload in subclasses to customise filename generation. 413 | """ 414 | return frame.f_code.co_filename 415 | 416 | @staticmethod 417 | def _getline(filename, lineno, global_dict): 418 | """ 419 | Overload in subclasses to customise source retrieval. 420 | """ 421 | return linecache.getline(filename, lineno, global_dict) 422 | 423 | def _mergeFileTiming(self, rebuild=False): 424 | merged_file_dict = self.merged_file_dict 425 | if merged_file_dict and not rebuild: 426 | return merged_file_dict 427 | merged_file_dict.clear() 428 | # Regroup by module, to find all duplicates from other threads. 429 | by_global_dict = defaultdict(list) 430 | for file_timing_list in self.file_dict.values(): 431 | for file_timing in file_timing_list: 432 | by_global_dict[ 433 | id(file_timing.global_dict) 434 | ].append( 435 | file_timing, 436 | ) 437 | # Resolve name conflicts. 438 | global_to_named_dict = {} 439 | for global_dict_id, file_timing_list in by_global_dict.items(): 440 | file_timing = file_timing_list[0] 441 | name = file_timing.filename 442 | if name in merged_file_dict: 443 | counter = count() 444 | base_name = name 445 | while name in merged_file_dict: 446 | name = base_name + '_%i' % next(counter) 447 | global_to_named_dict[global_dict_id] = merged_file_dict[name] = FileTiming( 448 | name, 449 | file_timing.global_dict, 450 | file_timing.profiler, # Note: should be self 451 | ) 452 | # Add all file timings from one module together under its 453 | # deduplicated name. This needs to happen after all names 454 | # are generated and all empty file timings are created so 455 | # call events cross-references can be remapped. 456 | for merged_file_timing in merged_file_dict.values(): 457 | line_dict = merged_file_timing.line_dict 458 | for file_timing in by_global_dict[id(merged_file_timing.global_dict)]: 459 | for line, other_code_dict in file_timing.line_dict.items(): 460 | code_dict = line_dict[line] 461 | for code, ( 462 | other_hits, 463 | other_duration, 464 | ) in other_code_dict.items(): 465 | entry = code_dict[code] 466 | entry[0] += other_hits 467 | entry[1] += other_duration 468 | call_dict = merged_file_timing.call_dict 469 | for key, ( 470 | other_callee_file_timing, 471 | other_hits, 472 | other_duration, 473 | ) in file_timing.call_dict.items(): 474 | try: 475 | entry = call_dict[key] 476 | except KeyError: 477 | entry = call_dict[key] = [ 478 | global_to_named_dict[ 479 | id(other_callee_file_timing.global_dict) 480 | ], 481 | other_hits, 482 | other_duration, 483 | ] 484 | else: 485 | entry[1] += other_hits 486 | entry[2] += other_duration 487 | return merged_file_dict 488 | 489 | def getFilenameSet(self): 490 | """ 491 | Returns a set of profiled file names. 492 | 493 | Note: "file name" is used loosely here. See python documentation for 494 | co_filename, linecache module and PEP302. It may not be a valid 495 | filesystem path. 496 | """ 497 | result = set(self._mergeFileTiming()) 498 | # Ignore profiling code. __file__ does not always provide consistent 499 | # results with f_code.co_filename (ex: easy_install with zipped egg), 500 | # so inspect current frame instead. 501 | # Get current file from one of pprofile methods. Compatible with 502 | # implementations that do not have the inspect.currentframe() method 503 | # (e.g. IronPython). 504 | # XXX: Assumes that all of pprofile code is in a single file. 505 | # XXX: Assumes that _initStack exists in pprofile module. 506 | result.discard(inspect.getsourcefile(_initStack)) 507 | return result 508 | 509 | def _getFileNameList(self, filename, may_sort=True): 510 | if filename is None: 511 | filename = self.getFilenameSet() 512 | elif isinstance(filename, basestring): 513 | return [filename] 514 | if may_sort: 515 | try: 516 | # Detect if filename is an ordered data type. 517 | filename[:0] 518 | except TypeError: 519 | # Not ordered, sort. 520 | file_dict = self._mergeFileTiming() 521 | filename = sorted(filename, reverse=True, 522 | key=lambda x: file_dict[x].getSortKey() 523 | ) 524 | return filename 525 | 526 | def _iterOutFiles(self, filename=None, commandline=None): 527 | """ 528 | Yields path, data, mimetype for each file involved on or produced by 529 | profiling. 530 | """ 531 | out = io.StringIO() 532 | self.callgrind( 533 | out, 534 | filename=filename, 535 | commandline=commandline, 536 | relative_path=True, 537 | ) 538 | yield ( 539 | 'cachegrind.out.pprofile', 540 | out.getvalue(), 541 | 'application/x-kcachegrind', 542 | ) 543 | for name in self._getFileNameList(filename, may_sort=False): 544 | lines = ''.join(self._iterRawFile(name)) 545 | if lines: 546 | if isinstance(lines, unicode): 547 | lines = lines.encode('utf-8') 548 | yield ( 549 | _relpath(name), 550 | lines, 551 | 'text/x-python', 552 | ) 553 | 554 | def getCallgrindMIME(self, out, filename=None, commandline=None, relative_path=True): 555 | """ 556 | Write to "out" a mime-multipart representation of: 557 | - callgrind profiling statistics (cachegrind.out.pprofile) 558 | - all involved python code, including Python Scripts without hierarchy 559 | (the rest) 560 | and return its mimetype. 561 | To unpack resulting file, see "unpack a MIME message" in 562 | http://docs.python.org/2/library/email-examples.html 563 | Or get demultipart from 564 | https://pypi.python.org/pypi/demultipart 565 | 566 | relative_path: 567 | Ignored. 568 | """ 569 | _ = relative_path 570 | result = MIMEMultipart() 571 | base_type_dict = { 572 | 'application': MIMEApplication, 573 | 'text': MIMEText, 574 | } 575 | encoder_dict = { 576 | 'application/x-kcachegrind': encode_quopri, 577 | 'text/x-python': 'utf-8', 578 | 'text/plain': 'utf-8', 579 | } 580 | for path, data, mimetype in self._iterOutFiles( 581 | filename=filename, 582 | commandline=commandline, 583 | ): 584 | base_type, sub_type = mimetype.split('/') 585 | chunk = base_type_dict[base_type]( 586 | data, 587 | sub_type, 588 | encoder_dict.get(mimetype), 589 | ) 590 | chunk.add_header( 591 | 'Content-Disposition', 592 | 'attachment', 593 | filename=path, 594 | ) 595 | result.attach(chunk) 596 | out.write(result.as_string()) 597 | return result['content-type'] 598 | 599 | def getCallgrindZip(self, out=None, filename=None, commandline=None, relative_path=True): 600 | """ 601 | Write to "out" a serialised zip archive containing: 602 | - callgrind profiling statistics (cachegrind.out.pprofile) 603 | - all involved python code, including Python Scripts without hierarchy 604 | (the rest) 605 | as a byte string and the "application/zip" mimetype. 606 | 607 | relative_path: 608 | Ignored. 609 | """ 610 | _ = relative_path 611 | with zipfile.ZipFile( 612 | out, 613 | mode='w', 614 | compression=zipfile.ZIP_DEFLATED, 615 | ) as outfile: 616 | for path, data, _ in self._iterOutFiles( 617 | filename=filename, 618 | commandline=commandline, 619 | ): 620 | outfile.writestr(path, data) 621 | return 'application/zip' 622 | 623 | def callgrind(self, out, filename=None, commandline=None, relative_path=False): 624 | """ 625 | Dump statistics in callgrind format. 626 | Contains: 627 | - per-line hit count, time and time-per-hit 628 | - call associations (call tree) 629 | Note: hit count is not inclusive, in that it is not the sum of all 630 | hits inside that call. 631 | Time unit: microsecond (1e-6 second). 632 | out (file-ish opened for writing) 633 | Destination of callgrind profiling data. 634 | Encoding should be chosen to be able to represent characters in 635 | filesystem path and (when provided) commandline. 636 | Setting an encoding error handler other than "raise" (default) 637 | will likely result in profiling results being unable to locate 638 | code for annotation. 639 | filename (str, collection of str) 640 | If provided, dump stats for given source file(s) only. 641 | By default, list for all known files. 642 | commandline (anything with __str__) 643 | If provided, will be output as the command line used to generate 644 | this profiling data. 645 | relative_path (bool) 646 | When True, absolute elements are stripped from path. Useful when 647 | maintaining several copies of source trees with their own 648 | profiling result, so kcachegrind does not look in system-wide 649 | files which may not match with profiled code. 650 | """ 651 | print(u'# callgrind format', file=out) 652 | print(u'version: 1', file=out) 653 | print(u'creator: pprofile', file=out) 654 | print(u'event: usphit :microseconds/hit', file=out) 655 | print(u'events: hits microseconds usphit', file=out) 656 | if commandline is not None: 657 | print(u'cmd:', commandline, file=out) 658 | file_dict = self._mergeFileTiming() 659 | if relative_path: 660 | convertPath = _relpath 661 | else: 662 | convertPath = lambda x: x 663 | if os.path.sep != "/": 664 | # qCacheGrind (windows build) needs at least one UNIX separator 665 | # in path to find the file. Adapt here even if this is probably 666 | # more of a qCacheGrind issue... 667 | convertPath = lambda x, cascade=convertPath: cascade( 668 | '/'.join(x.split(os.path.sep)) 669 | ) 670 | code_to_name_dict = {} 671 | homonym_counter = {} 672 | def getCodeName(filename, code): 673 | # Tracks code objects globally, because callee information needs 674 | # to be consistent accross files. 675 | # Inside a file, grants unique names to each code object. 676 | try: 677 | return code_to_name_dict[code] 678 | except KeyError: 679 | name = code.co_name + ':%i' % code.co_firstlineno 680 | key = (filename, name) 681 | homonym_count = homonym_counter.get(key, 0) 682 | if homonym_count: 683 | name += '_%i' % homonym_count 684 | homonym_counter[key] = homonym_count + 1 685 | code_to_name_dict[code] = name 686 | return name 687 | for current_file in self._getFileNameList(filename, may_sort=False): 688 | file_timing = file_dict[current_file] 689 | print(u'fl=%s' % convertPath(current_file), file=out) 690 | # When a local callable is created an immediately executed, this 691 | # loop would start a new "fn=" section but would not end it before 692 | # emitting "cfn=" lines, making the callee appear as not being 693 | # called by interrupted "fn=" section. 694 | # So dispatch all functions in a first pass, and build 695 | # uninterrupted sections in a second pass. 696 | # Note: cost line is a list just to be mutable. A single item is 697 | # expected. 698 | func_dict = defaultdict(lambda: defaultdict(lambda: ([], []))) 699 | for lineno, code, hits, duration in file_timing.iterHits(): 700 | func_dict[getCodeName(current_file, code)][lineno][0].append( 701 | (hits, int(duration * 1000000)), 702 | ) 703 | for ( 704 | lineno, 705 | caller, 706 | call_hits, call_duration, 707 | callee_file, callee, 708 | ) in file_timing.iterCalls(): 709 | call_ticks = int(call_duration * 1000000) 710 | func_call_list = func_dict[ 711 | getCodeName(current_file, caller) 712 | ][lineno][1] 713 | append = func_call_list.append 714 | append(u'cfl=' + convertPath(callee_file)) 715 | append(u'cfn=' + getCodeName(callee_file, callee)) 716 | append(u'calls=%i %i' % (call_hits, callee.co_firstlineno)) 717 | append(u'%i %i %i %i' % (lineno, call_hits, call_ticks, call_ticks // call_hits)) 718 | for func_name, line_dict in func_dict.items(): 719 | print(u'fn=%s' % func_name, file=out) 720 | for lineno, (func_hit_list, func_call_list) in sorted(line_dict.items()): 721 | if func_hit_list: 722 | # Multiple function objects may "reside" on the same 723 | # line of the same file (same global dict). 724 | # Sum these up and produce a single cachegrind event. 725 | hits = sum(x for x, _ in func_hit_list) 726 | ticks = sum(x for _, x in func_hit_list) 727 | print( 728 | u'%i %i %i %i' % ( 729 | lineno, 730 | hits, 731 | ticks, 732 | ticks // hits, 733 | ), 734 | file=out, 735 | ) 736 | for line in func_call_list: 737 | print(line, file=out) 738 | 739 | def annotate(self, out, filename=None, commandline=None, relative_path=False): 740 | """ 741 | Dump annotated source code with current profiling statistics to "out" 742 | file. 743 | Time unit: second. 744 | out (file-ish opened for writing) 745 | Destination of annotated sources. 746 | Encoding and encoding error handling should be chosen so as to be 747 | able to represent all characters present in source code (ex: utf-8, 748 | or ascii with "replace" error handler). 749 | filename (str, collection of str) 750 | If provided, dump stats for given source file(s) only. 751 | If unordered collection, it will get sorted by decreasing total 752 | file score (total time if available, then total hit count). 753 | By default, list for all known files. 754 | commandline (anything with __str__) 755 | If provided, will be output as the command line used to generate 756 | this annotation. 757 | relative_path (bool) 758 | For compatibility with callgrind. Ignored. 759 | """ 760 | file_dict = self._mergeFileTiming() 761 | total_time = self.total_time 762 | if commandline is not None: 763 | print(u'Command line:', commandline, file=out) 764 | print(u'Total duration: %gs' % total_time, file=out) 765 | if not total_time: 766 | return 767 | def percent(value, scale): 768 | if scale == 0: 769 | return 0 770 | return value * 100 / scale 771 | for name in self._getFileNameList(filename): 772 | file_timing = file_dict[name] 773 | file_total_time = file_timing.getTotalTime() 774 | call_list_by_line = file_timing.getCallListByLine() 775 | print(u'File: %s' % name, file=out) 776 | print(u'File duration: %gs (%.2f%%)' % (file_total_time, 777 | percent(file_total_time, total_time)), file=out) 778 | print(_ANNOTATE_HEADER, file=out) 779 | print(_ANNOTATE_HORIZONTAL_LINE, file=out) 780 | last_line = file_timing.getLastLine() 781 | for lineno, line in LineIterator( 782 | self._getline, 783 | file_timing.filename, 784 | file_timing.global_dict, 785 | ): 786 | if not line and lineno > last_line: 787 | break 788 | hits, duration = file_timing.getHitStatsFor(lineno) 789 | print(_ANNOTATE_FORMAT % { 790 | u'lineno': lineno, 791 | u'hits': hits, 792 | u'time': duration, 793 | u'time_per_hit': duration / hits if hits else 0, 794 | u'percent': percent(duration, total_time), 795 | u'line': (line or u'').rstrip(), 796 | }, file=out) 797 | for ( 798 | _, 799 | call_hits, call_duration, 800 | callee_file, callee, 801 | ) in call_list_by_line.get(lineno, ()): 802 | print(_ANNOTATE_CALL_FORMAT % { 803 | u'hits': call_hits, 804 | u'time': call_duration, 805 | u'time_per_hit': call_duration / call_hits, 806 | u'percent': percent(call_duration, total_time), 807 | u'callee_file': callee_file, 808 | u'callee_line': callee.co_firstlineno, 809 | u'callee_name': callee.co_name, 810 | }, file=out) 811 | 812 | def _iterRawFile(self, name): 813 | file_timing = self._mergeFileTiming()[name] 814 | for lineno in count(1): 815 | line = self._getline(file_timing.filename, lineno, 816 | file_timing.global_dict) 817 | if not line: 818 | break 819 | yield line 820 | 821 | def iterSource(self): 822 | """ 823 | Iterator over all involved files. 824 | Yields 2-tuple composed of file path and an iterator over 825 | (non-annotated) source lines. 826 | 827 | Can be used to generate a file tree for use with kcachegrind, for 828 | example. 829 | """ 830 | for name in self.getFilenameSet(): 831 | yield name, self._iterRawFile(name) 832 | 833 | # profile/cProfile-like API 834 | def dump_stats(self, filename): 835 | """ 836 | Similar to profile.Profile.dump_stats - but different output format ! 837 | """ 838 | if _isCallgrindName(filename): 839 | with open(filename, 'w') as out: 840 | self.callgrind(out) 841 | else: 842 | with io.open(filename, 'w', errors='replace') as out: 843 | self.annotate(out) 844 | 845 | def print_stats(self): 846 | """ 847 | Similar to profile.Profile.print_stats . 848 | Returns None. 849 | """ 850 | self.annotate(EncodeOrReplaceWriter(sys.stdout)) 851 | 852 | class ProfileRunnerBase(object): 853 | def __call__(self): 854 | return self 855 | 856 | def __enter__(self): 857 | raise NotImplementedError 858 | 859 | def __exit__(self, exc_type, exc_val, exc_tb): 860 | raise NotImplementedError 861 | 862 | # profile/cProfile-like API 863 | def runctx(self, cmd, globals, locals): 864 | """Similar to profile.Profile.runctx .""" 865 | with self(): 866 | exec(cmd, globals, locals) 867 | return self 868 | 869 | def runcall(self, func, *args, **kw): 870 | """Similar to profile.Profile.runcall .""" 871 | with self(): 872 | return func(*args, **kw) 873 | 874 | def runfile(self, fd, argv, fd_name='', compile_flags=0, 875 | dont_inherit=1, globals={}): 876 | with fd: 877 | code = compile(fd.read(), fd_name, 'exec', flags=compile_flags, 878 | dont_inherit=dont_inherit) 879 | original_sys_argv = list(sys.argv) 880 | for name, docstring in zip(code.co_names, code.co_consts): 881 | if name == '__doc__': 882 | break 883 | else: 884 | docstring = None 885 | original_main = sys.modules.get('__main__') 886 | # XXX: is there a better way to get hold of module type ? 887 | code_module = type(original_main)('__main__', docstring) 888 | ctx_globals = code_module.__dict__ 889 | ctx_globals.update(globals) 890 | ctx_globals['__builtins__'] = __builtins__ 891 | ctx_globals['__file__'] = fd_name 892 | ctx_globals['__name__'] = '__main__' 893 | ctx_globals['__package__'] = None 894 | sys.modules['__main__'] = code_module 895 | try: 896 | sys.argv[:] = argv 897 | return self.runctx(code, ctx_globals, None) 898 | finally: 899 | sys.argv[:] = original_sys_argv 900 | sys.modules['__main__'] = original_main 901 | 902 | def runpath(self, path, argv): 903 | original_sys_path = list(sys.path) 904 | try: 905 | sys.path.insert(0, os.path.dirname(path)) 906 | return self.runfile(open(path, 'rb'), argv, fd_name=path) 907 | finally: 908 | sys.path[:] = original_sys_path 909 | 910 | def runmodule(self, module, argv): 911 | original_sys_argv = list(sys.argv) 912 | original_sys_path0 = sys.path[0] 913 | try: 914 | sys.path[0] = os.getcwd() 915 | sys.argv[:] = argv 916 | with self(): 917 | runpy.run_module(module, run_name='__main__', alter_sys=True) 918 | finally: 919 | sys.argv[:] = original_sys_argv 920 | sys.path[0] = original_sys_path0 921 | return self 922 | 923 | class Profile(ProfileBase, ProfileRunnerBase): 924 | """ 925 | Deterministic, recursive, line-granularity, profiling class. 926 | 927 | Does not require any source code change to work. 928 | If the performance hit is too large, it can benefit from some 929 | integration (calling enable/disable around selected code chunks). 930 | 931 | The sum of time spent in all profiled lines is less than the total 932 | profiled time reported. This is (part of) profiling overhead. 933 | This also mans that sum of time-spent-on-line percentage is less than 100%. 934 | 935 | All times are "internal time", ie they do not count time spent inside 936 | called (profilable, so python) functions. 937 | """ 938 | __slots__ = ( 939 | '_global_trace', 940 | '_local_trace', 941 | 'stack', 942 | 'enabled_start', 943 | ) 944 | 945 | def __init__(self, verbose=False): 946 | super(Profile, self).__init__() 947 | if verbose: 948 | def decorator(func): 949 | @wraps(func) 950 | def wrapper(frame, event, arg, _traceEvent=self._traceEvent): 951 | _traceEvent(frame, event) 952 | return func(frame, event, arg) 953 | return wrapper 954 | self._global_trace = decorator(self._real_global_trace) 955 | self._local_trace = decorator(self._real_local_trace) 956 | else: 957 | self._global_trace = self._real_global_trace 958 | self._local_trace = self._real_local_trace 959 | self.stack = None 960 | self.enabled_start = None 961 | 962 | def _enable(self): 963 | """ 964 | Overload this method when subclassing. Called before actually 965 | enabling trace. 966 | """ 967 | self.stack = _initStack() 968 | self.enabled_start = time() 969 | 970 | def enable(self): 971 | """ 972 | Enable profiling. 973 | """ 974 | if self.enabled_start: 975 | warn('Duplicate "enable" call') 976 | else: 977 | self._enable() 978 | sys.settrace(self._global_trace) 979 | 980 | def _disable(self): 981 | """ 982 | Overload this method when subclassing. Called after actually disabling 983 | trace. 984 | """ 985 | self.total_time += time() - self.enabled_start 986 | self.enabled_start = None 987 | self.stack = None 988 | 989 | def disable(self): 990 | """ 991 | Disable profiling. 992 | """ 993 | if self.enabled_start: 994 | sys.settrace(None) 995 | self._disable() 996 | else: 997 | warn('Duplicate "disable" call') 998 | 999 | def __enter__(self): 1000 | """ 1001 | __enter__() -> self 1002 | """ 1003 | self.enable() 1004 | return self 1005 | 1006 | def __exit__(self, exc_type, exc_val, exc_tb): 1007 | """ 1008 | __exit__(*excinfo) -> None. Disables profiling. 1009 | """ 1010 | self.disable() 1011 | 1012 | def _traceEvent(self, frame, event): 1013 | f_code = frame.f_code 1014 | lineno = frame.f_lineno or 0 1015 | print('%10.6f%s%s %s:%s %s+%s' % ( 1016 | time() - self.enabled_start, 1017 | ' ' * len(self.stack[0]), 1018 | event, 1019 | f_code.co_filename, 1020 | lineno, 1021 | f_code.co_name, 1022 | lineno - f_code.co_firstlineno, 1023 | ), file=sys.stderr) 1024 | 1025 | def _real_global_trace(self, frame, event, arg): 1026 | local_trace = self._local_trace 1027 | if local_trace is not None: 1028 | event_time = time() 1029 | callee_entry = [event_time, 0, frame.f_lineno or 0, event_time, 0] 1030 | try: 1031 | stack, callee_dict = self.stack 1032 | except TypeError: 1033 | return None 1034 | try: 1035 | caller_entry = stack[-1] 1036 | except IndexError: 1037 | pass 1038 | else: 1039 | # Suspend caller frame 1040 | frame_time, frame_discount, lineno, line_time, line_duration = caller_entry 1041 | caller_entry[4] = event_time - line_time + line_duration 1042 | callee_dict[(frame.f_back.f_code, frame.f_code)].append(callee_entry) 1043 | stack.append(callee_entry) 1044 | return local_trace 1045 | 1046 | def _real_local_trace(self, frame, event, arg): 1047 | if event == 'line' or event == 'return': 1048 | event_time = time() 1049 | try: 1050 | stack, callee_dict = self.stack 1051 | except TypeError: 1052 | return None 1053 | try: 1054 | stack_entry = stack[-1] 1055 | except IndexError: 1056 | warn('Profiling stack underflow, disabling.') 1057 | self.disable() 1058 | return None 1059 | frame_time, frame_discount, lineno, line_time, line_duration = stack_entry 1060 | file_timing = self._getFileTiming(frame) 1061 | file_timing.hit(frame.f_code, lineno, 1062 | event_time - line_time + line_duration) 1063 | if event == 'line': 1064 | # Start a new line 1065 | stack_entry[2] = frame.f_lineno or 0 1066 | stack_entry[3] = event_time 1067 | stack_entry[4] = 0 1068 | else: 1069 | # 'return' event, is still callee 1070 | # Resume caller frame 1071 | stack.pop() 1072 | stack[-1][3] = event_time 1073 | caller_frame = frame.f_back 1074 | caller_code = caller_frame.f_code 1075 | callee_code = frame.f_code 1076 | callee_entry_list = callee_dict[(caller_code, callee_code)] 1077 | callee_entry_list.pop() 1078 | call_duration = event_time - frame_time 1079 | if callee_entry_list: 1080 | # Callee is also somewhere up the stack, so discount this 1081 | # call duration from it. 1082 | callee_entry_list[-1][1] += call_duration 1083 | self._getFileTiming(caller_frame).call( 1084 | caller_code, caller_frame.f_lineno or 0, 1085 | file_timing, 1086 | callee_code, call_duration - frame_discount, 1087 | frame, 1088 | ) 1089 | return self._local_trace 1090 | 1091 | # profile/cProfile-like API 1092 | def run(self, cmd): 1093 | """Similar to profile.Profile.run .""" 1094 | from . import __main__ 1095 | dikt = __main__.__dict__ 1096 | return self.runctx(cmd, dikt, dikt) 1097 | 1098 | class ThreadProfile(Profile): 1099 | """ 1100 | threading.Thread-aware version of Profile class. 1101 | 1102 | Threads started after enable() call will be profiled. 1103 | After disable() call, threads will need to be switched into and trigger a 1104 | trace event (typically a "line" event) before they can notice the 1105 | disabling. 1106 | """ 1107 | __slots__ = ('_local_trace_backup', ) 1108 | 1109 | stack = LocalDescriptor(_initStack) 1110 | global_dict = LocalDescriptor(dict) 1111 | 1112 | def __init__(self, **kw): 1113 | super(ThreadProfile, self).__init__(**kw) 1114 | self._local_trace_backup = self._local_trace 1115 | 1116 | def _enable(self): 1117 | self._local_trace = self._local_trace_backup 1118 | threading.settrace(self._global_trace) 1119 | super(ThreadProfile, self)._enable() 1120 | 1121 | def _disable(self): 1122 | super(ThreadProfile, self)._disable() 1123 | threading.settrace(None) 1124 | self._local_trace = None 1125 | 1126 | class StatisticProfile(ProfileBase, ProfileRunnerBase): 1127 | """ 1128 | Statistic profiling class. 1129 | 1130 | This class does not gather its own samples by itself. 1131 | Instead, it must be provided with call stacks (as returned by 1132 | sys._getframe() or sys._current_frames()). 1133 | """ 1134 | def sample(self, frame): 1135 | getFileTiming = self._getFileTiming 1136 | called_timing = getFileTiming(frame) 1137 | called_code = frame.f_code 1138 | called_timing.hit(called_code, frame.f_lineno or 0, 0) 1139 | while True: 1140 | caller = frame.f_back 1141 | if caller is None: 1142 | break 1143 | caller_timing = getFileTiming(caller) 1144 | caller_code = caller.f_code 1145 | caller_timing.call(caller_code, caller.f_lineno or 0, 1146 | called_timing, called_code, 0, frame) 1147 | called_timing = caller_timing 1148 | frame = caller 1149 | called_code = caller_code 1150 | 1151 | def __call__(self, period=.001, single=True, group=None, name=None): 1152 | """ 1153 | Instanciate StatisticThread. 1154 | 1155 | >>> s_profile = StatisticProfile() 1156 | >>> with s_profile(single=False): 1157 | >>> # Code to profile 1158 | Is equivalent to: 1159 | >>> s_profile = StatisticProfile() 1160 | >>> s_thread = StatisticThread(profiler=s_profile, single=False) 1161 | >>> with s_thread: 1162 | >>> # Code to profile 1163 | """ 1164 | return StatisticThread( 1165 | profiler=self, period=period, single=single, group=group, 1166 | name=name, 1167 | ) 1168 | 1169 | # BBB 1170 | StatisticalProfile = StatisticProfile 1171 | 1172 | class StatisticThread(threading.Thread, ProfileRunnerBase): 1173 | """ 1174 | Usage in a nutshell: 1175 | with StatisticThread() as profiler_thread: 1176 | # do stuff 1177 | profiler_thread.profiler.print_stats() 1178 | """ 1179 | __slots__ = ( 1180 | '_test', 1181 | '_start_time', 1182 | 'clean_exit', 1183 | ) 1184 | 1185 | def __init__(self, profiler=None, period=.001, single=True, group=None, name=None): 1186 | """ 1187 | profiler (None or StatisticProfile instance) 1188 | Available on instances as the "profiler" read-only property. 1189 | If None, a new profiler instance will be created. 1190 | period (float) 1191 | How many seconds to wait between consecutive samples. 1192 | The smaller, the more profiling overhead, but the faster results 1193 | become meaningful. 1194 | The larger, the less profiling overhead, but requires long profiling 1195 | session to get meaningful results. 1196 | single (bool) 1197 | Profile only the thread which created this instance. 1198 | group, name 1199 | See Python's threading.Thread API. 1200 | """ 1201 | if profiler is None: 1202 | profiler = StatisticProfile() 1203 | if single: 1204 | self._test = lambda x, ident=threading.current_thread().ident: ident == x 1205 | else: 1206 | self._test = None 1207 | super(StatisticThread, self).__init__( 1208 | group=group, 1209 | name=name, 1210 | ) 1211 | self._stop_event = threading.Event() 1212 | self._period = period 1213 | self._profiler = profiler 1214 | profiler.total_time = 0 1215 | self.daemon = True 1216 | self.clean_exit = False 1217 | 1218 | @property 1219 | def profiler(self): 1220 | return self._profiler 1221 | 1222 | def start(self): 1223 | self.clean_exit = False 1224 | self._can_run = True 1225 | self._start_time = time() 1226 | super(StatisticThread, self).start() 1227 | 1228 | def stop(self): 1229 | """ 1230 | Request thread to stop. 1231 | Does not wait for actual termination (use join() method). 1232 | """ 1233 | if self.is_alive(): 1234 | self._can_run = False 1235 | self._stop_event.set() 1236 | self._profiler.total_time += time() - self._start_time 1237 | self._start_time = None 1238 | 1239 | def __enter__(self): 1240 | """ 1241 | __enter__() -> self 1242 | """ 1243 | self.start() 1244 | return self 1245 | 1246 | def __exit__(self, exc_type, exc_val, exc_tb): 1247 | """ 1248 | __exit__(*excinfo) -> None. Stops and joins profiling thread. 1249 | """ 1250 | self.stop() 1251 | self.join() 1252 | 1253 | def run(self): 1254 | current_frames = sys._current_frames 1255 | test = self._test 1256 | if test is None: 1257 | test = lambda x, ident=threading.current_thread().ident: ident != x 1258 | sample = self._profiler.sample 1259 | stop_event = self._stop_event 1260 | wait = partial(stop_event.wait, self._period) 1261 | while self._can_run: 1262 | for ident, frame in iterframes(current_frames=current_frames): 1263 | if test(ident): 1264 | sample(frame) 1265 | frame = None 1266 | wait() 1267 | stop_event.clear() 1268 | self.clean_exit = True 1269 | 1270 | def callgrind(self, *args, **kw): 1271 | warn('deprecated', DeprecationWarning) 1272 | return self._profiler.callgrind(*args, **kw) 1273 | 1274 | def annotate(self, *args, **kw): 1275 | warn('deprecated', DeprecationWarning) 1276 | return self._profiler.annotate(*args, **kw) 1277 | 1278 | def dump_stats(self, *args, **kw): 1279 | warn('deprecated', DeprecationWarning) 1280 | return self._profiler.dump_stats(*args, **kw) 1281 | 1282 | def print_stats(self, *args, **kw): 1283 | warn('deprecated', DeprecationWarning) 1284 | return self._profiler.print_stats(*args, **kw) 1285 | 1286 | def iterSource(self, *args, **kw): 1287 | warn('deprecated', DeprecationWarning) 1288 | return self._profiler.iterSource(*args, **kw) 1289 | 1290 | # BBB 1291 | StatisticalThread = StatisticThread 1292 | 1293 | # profile/cProfile-like API (no sort parameter !) 1294 | def _run(threads, verbose, func_name, filename, *args, **kw): 1295 | if threads: 1296 | klass = ThreadProfile 1297 | else: 1298 | klass = Profile 1299 | prof = klass(verbose=verbose) 1300 | try: 1301 | try: 1302 | getattr(prof, func_name)(*args, **kw) 1303 | except SystemExit: 1304 | pass 1305 | finally: 1306 | if filename is None: 1307 | prof.print_stats() 1308 | else: 1309 | prof.dump_stats(filename) 1310 | 1311 | def run(cmd, filename=None, threads=True, verbose=False): 1312 | """Similar to profile.run .""" 1313 | _run(threads, verbose, 'run', filename, cmd) 1314 | 1315 | def runctx(cmd, globals, locals, filename=None, threads=True, verbose=False): 1316 | """Similar to profile.runctx .""" 1317 | _run(threads, verbose, 'runctx', filename, cmd, globals, locals) 1318 | 1319 | def runfile(fd, argv, fd_name='', compile_flags=0, dont_inherit=1, 1320 | filename=None, threads=True, verbose=False): 1321 | """ 1322 | Run code from given file descriptor with profiling enabled. 1323 | Closes fd before executing contained code. 1324 | """ 1325 | _run(threads, verbose, 'runfile', filename, fd, argv, fd_name, 1326 | compile_flags, dont_inherit) 1327 | 1328 | def runpath(path, argv, filename=None, threads=True, verbose=False): 1329 | """ 1330 | Run code from open-accessible file path with profiling enabled. 1331 | """ 1332 | _run(threads, verbose, 'runpath', filename, path, argv) 1333 | 1334 | _allsep = os.sep + (os.altsep or '') 1335 | 1336 | def _relpath(name): 1337 | """ 1338 | Strip absolute components from path. 1339 | Inspired from zipfile.write(). 1340 | """ 1341 | return os.path.normpath(os.path.splitdrive(name)[1]).lstrip(_allsep) 1342 | 1343 | def main(argv=None, stdin=None): 1344 | if argv is None: 1345 | argv = sys.argv 1346 | format_dict = { 1347 | 'text': 'annotate', 1348 | 'callgrind': 'callgrind', 1349 | 'callgrindzip': 'getCallgrindZip', 1350 | } 1351 | 1352 | parser = argparse.ArgumentParser(argv[0]) 1353 | parser.add_argument('script', help='Python script to execute (optionaly ' 1354 | 'followed by its arguments)', nargs='?') 1355 | parser.add_argument('argv', nargs=argparse.REMAINDER) 1356 | parser.add_argument('-o', '--out', default='-', 1357 | help='Write annotated sources to this file. Defaults to stdout.') 1358 | parser.add_argument('-z', '--zipfile', 1359 | help='Name of a zip file to generate from all involved source files. ' 1360 | 'Useful with callgrind output.') 1361 | parser.add_argument('-t', '--threads', default=1, type=int, help='If ' 1362 | 'non-zero, trace threads spawned by program. Default: %(default)s') 1363 | parser.add_argument('-f', '--format', choices=format_dict, 1364 | help='Format in which output is generated. If not set, auto-detected ' 1365 | 'from filename if provided, falling back to "text".') 1366 | parser.add_argument('-v', '--verbose', action='store_true', 1367 | help='Enable profiler internal tracing output. Cryptic and verbose.') 1368 | parser.add_argument('-s', '--statistic', default=0, type=float, 1369 | help='Use this period for statistic profiling, or use deterministic ' 1370 | 'profiling when 0.') 1371 | parser.add_argument('-m', dest='module', 1372 | help='Searches sys.path for the named module and runs the ' 1373 | 'corresponding .py file as a script. When given, positional arguments ' 1374 | 'become sys.argv[1:]') 1375 | 1376 | group = parser.add_argument_group( 1377 | title='Filtering', 1378 | description='Allows excluding (and re-including) code from ' 1379 | '"file names" matching regular expressions. ' 1380 | '"file name" follows the semantics of python\'s "co_filename": ' 1381 | 'it may be a valid path, of an existing or non-existing file, ' 1382 | 'but it may be some arbitrary string too.' 1383 | ) 1384 | group.add_argument('--exclude-syspath', action='store_true', 1385 | help='Exclude all from default "sys.path". Beware: this will also ' 1386 | 'exclude properly-installed non-standard modules, which may not be ' 1387 | 'what you want.') 1388 | group.add_argument('--exclude', action='append', default=[], 1389 | help='Exclude files whose name starts with any pattern.') 1390 | group.add_argument('--include', action='append', default=[], 1391 | help='Include files whose name would have otherwise excluded. ' 1392 | 'If no exclusion was specified, all paths are excluded first.') 1393 | 1394 | options = parser.parse_args(argv[1:]) 1395 | if options.exclude_syspath: 1396 | options.exclude.extend('^' + re.escape(x) for x in sys.path) 1397 | if options.include and not options.exclude: 1398 | options.exclude.append('') # All-matching regex 1399 | if options.verbose: 1400 | if options.exclude: 1401 | print('Excluding:', file=sys.stderr) 1402 | for regex in options.exclude: 1403 | print('\t' + regex, file=sys.stderr) 1404 | if options.include: 1405 | print('But including:', file=sys.stderr) 1406 | for regex in options.include: 1407 | print('\t' + regex, file=sys.stderr) 1408 | 1409 | if options.module is None: 1410 | if options.script is None: 1411 | parser.error('too few arguments') 1412 | args = [options.script] + options.argv 1413 | runner_method_kw = { 1414 | 'path': args[0], 1415 | 'argv': args, 1416 | } 1417 | runner_method_id = 'runpath' 1418 | elif stdin is not None and options.module == '-': 1419 | # Undocumented way of using -m, used internaly by %%pprofile 1420 | args = [''] 1421 | if options.script is not None: 1422 | args.append(options.script) 1423 | args.extend(options.argv) 1424 | from . import __main__ 1425 | runner_method_kw = { 1426 | 'fd': stdin, 1427 | 'argv': args, 1428 | 'fd_name': '', 1429 | 'globals': __main__.__dict__, 1430 | } 1431 | runner_method_id = 'runfile' 1432 | else: 1433 | args = [options.module] 1434 | if options.script is not None: 1435 | args.append(options.script) 1436 | args.extend(options.argv) 1437 | runner_method_kw = { 1438 | 'module': options.module, 1439 | 'argv': args, 1440 | } 1441 | runner_method_id = 'runmodule' 1442 | if options.format is None: 1443 | if os.path.splitext(options.out)[1] == os.path.extsep + 'zip': 1444 | options.format = 'callgrindzip' 1445 | elif _isCallgrindName(options.out): 1446 | options.format = 'callgrind' 1447 | else: 1448 | options.format = 'text' 1449 | relative_path = options.format == 'callgrind' and options.zipfile 1450 | if options.statistic: 1451 | prof = StatisticalProfile() 1452 | runner = StatisticalThread( 1453 | profiler=prof, 1454 | period=options.statistic, 1455 | single=not options.threads, 1456 | ) 1457 | else: 1458 | if options.threads: 1459 | klass = ThreadProfile 1460 | else: 1461 | klass = Profile 1462 | prof = runner = klass(verbose=options.verbose) 1463 | try: 1464 | getattr(runner, runner_method_id)(**runner_method_kw) 1465 | finally: 1466 | if options.out == '-': 1467 | out = EncodeOrReplaceWriter(sys.stdout) 1468 | close = lambda: None 1469 | else: 1470 | if options.format == 'callgrindzip': 1471 | out = io.open(options.out, 'wb') 1472 | else: 1473 | out = io.open(options.out, 'w', errors='replace') 1474 | close = out.close 1475 | if options.exclude: 1476 | exclusion_search_list = [ 1477 | re.compile(x).search for x in options.exclude 1478 | ] 1479 | include_search_list = [ 1480 | re.compile(x).search for x in options.include 1481 | ] 1482 | filename_set = { 1483 | x for x in prof.getFilenameSet() 1484 | if not ( 1485 | any(y(x) for y in exclusion_search_list) and 1486 | not any(y(x) for y in include_search_list) 1487 | ) 1488 | } 1489 | else: 1490 | filename_set = None 1491 | commandline = quoteCommandline(args) 1492 | getattr(prof, format_dict[options.format])( 1493 | out, 1494 | filename=filename_set, 1495 | # python2 repr returns bytes, python3 repr returns unicode 1496 | commandline=getattr( 1497 | commandline, 1498 | 'decode', 1499 | lambda _: commandline, 1500 | )('ascii'), 1501 | relative_path=relative_path, 1502 | ) 1503 | close() 1504 | zip_path = options.zipfile 1505 | if zip_path: 1506 | if relative_path: 1507 | convertPath = _relpath 1508 | else: 1509 | convertPath = lambda x: x 1510 | with zipfile.ZipFile( 1511 | zip_path, 1512 | mode='w', 1513 | compression=zipfile.ZIP_DEFLATED, 1514 | ) as zip_file: 1515 | for name, lines in prof.iterSource(): 1516 | zip_file.writestr( 1517 | convertPath(name), 1518 | ''.join(lines) 1519 | ) 1520 | if options.statistic and not runner.clean_exit: 1521 | # Mostly useful for regresion testing, as exceptions raised in threads 1522 | # do not change exit status. 1523 | sys.exit(1) 1524 | 1525 | def pprofile(line, cell=None): 1526 | """ 1527 | Profile line execution. 1528 | """ 1529 | if cell is None: 1530 | # TODO: detect and use arguments (statistical profiling, ...) ? 1531 | return run(line) 1532 | return main( 1533 | ['%%pprofile', '-m', '-'] + shlex.split(line), 1534 | io.StringIO(cell), 1535 | ) 1536 | try: 1537 | register_line_cell_magic(pprofile) 1538 | except Exception: 1539 | # ipython can be imported, but may not be currently running. 1540 | pass 1541 | del pprofile 1542 | 1543 | from ._version import get_versions 1544 | __version__ = get_versions()['version'] 1545 | del get_versions 1546 | -------------------------------------------------------------------------------- /pprofile/__main__.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | # Copyright (C) 2020-2024 Vincent Pelletier 3 | # 4 | # This program is free software; you can redistribute it and/or 5 | # modify it under the terms of the GNU General Public License 6 | # as published by the Free Software Foundation; either version 2 7 | # of the License, or (at your option) any later version. 8 | # 9 | # This program is distributed in the hope that it will be useful, 10 | # but WITHOUT ANY WARRANTY; without even the implied warranty of 11 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 12 | # GNU General Public License for more details. 13 | # 14 | # You should have received a copy of the GNU General Public License 15 | # along with this program; if not, write to the Free Software 16 | # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA. 17 | 18 | from . import main 19 | 20 | main() 21 | -------------------------------------------------------------------------------- /pprofile/_version.py: -------------------------------------------------------------------------------- 1 | 2 | # This file helps to compute a version number in source trees obtained from 3 | # git-archive tarball (such as those provided by githubs download-from-tag 4 | # feature). Distribution tarballs (built by setup.py sdist) and build 5 | # directories (produced by setup.py build) will contain a much shorter file 6 | # that just contains the computed version number. 7 | 8 | # This file is released into the public domain. Generated by 9 | # versioneer-0.19 (https://github.com/python-versioneer/python-versioneer) 10 | 11 | """Git implementation of _version.py.""" 12 | 13 | import errno 14 | import os 15 | import re 16 | import subprocess 17 | import sys 18 | 19 | 20 | def get_keywords(): 21 | """Get the keywords needed to look up the version information.""" 22 | # these strings will be replaced by git during git-archive. 23 | # setup.py/versioneer.py will grep for the variable names, so they must 24 | # each be defined on a line of their own. _version.py will just call 25 | # get_keywords(). 26 | git_refnames = " (HEAD -> master, tag: 2.2.0)" 27 | git_full = "14ef3a8cb2670451d09fc45c61f72ac35620a8ee" 28 | git_date = "2024-09-06 07:00:09 +0200" 29 | keywords = {"refnames": git_refnames, "full": git_full, "date": git_date} 30 | return keywords 31 | 32 | 33 | class VersioneerConfig: 34 | """Container for Versioneer configuration parameters.""" 35 | 36 | 37 | def get_config(): 38 | """Create, populate and return the VersioneerConfig() object.""" 39 | # these strings are filled in when 'setup.py versioneer' creates 40 | # _version.py 41 | cfg = VersioneerConfig() 42 | cfg.VCS = "git" 43 | cfg.style = "pep440" 44 | cfg.tag_prefix = "" 45 | cfg.parentdir_prefix = "pprofile-" 46 | cfg.versionfile_source = "pprofile/_version.py" 47 | cfg.verbose = False 48 | return cfg 49 | 50 | 51 | class NotThisMethod(Exception): 52 | """Exception raised if a method is not valid for the current scenario.""" 53 | 54 | 55 | LONG_VERSION_PY = {} 56 | HANDLERS = {} 57 | 58 | 59 | def register_vcs_handler(vcs, method): # decorator 60 | """Create decorator to mark a method as the handler of a VCS.""" 61 | def decorate(f): 62 | """Store f in HANDLERS[vcs][method].""" 63 | if vcs not in HANDLERS: 64 | HANDLERS[vcs] = {} 65 | HANDLERS[vcs][method] = f 66 | return f 67 | return decorate 68 | 69 | 70 | def run_command(commands, args, cwd=None, verbose=False, hide_stderr=False, 71 | env=None): 72 | """Call the given command(s).""" 73 | assert isinstance(commands, list) 74 | p = None 75 | for c in commands: 76 | try: 77 | dispcmd = str([c] + args) 78 | # remember shell=False, so use git.cmd on windows, not just git 79 | p = subprocess.Popen([c] + args, cwd=cwd, env=env, 80 | stdout=subprocess.PIPE, 81 | stderr=(subprocess.PIPE if hide_stderr 82 | else None)) 83 | break 84 | except EnvironmentError: 85 | e = sys.exc_info()[1] 86 | if e.errno == errno.ENOENT: 87 | continue 88 | if verbose: 89 | print("unable to run %s" % dispcmd) 90 | print(e) 91 | return None, None 92 | else: 93 | if verbose: 94 | print("unable to find command, tried %s" % (commands,)) 95 | return None, None 96 | stdout = p.communicate()[0].strip().decode() 97 | if p.returncode != 0: 98 | if verbose: 99 | print("unable to run %s (error)" % dispcmd) 100 | print("stdout was %s" % stdout) 101 | return None, p.returncode 102 | return stdout, p.returncode 103 | 104 | 105 | def versions_from_parentdir(parentdir_prefix, root, verbose): 106 | """Try to determine the version from the parent directory name. 107 | 108 | Source tarballs conventionally unpack into a directory that includes both 109 | the project name and a version string. We will also support searching up 110 | two directory levels for an appropriately named parent directory 111 | """ 112 | rootdirs = [] 113 | 114 | for i in range(3): 115 | dirname = os.path.basename(root) 116 | if dirname.startswith(parentdir_prefix): 117 | return {"version": dirname[len(parentdir_prefix):], 118 | "full-revisionid": None, 119 | "dirty": False, "error": None, "date": None} 120 | else: 121 | rootdirs.append(root) 122 | root = os.path.dirname(root) # up a level 123 | 124 | if verbose: 125 | print("Tried directories %s but none started with prefix %s" % 126 | (str(rootdirs), parentdir_prefix)) 127 | raise NotThisMethod("rootdir doesn't start with parentdir_prefix") 128 | 129 | 130 | @register_vcs_handler("git", "get_keywords") 131 | def git_get_keywords(versionfile_abs): 132 | """Extract version information from the given file.""" 133 | # the code embedded in _version.py can just fetch the value of these 134 | # keywords. When used from setup.py, we don't want to import _version.py, 135 | # so we do it with a regexp instead. This function is not used from 136 | # _version.py. 137 | keywords = {} 138 | try: 139 | f = open(versionfile_abs, "r") 140 | for line in f.readlines(): 141 | if line.strip().startswith("git_refnames ="): 142 | mo = re.search(r'=\s*"(.*)"', line) 143 | if mo: 144 | keywords["refnames"] = mo.group(1) 145 | if line.strip().startswith("git_full ="): 146 | mo = re.search(r'=\s*"(.*)"', line) 147 | if mo: 148 | keywords["full"] = mo.group(1) 149 | if line.strip().startswith("git_date ="): 150 | mo = re.search(r'=\s*"(.*)"', line) 151 | if mo: 152 | keywords["date"] = mo.group(1) 153 | f.close() 154 | except EnvironmentError: 155 | pass 156 | return keywords 157 | 158 | 159 | @register_vcs_handler("git", "keywords") 160 | def git_versions_from_keywords(keywords, tag_prefix, verbose): 161 | """Get version information from git keywords.""" 162 | if not keywords: 163 | raise NotThisMethod("no keywords at all, weird") 164 | date = keywords.get("date") 165 | if date is not None: 166 | # Use only the last line. Previous lines may contain GPG signature 167 | # information. 168 | date = date.splitlines()[-1] 169 | 170 | # git-2.2.0 added "%cI", which expands to an ISO-8601 -compliant 171 | # datestamp. However we prefer "%ci" (which expands to an "ISO-8601 172 | # -like" string, which we must then edit to make compliant), because 173 | # it's been around since git-1.5.3, and it's too difficult to 174 | # discover which version we're using, or to work around using an 175 | # older one. 176 | date = date.strip().replace(" ", "T", 1).replace(" ", "", 1) 177 | refnames = keywords["refnames"].strip() 178 | if refnames.startswith("$Format"): 179 | if verbose: 180 | print("keywords are unexpanded, not using") 181 | raise NotThisMethod("unexpanded keywords, not a git-archive tarball") 182 | refs = set([r.strip() for r in refnames.strip("()").split(",")]) 183 | # starting in git-1.8.3, tags are listed as "tag: foo-1.0" instead of 184 | # just "foo-1.0". If we see a "tag: " prefix, prefer those. 185 | TAG = "tag: " 186 | tags = set([r[len(TAG):] for r in refs if r.startswith(TAG)]) 187 | if not tags: 188 | # Either we're using git < 1.8.3, or there really are no tags. We use 189 | # a heuristic: assume all version tags have a digit. The old git %d 190 | # expansion behaves like git log --decorate=short and strips out the 191 | # refs/heads/ and refs/tags/ prefixes that would let us distinguish 192 | # between branches and tags. By ignoring refnames without digits, we 193 | # filter out many common branch names like "release" and 194 | # "stabilization", as well as "HEAD" and "master". 195 | tags = set([r for r in refs if re.search(r'\d', r)]) 196 | if verbose: 197 | print("discarding '%s', no digits" % ",".join(refs - tags)) 198 | if verbose: 199 | print("likely tags: %s" % ",".join(sorted(tags))) 200 | for ref in sorted(tags): 201 | # sorting will prefer e.g. "2.0" over "2.0rc1" 202 | if ref.startswith(tag_prefix): 203 | r = ref[len(tag_prefix):] 204 | if verbose: 205 | print("picking %s" % r) 206 | return {"version": r, 207 | "full-revisionid": keywords["full"].strip(), 208 | "dirty": False, "error": None, 209 | "date": date} 210 | # no suitable tags, so version is "0+unknown", but full hex is still there 211 | if verbose: 212 | print("no suitable tags, using unknown + full revision id") 213 | return {"version": "0+unknown", 214 | "full-revisionid": keywords["full"].strip(), 215 | "dirty": False, "error": "no suitable tags", "date": None} 216 | 217 | 218 | @register_vcs_handler("git", "pieces_from_vcs") 219 | def git_pieces_from_vcs(tag_prefix, root, verbose, run_command=run_command): 220 | """Get version from 'git describe' in the root of the source tree. 221 | 222 | This only gets called if the git-archive 'subst' keywords were *not* 223 | expanded, and _version.py hasn't already been rewritten with a short 224 | version string, meaning we're inside a checked out source tree. 225 | """ 226 | GITS = ["git"] 227 | if sys.platform == "win32": 228 | GITS = ["git.cmd", "git.exe"] 229 | 230 | out, rc = run_command(GITS, ["rev-parse", "--git-dir"], cwd=root, 231 | hide_stderr=True) 232 | if rc != 0: 233 | if verbose: 234 | print("Directory %s not under git control" % root) 235 | raise NotThisMethod("'git rev-parse --git-dir' returned error") 236 | 237 | # if there is a tag matching tag_prefix, this yields TAG-NUM-gHEX[-dirty] 238 | # if there isn't one, this yields HEX[-dirty] (no NUM) 239 | describe_out, rc = run_command(GITS, ["describe", "--tags", "--dirty", 240 | "--always", "--long", 241 | "--match", "%s*" % tag_prefix], 242 | cwd=root) 243 | # --long was added in git-1.5.5 244 | if describe_out is None: 245 | raise NotThisMethod("'git describe' failed") 246 | describe_out = describe_out.strip() 247 | full_out, rc = run_command(GITS, ["rev-parse", "HEAD"], cwd=root) 248 | if full_out is None: 249 | raise NotThisMethod("'git rev-parse' failed") 250 | full_out = full_out.strip() 251 | 252 | pieces = {} 253 | pieces["long"] = full_out 254 | pieces["short"] = full_out[:7] # maybe improved later 255 | pieces["error"] = None 256 | 257 | # parse describe_out. It will be like TAG-NUM-gHEX[-dirty] or HEX[-dirty] 258 | # TAG might have hyphens. 259 | git_describe = describe_out 260 | 261 | # look for -dirty suffix 262 | dirty = git_describe.endswith("-dirty") 263 | pieces["dirty"] = dirty 264 | if dirty: 265 | git_describe = git_describe[:git_describe.rindex("-dirty")] 266 | 267 | # now we have TAG-NUM-gHEX or HEX 268 | 269 | if "-" in git_describe: 270 | # TAG-NUM-gHEX 271 | mo = re.search(r'^(.+)-(\d+)-g([0-9a-f]+)$', git_describe) 272 | if not mo: 273 | # unparseable. Maybe git-describe is misbehaving? 274 | pieces["error"] = ("unable to parse git-describe output: '%s'" 275 | % describe_out) 276 | return pieces 277 | 278 | # tag 279 | full_tag = mo.group(1) 280 | if not full_tag.startswith(tag_prefix): 281 | if verbose: 282 | fmt = "tag '%s' doesn't start with prefix '%s'" 283 | print(fmt % (full_tag, tag_prefix)) 284 | pieces["error"] = ("tag '%s' doesn't start with prefix '%s'" 285 | % (full_tag, tag_prefix)) 286 | return pieces 287 | pieces["closest-tag"] = full_tag[len(tag_prefix):] 288 | 289 | # distance: number of commits since tag 290 | pieces["distance"] = int(mo.group(2)) 291 | 292 | # commit: short hex revision ID 293 | pieces["short"] = mo.group(3) 294 | 295 | else: 296 | # HEX: no tags 297 | pieces["closest-tag"] = None 298 | count_out, rc = run_command(GITS, ["rev-list", "HEAD", "--count"], 299 | cwd=root) 300 | pieces["distance"] = int(count_out) # total number of commits 301 | 302 | # commit date: see ISO-8601 comment in git_versions_from_keywords() 303 | date = run_command(GITS, ["show", "-s", "--format=%ci", "HEAD"], 304 | cwd=root)[0].strip() 305 | # Use only the last line. Previous lines may contain GPG signature 306 | # information. 307 | date = date.splitlines()[-1] 308 | pieces["date"] = date.strip().replace(" ", "T", 1).replace(" ", "", 1) 309 | 310 | return pieces 311 | 312 | 313 | def plus_or_dot(pieces): 314 | """Return a + if we don't already have one, else return a .""" 315 | if "+" in pieces.get("closest-tag", ""): 316 | return "." 317 | return "+" 318 | 319 | 320 | def render_pep440(pieces): 321 | """Build up version string, with post-release "local version identifier". 322 | 323 | Our goal: TAG[+DISTANCE.gHEX[.dirty]] . Note that if you 324 | get a tagged build and then dirty it, you'll get TAG+0.gHEX.dirty 325 | 326 | Exceptions: 327 | 1: no tags. git_describe was just HEX. 0+untagged.DISTANCE.gHEX[.dirty] 328 | """ 329 | if pieces["closest-tag"]: 330 | rendered = pieces["closest-tag"] 331 | if pieces["distance"] or pieces["dirty"]: 332 | rendered += plus_or_dot(pieces) 333 | rendered += "%d.g%s" % (pieces["distance"], pieces["short"]) 334 | if pieces["dirty"]: 335 | rendered += ".dirty" 336 | else: 337 | # exception #1 338 | rendered = "0+untagged.%d.g%s" % (pieces["distance"], 339 | pieces["short"]) 340 | if pieces["dirty"]: 341 | rendered += ".dirty" 342 | return rendered 343 | 344 | 345 | def render_pep440_pre(pieces): 346 | """TAG[.post0.devDISTANCE] -- No -dirty. 347 | 348 | Exceptions: 349 | 1: no tags. 0.post0.devDISTANCE 350 | """ 351 | if pieces["closest-tag"]: 352 | rendered = pieces["closest-tag"] 353 | if pieces["distance"]: 354 | rendered += ".post0.dev%d" % pieces["distance"] 355 | else: 356 | # exception #1 357 | rendered = "0.post0.dev%d" % pieces["distance"] 358 | return rendered 359 | 360 | 361 | def render_pep440_post(pieces): 362 | """TAG[.postDISTANCE[.dev0]+gHEX] . 363 | 364 | The ".dev0" means dirty. Note that .dev0 sorts backwards 365 | (a dirty tree will appear "older" than the corresponding clean one), 366 | but you shouldn't be releasing software with -dirty anyways. 367 | 368 | Exceptions: 369 | 1: no tags. 0.postDISTANCE[.dev0] 370 | """ 371 | if pieces["closest-tag"]: 372 | rendered = pieces["closest-tag"] 373 | if pieces["distance"] or pieces["dirty"]: 374 | rendered += ".post%d" % pieces["distance"] 375 | if pieces["dirty"]: 376 | rendered += ".dev0" 377 | rendered += plus_or_dot(pieces) 378 | rendered += "g%s" % pieces["short"] 379 | else: 380 | # exception #1 381 | rendered = "0.post%d" % pieces["distance"] 382 | if pieces["dirty"]: 383 | rendered += ".dev0" 384 | rendered += "+g%s" % pieces["short"] 385 | return rendered 386 | 387 | 388 | def render_pep440_old(pieces): 389 | """TAG[.postDISTANCE[.dev0]] . 390 | 391 | The ".dev0" means dirty. 392 | 393 | Exceptions: 394 | 1: no tags. 0.postDISTANCE[.dev0] 395 | """ 396 | if pieces["closest-tag"]: 397 | rendered = pieces["closest-tag"] 398 | if pieces["distance"] or pieces["dirty"]: 399 | rendered += ".post%d" % pieces["distance"] 400 | if pieces["dirty"]: 401 | rendered += ".dev0" 402 | else: 403 | # exception #1 404 | rendered = "0.post%d" % pieces["distance"] 405 | if pieces["dirty"]: 406 | rendered += ".dev0" 407 | return rendered 408 | 409 | 410 | def render_git_describe(pieces): 411 | """TAG[-DISTANCE-gHEX][-dirty]. 412 | 413 | Like 'git describe --tags --dirty --always'. 414 | 415 | Exceptions: 416 | 1: no tags. HEX[-dirty] (note: no 'g' prefix) 417 | """ 418 | if pieces["closest-tag"]: 419 | rendered = pieces["closest-tag"] 420 | if pieces["distance"]: 421 | rendered += "-%d-g%s" % (pieces["distance"], pieces["short"]) 422 | else: 423 | # exception #1 424 | rendered = pieces["short"] 425 | if pieces["dirty"]: 426 | rendered += "-dirty" 427 | return rendered 428 | 429 | 430 | def render_git_describe_long(pieces): 431 | """TAG-DISTANCE-gHEX[-dirty]. 432 | 433 | Like 'git describe --tags --dirty --always -long'. 434 | The distance/hash is unconditional. 435 | 436 | Exceptions: 437 | 1: no tags. HEX[-dirty] (note: no 'g' prefix) 438 | """ 439 | if pieces["closest-tag"]: 440 | rendered = pieces["closest-tag"] 441 | rendered += "-%d-g%s" % (pieces["distance"], pieces["short"]) 442 | else: 443 | # exception #1 444 | rendered = pieces["short"] 445 | if pieces["dirty"]: 446 | rendered += "-dirty" 447 | return rendered 448 | 449 | 450 | def render(pieces, style): 451 | """Render the given version pieces into the requested style.""" 452 | if pieces["error"]: 453 | return {"version": "unknown", 454 | "full-revisionid": pieces.get("long"), 455 | "dirty": None, 456 | "error": pieces["error"], 457 | "date": None} 458 | 459 | if not style or style == "default": 460 | style = "pep440" # the default 461 | 462 | if style == "pep440": 463 | rendered = render_pep440(pieces) 464 | elif style == "pep440-pre": 465 | rendered = render_pep440_pre(pieces) 466 | elif style == "pep440-post": 467 | rendered = render_pep440_post(pieces) 468 | elif style == "pep440-old": 469 | rendered = render_pep440_old(pieces) 470 | elif style == "git-describe": 471 | rendered = render_git_describe(pieces) 472 | elif style == "git-describe-long": 473 | rendered = render_git_describe_long(pieces) 474 | else: 475 | raise ValueError("unknown style '%s'" % style) 476 | 477 | return {"version": rendered, "full-revisionid": pieces["long"], 478 | "dirty": pieces["dirty"], "error": None, 479 | "date": pieces.get("date")} 480 | 481 | 482 | def get_versions(): 483 | """Get version information or return default if unable to do so.""" 484 | # I am in _version.py, which lives at ROOT/VERSIONFILE_SOURCE. If we have 485 | # __file__, we can work backwards from there to the root. Some 486 | # py2exe/bbfreeze/non-CPython implementations don't do __file__, in which 487 | # case we can only use expanded keywords. 488 | 489 | cfg = get_config() 490 | verbose = cfg.verbose 491 | 492 | try: 493 | return git_versions_from_keywords(get_keywords(), cfg.tag_prefix, 494 | verbose) 495 | except NotThisMethod: 496 | pass 497 | 498 | try: 499 | root = os.path.realpath(__file__) 500 | # versionfile_source is the relative path from the top of the source 501 | # tree (where the .git directory might live) to this file. Invert 502 | # this to find the root from __file__. 503 | for i in cfg.versionfile_source.split('/'): 504 | root = os.path.dirname(root) 505 | except NameError: 506 | return {"version": "0+unknown", "full-revisionid": None, 507 | "dirty": None, 508 | "error": "unable to find root of source tree", 509 | "date": None} 510 | 511 | try: 512 | pieces = git_pieces_from_vcs(cfg.tag_prefix, root, verbose) 513 | return render(pieces, cfg.style) 514 | except NotThisMethod: 515 | pass 516 | 517 | try: 518 | if cfg.parentdir_prefix: 519 | return versions_from_parentdir(cfg.parentdir_prefix, root, verbose) 520 | except NotThisMethod: 521 | pass 522 | 523 | return {"version": "0+unknown", "full-revisionid": None, 524 | "dirty": None, 525 | "error": "unable to compute version", "date": None} 526 | -------------------------------------------------------------------------------- /pprofile/zope.py: -------------------------------------------------------------------------------- 1 | # Copyright (C) 2016-2024 Vincent Pelletier 2 | # 3 | # This program is free software; you can redistribute it and/or 4 | # modify it under the terms of the GNU General Public License 5 | # as published by the Free Software Foundation; either version 2 6 | # of the License, or (at your option) any later version. 7 | # 8 | # This program is distributed in the hope that it will be useful, 9 | # but WITHOUT ANY WARRANTY; without even the implied warranty of 10 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 11 | # GNU General Public License for more details. 12 | # 13 | # You should have received a copy of the GNU General Public License 14 | # along with this program; if not, write to the Free Software 15 | # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA. 16 | """ 17 | Zope-friendly layer for pprofile. 18 | 19 | In Zope: 20 | - Executed code is not necessarily a valid FS path (ex: Python Scripts) 21 | - Executed code is not available to the machine where profiling results are 22 | analysed. 23 | - Restricted Python cannot manipulate all desired types, and one may want to 24 | trigger profiling from its level. 25 | 26 | This layer addresses all these issues, by making interesting pprofile classes 27 | accessible to restricted python and bundling source code wxith profiling 28 | results. 29 | 30 | NOTE: This does allow anyone able to get profiler output to get whole source 31 | files from your server. So better keep good track of who can profile and/or 32 | where profiling results end. Alone, this module won't be accessible from 33 | Restricted Python. 34 | 35 | Example deterministic usage: 36 | # Get profiler (how you get to zpprofile module depends on your 37 | # application). 38 | profiler = zpprofile.getProfiler() 39 | # Get callable (to not profile how it is retrieved). 40 | func = context.somethingOrOther 41 | # Actually profile stuff 42 | with profiler: 43 | func() 44 | # Build response 45 | response = context.REQUEST.RESPONSE 46 | data, content_type = profiler.asZip() 47 | response.setHeader('content-type', content_type) 48 | response.setHeader( 49 | 'content-disposition', 50 | 'attachment; filename="' + func.id + '.zip"', 51 | ) 52 | # Push response immediately (hopefully, profiled function did not write 53 | # anything on its own). 54 | response.write(data) 55 | # Make transaction fail, so any otherwise persistent change made by 56 | # profiled function is undone - note that many caches will still have 57 | # been warmed up, just as with any other code. 58 | raise Exception('profiling') 59 | 60 | Example statistic usage (to profile other running threads): 61 | from time import sleep 62 | # Get profiler (how you get to zpprofile module depends on your 63 | # application). 64 | profiler, thread = zpprofile.getStatisticalProfilerAndThread(single=False) 65 | # Actually profile whatever is going on in the same process, just waiting. 66 | with thread: 67 | sleep(60) 68 | # Build response 69 | response = context.REQUEST.RESPONSE 70 | data, content_type = profiler.asZip() 71 | response.setHeader('content-type', content_type) 72 | response.setHeader( 73 | 'content-disposition', 74 | 'attachment; filename="statistical_' + 75 | DateTime().strftime('%Y%m%d%H%M%S') + 76 | '.zip"', 77 | ) 78 | return data 79 | """ 80 | from __future__ import print_function 81 | import dis 82 | import functools 83 | import gc 84 | from io import StringIO, BytesIO 85 | from importlib import import_module 86 | import itertools 87 | from collections import defaultdict 88 | import sys 89 | import pprofile 90 | 91 | if sys.version_info >= (3, ): 92 | unicode = str 93 | 94 | def getFuncCodeOrNone(module, attribute_path): 95 | try: 96 | value = import_module(module) 97 | for attribute in attribute_path: 98 | value = getattr(value, attribute) 99 | value = value.__code__ 100 | except (ImportError, AttributeError): 101 | print('Could not reach func_code of module %r, attribute path %r' % (module, attribute_path)) 102 | return None 103 | return value 104 | 105 | DB_query_func_code = getFuncCodeOrNone('Products.ZMySQLDA.db', ('DB', '_query')) 106 | ZODB_setstate_func_code = getFuncCodeOrNone('ZODB.Connection', ('Connection', 'setstate')) 107 | PythonExpr__call__func_code = getFuncCodeOrNone('zope.tales.pythonexpr', ('PythonExpr', '__call__')) 108 | ZRPythonExpr__call__func_code = getFuncCodeOrNone('Products.PageTemplates.ZRPythonExpr', ('PythonExpr', '__call__')) 109 | DT_UtilEvaleval_func_code = getFuncCodeOrNone('DocumentTemplate.DT_Util', ('Eval', 'eval')) 110 | SharedDCScriptsBindings_bindAndExec_func_code = getFuncCodeOrNone('Shared.DC.Scripts.Bindings', ('Bindings', '_bindAndExec')) 111 | PythonScript_exec_func_code = getFuncCodeOrNone('Products.PythonScripts.PythonScript', ('PythonScript', '_exec')) 112 | 113 | # OFS.Traversable.Traversable.unrestrictedTraverse overwites its path argument, 114 | # preventing post-invocation introspection. As it does not mutate the argument, 115 | # it is still possible to inspect using such controlled intermediate function. 116 | def unrestrictedTraverse_spy(self, path, *args, **kw): 117 | return orig_unrestrictedTraverse(self, path, *args, **kw) 118 | unrestrictedTraverse_spy_func_code = unrestrictedTraverse_spy.__code__ 119 | try: 120 | import OFS.Traversable 121 | orig_unrestrictedTraverse = OFS.Traversable.Traversable.unrestrictedTraverse 122 | except (ImportError, AttributeError): 123 | pass 124 | else: 125 | functools.update_wrapper(unrestrictedTraverse_spy, orig_unrestrictedTraverse) 126 | OFS.Traversable.Traversable.unrestrictedTraverse = unrestrictedTraverse_spy 127 | 128 | PYTHON_EXPR_FUNC_CODE_SET = (ZRPythonExpr__call__func_code, PythonExpr__call__func_code) 129 | 130 | class ZopeFileTiming(pprofile.FileTiming): 131 | def call(self, code, line, callee_file_timing, callee, duration, frame): 132 | f_code = frame.f_code 133 | if f_code is DB_query_func_code: 134 | self.profiler.sql_dict[frame.f_locals['query']].append(duration) 135 | elif f_code is ZODB_setstate_func_code: 136 | f_locals = frame.f_locals 137 | obj = f_locals['obj'] 138 | try: 139 | oid = obj._p_oid 140 | except AttributeError: 141 | pass 142 | else: 143 | self.profiler.zodb_dict[ 144 | f_locals['self'].db().database_name 145 | ][oid].append(duration) 146 | elif f_code is unrestrictedTraverse_spy_func_code: 147 | f_locals = frame.f_locals 148 | self.profiler.traverse_dict[ 149 | (repr(f_locals['self']), repr(f_locals['path'])) 150 | ].append(duration) 151 | super(ZopeFileTiming, self).call( 152 | code, line, callee_file_timing, callee, duration, frame, 153 | ) 154 | 155 | def tabulate(title_list, row_list): 156 | # de-lazify 157 | row_list = list(row_list) 158 | column_count = len(title_list) 159 | max_width_list = [len(x) for x in title_list] 160 | for row in row_list: 161 | assert len(row) == column_count, repr(row) 162 | for index, value in enumerate(row): 163 | max_width_list[index] = max(max_width_list[index], len(unicode(value))) 164 | format_string = u''.join(u'| %%-%is ' % x for x in max_width_list) + u'|\n' 165 | out = StringIO() 166 | write = out.write 167 | write(format_string % tuple(title_list)) 168 | write(u''.join(u'+' + (u'-' * (x + 2)) for x in max_width_list) + u'+\n') 169 | for row in row_list: 170 | write(format_string % tuple(row)) 171 | return out.getvalue() 172 | 173 | def disassemble(co, lasti=-1): 174 | """Disassemble a code object.""" 175 | # Taken from dis.disassemble, returns disassembled code instead of printing 176 | # it (the fuck python ?). 177 | # Also, unicodified. 178 | # Also, use % operator instead of string operations. 179 | # Also, one statement per line. 180 | out = StringIO() 181 | code = co.co_code 182 | labels = dis.findlabels(code) 183 | linestarts = dict(dis.findlinestarts(co)) 184 | n = len(code) 185 | i = 0 186 | extended_arg = 0 187 | free = None 188 | while i < n: 189 | c = code[i] 190 | op = ord(c) 191 | if i in linestarts: 192 | if i > 0: 193 | print(end=u'\n', file=out) 194 | print(u'%3d' % linestarts[i], end=u' ', file=out) 195 | else: 196 | print(u' ', end=u' ', file=out) 197 | 198 | if i == lasti: 199 | print(u'-->', end=u' ', file=out) 200 | else: 201 | print(u' ', end=u' ', file=out) 202 | if i in labels: 203 | print(u'>>', end=u' ', file=out) 204 | else: 205 | print(u' ', end=u' ', file=out) 206 | print(u'%4i' % i, end=u' ', file=out) 207 | print(u'%-20s' % dis.opname[op], end=u' ', file=out) 208 | i = i + 1 209 | if op >= dis.HAVE_ARGUMENT: 210 | oparg = ord(code[i]) + ord(code[i + 1]) * 256 + extended_arg 211 | extended_arg = 0 212 | i = i + 2 213 | if op == dis.EXTENDED_ARG: 214 | extended_arg = oparg * 65536 215 | print(u'%5i' % oparg, end=u' ', file=out) 216 | if op in dis.hasconst: 217 | print(u'(%r)' % co.co_consts[oparg], end=u' ', file=out) 218 | elif op in dis.hasname: 219 | print(u'(%s)' % co.co_names[oparg], end=u' ', file=out) 220 | elif op in dis.hasjrel: 221 | print(u'(to %r)' % (i + oparg), end=u' ', file=out) 222 | elif op in dis.haslocal: 223 | print(u'(%s)' % co.co_varnames[oparg], end=u' ', file=out) 224 | elif op in dis.hascompare: 225 | print(u'(%s)' % dis.cmp_op[oparg], end=u' ', file=out) 226 | elif op in dis.hasfree: 227 | if free is None: 228 | free = co.co_cellvars + co.co_freevars 229 | print(u'(%s)' % free[oparg], end=u' ', file=out) 230 | print(end=u'\n', file=out) 231 | return out.getvalue() 232 | 233 | class ZopeMixIn(object): 234 | virtual__slots__ = ( 235 | 'sql_dict', 236 | 'zodb_dict', 237 | 'fake_source_dict', 238 | 'traverse_dict', 239 | 'keep_alive', # until they see the cake 240 | ) 241 | __allow_access_to_unprotected_subobjects__ = 1 242 | FileTiming = ZopeFileTiming 243 | 244 | def __init__(self): 245 | super(ZopeMixIn, self).__init__() 246 | self.sql_dict = defaultdict(list) 247 | self.zodb_dict = defaultdict(lambda: defaultdict(list)) 248 | self.fake_source_dict = {} 249 | self.traverse_dict = defaultdict(list) 250 | self.keep_alive = [] 251 | 252 | def _enable(self): 253 | gc.disable() 254 | super(ZopeMixIn, self)._enable() 255 | 256 | def _disable(self): 257 | super(ZopeMixIn, self)._disable() 258 | gc.enable() 259 | 260 | def _getline(self, filename, lineno, global_dict): 261 | line_list = self.fake_source_dict.get(filename) 262 | if line_list is None: 263 | return super(ZopeMixIn, self)._getline( 264 | filename, 265 | lineno, 266 | global_dict, 267 | ) 268 | assert lineno > 0 269 | try: 270 | return line_list[lineno - 1] 271 | except IndexError: 272 | return '' 273 | 274 | def _rememberFile(self, source, suggested_name, extension): 275 | filename = suggested_name 276 | setdefault = self.fake_source_dict.setdefault 277 | suffix = itertools.count() 278 | source = source.splitlines(True) 279 | while setdefault(filename + extension, source) != source: 280 | filename = suggested_name + '_%i' % next(suffix) 281 | return filename + extension 282 | 283 | def _getFileTiming(self, frame): 284 | try: 285 | return self.global_dict[id(frame.f_globals)] 286 | except KeyError: 287 | frame_globals = frame.f_globals 288 | evaluator_frame = frame.f_back 289 | while evaluator_frame is not None: 290 | evaluator_code = evaluator_frame.f_code 291 | if evaluator_code is PythonScript_exec_func_code: 292 | if evaluator_frame.f_locals.get('safe_globals') is frame_globals: 293 | evaluated_module_unique = evaluator_frame.f_locals['function_code'] 294 | break 295 | elif evaluator_frame.f_locals.get('g') is frame_globals: 296 | evaluated_module_unique = evaluator_frame.f_locals['fcode'] 297 | break 298 | if ( 299 | evaluator_code is PythonScript_exec_func_code and ( 300 | evaluator_frame.f_locals.get('safe_globals') is frame_globals or 301 | evaluator_frame.f_locals.get('g') is frame_globals 302 | ) 303 | ): 304 | evaluated_module_unique = evaluator_frame.f_locals['fcode'] 305 | break 306 | elif ( 307 | evaluator_code is DT_UtilEvaleval_func_code and 308 | evaluator_frame.f_locals.get('d') is frame_globals 309 | ): 310 | evaluated_module_unique = evaluator_frame.f_locals['code'] 311 | break 312 | elif ( 313 | evaluator_code in PYTHON_EXPR_FUNC_CODE_SET and 314 | evaluator_frame.f_locals.get('vars') is frame_globals 315 | ): 316 | evaluated_module_unique = evaluator_frame.f_locals[ 317 | 'self' 318 | ]._code 319 | break 320 | evaluator_frame = evaluator_frame.f_back 321 | else: 322 | # No evaluator found 323 | evaluator_frame = frame 324 | evaluated_module_unique = frame_globals 325 | try: 326 | file_timing = self.global_dict[id(evaluated_module_unique)] 327 | except KeyError: 328 | # Unknown module, guess its name. 329 | if evaluator_frame is frame: 330 | # No evaluator found. 331 | # The answer was not in the stack. 332 | # Maybe its name is actually fine ? 333 | name = self._getFilename(frame) 334 | if not super(ZopeMixIn, self)._getline( 335 | name, 336 | 1, 337 | frame.f_globals, 338 | ): 339 | # Shared.DC.Scripts preamble is directly called by 340 | # _bindAndExec. 341 | if getattr( 342 | frame.f_back, 343 | 'f_code', 344 | None, 345 | ) is SharedDCScriptsBindings_bindAndExec_func_code: 346 | name = self._rememberFile( 347 | u'# This is an auto-generated preamble executed ' 348 | u'by Shared.DC.Scripts.Bindings before "actual" ' 349 | u'code.\n' + 350 | disassemble(frame.f_code), 351 | 'preamble', 352 | '.py.bytecode', 353 | ) 354 | else: 355 | # Could not find source, provide disassembled 356 | # bytecode as last resort. 357 | name = self._rememberFile( 358 | u'# Unidentified source for ' + 359 | name + '\n' + 360 | disassemble( 361 | frame.f_code, 362 | ), 363 | '%s.%s' % (name, frame.f_code.co_name), 364 | '.py.bytecode', 365 | ) 366 | else: 367 | # Evaluator found. 368 | if evaluator_code is PythonScript_exec_func_code: 369 | python_script = evaluator_frame.f_locals['self'] 370 | name = self._rememberFile( 371 | python_script.body().decode('utf-8'), 372 | python_script.id, 373 | '.py', 374 | ) 375 | elif evaluator_code is DT_UtilEvaleval_func_code: 376 | name = self._rememberFile( 377 | evaluator_frame.f_locals['self'].expr.decode( 378 | 'utf-8', 379 | ), 380 | 'DT_Util_Eval', 381 | '.py', 382 | ) 383 | elif evaluator_code in PYTHON_EXPR_FUNC_CODE_SET: 384 | source = evaluator_frame.f_locals['self'].text 385 | if not isinstance(source, unicode): 386 | source = source.decode('utf-8') 387 | name = self._rememberFile( 388 | source, 389 | 'PythonExpr', 390 | '.py', 391 | ) 392 | else: 393 | raise ValueError(evaluator_code) 394 | self.keep_alive.append(evaluated_module_unique) 395 | # Create FileTiming and store as module... 396 | self.global_dict[ 397 | id(evaluated_module_unique) 398 | ] = file_timing = self.FileTiming( 399 | name, 400 | frame_globals, 401 | self, 402 | ) 403 | # ...and for later deduplication (in case of multithreading). 404 | # file_dict modifications must be thread-safe to not lose 405 | # measures. setdefault is atomic, append is atomic. 406 | self.file_dict.setdefault(name, []).append(file_timing) 407 | # Alias module FileTiming to current globals, for faster future 408 | # lookup. 409 | self.global_dict[id(frame_globals)] = file_timing 410 | self.keep_alive.append(frame_globals) 411 | return file_timing 412 | 413 | def _iterOutFiles(self, *args, **kw): 414 | """ 415 | Yields path, data, mimetype for each file involved on or produced by 416 | profiling. 417 | """ 418 | for entry in super(ZopeMixIn, self)._iterOutFiles(*args, **kw): 419 | yield entry 420 | sql_name_template = 'query_%%0%ii-%%i_hits_%%6fs.sql' % len( 421 | str(len(self.sql_dict)), 422 | ) 423 | for index, (query, time_list) in enumerate( 424 | sorted( 425 | self.sql_dict.items(), 426 | key=lambda x: (sum(x[1]), len(x[1])), 427 | reverse=True, 428 | ), 429 | ): 430 | yield ( 431 | sql_name_template % ( 432 | index, 433 | len(time_list), 434 | sum(time_list), 435 | ), 436 | b'\n'.join(b'-- %10.6fs' % x for x in time_list) + b'\n' + query, 437 | 'application/sql', 438 | ) 439 | if self.zodb_dict: 440 | yield ( 441 | 'ZODB_setstate.txt', 442 | '\n\n'.join( 443 | ( 444 | '%s (%fs)\n' % ( 445 | db_name, 446 | sum(sum(x) for x in oid_dict.values()), 447 | ) 448 | ) + '\n'.join( 449 | '%s (%i): %s' % ( 450 | oid.encode('hex'), 451 | len(time_list), 452 | ', '.join('%fs' % x for x in time_list), 453 | ) 454 | for oid, time_list in oid_dict.items() 455 | ) 456 | for db_name, oid_dict in self.zodb_dict.items() 457 | ), 458 | 'text/plain', 459 | ) 460 | if self.traverse_dict: 461 | yield ( 462 | 'unrestrictedTraverse_pathlist.txt', 463 | tabulate( 464 | ('self', 'path', 'hit', 'total duration'), 465 | sorted( 466 | ( 467 | (context, path, len(duration_list), sum(duration_list)) 468 | for (context, path), duration_list in self.traverse_dict.items() 469 | ), 470 | key=lambda x: x[3], 471 | reverse=True, 472 | ), 473 | ), 474 | 'text/plain', 475 | ) 476 | 477 | def getCallgrindMIME(self): 478 | """ 479 | Return a mime-multipart representation of: 480 | - callgrind profiling statistics (cachegrind.out.pprofile) 481 | - any SQL query issued via ZMySQLDA (query_*.sql) 482 | - any persistent object load via ZODB.Connection (ZODB_setstate.txt) 483 | - any path argument given to unrestrictedTraverse 484 | (unrestrictedTraverse_pathlist.txt) 485 | - all involved python code, including Python Scripts without hierarchy 486 | (the rest) 487 | and the mimetype of this string. 488 | To unpack resulting file, see "unpack a MIME message" in 489 | http://docs.python.org/2/library/email-examples.html 490 | Or get demultipart from 491 | https://pypi.python.org/pypi/demultipart 492 | """ 493 | out = BytesIO() 494 | mimetype = super(ZopeMixIn, self).getCallgrindMIME(out) 495 | return out.getvalue(), mimetype 496 | 497 | def getCallgrindZip(self): 498 | """ 499 | Return a serialised zip archive containing: 500 | - callgrind profiling statistics (cachegrind.out.pprofile) 501 | - any SQL query issued via ZMySQLDA (query_*.sql) 502 | - any persistent object load via ZODB.Connection (ZODB_setstate.txt) 503 | - any path argument given to unrestrictedTraverse 504 | (unrestrictedTraverse_pathlist.txt) 505 | - all involved python code, including Python Scripts without hierarchy 506 | (the rest) 507 | and the "application/zip" mimetype. 508 | """ 509 | out = BytesIO() 510 | mimetype = super(ZopeMixIn, self).getCallgrindZip(out) 511 | return out.getvalue(), mimetype 512 | 513 | # BBB 514 | asMIMEString = getCallgrindMIME 515 | asZip = getCallgrindZip 516 | 517 | class ZopeProfiler(ZopeMixIn, pprofile.Profile): 518 | __slots__ = ZopeMixIn.virtual__slots__ 519 | 520 | class ZopeStatisticalProfile(ZopeMixIn, pprofile.StatisticalProfile): 521 | __slots__ = ZopeMixIn.virtual__slots__ 522 | 523 | class ZopeStatisticalThread(pprofile.StatisticalThread): 524 | __allow_access_to_unprotected_subobjects__ = 1 525 | 526 | # Intercept "verbose" parameter to prevent writing to stdout. 527 | def getProfiler(verbose=False, **kw): 528 | """ 529 | Get a Zope-friendly pprofile.Profile instance. 530 | """ 531 | return ZopeProfiler(**kw) 532 | 533 | def getStatisticalProfilerAndThread(**kw): 534 | """ 535 | Get Zope-friendly pprofile.StatisticalProfile and 536 | pprofile.StatisticalThread instances. 537 | Arguments are forwarded to StatisticalThread.__init__ . 538 | """ 539 | profiler = ZopeStatisticalProfile() 540 | return profiler, ZopeStatisticalThread( 541 | profiler=profiler, 542 | **kw 543 | ) 544 | -------------------------------------------------------------------------------- /setup.cfg: -------------------------------------------------------------------------------- 1 | [versioneer] 2 | VCS = git 3 | style = pep440 4 | versionfile_source = pprofile/_version.py 5 | versionfile_build = pprofile/_version.py 6 | tag_prefix = 7 | parentdir_prefix = pprofile- 8 | -------------------------------------------------------------------------------- /setup.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | # Copyright (C) 2013-2024 Vincent Pelletier 3 | # 4 | # This program is free software; you can redistribute it and/or 5 | # modify it under the terms of the GNU General Public License 6 | # as published by the Free Software Foundation; either version 2 7 | # of the License, or (at your option) any later version. 8 | # 9 | # This program is distributed in the hope that it will be useful, 10 | # but WITHOUT ANY WARRANTY; without even the implied warranty of 11 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 12 | # GNU General Public License for more details. 13 | # 14 | # You should have received a copy of the GNU General Public License 15 | # along with this program; if not, write to the Free Software 16 | # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA. 17 | from os.path import join, dirname 18 | import sys 19 | from setuptools import setup 20 | import versioneer 21 | 22 | description = open(join(dirname(__file__), 'README.rst')).read() 23 | setup( 24 | name='pprofile', 25 | version=versioneer.get_version(), 26 | author='Vincent Pelletier', 27 | author_email='plr.vincent@gmail.com', 28 | description=next(x for x in description.splitlines() if x.strip()), 29 | long_description='.. contents::\n\n' + description, 30 | long_description_content_type='text/x-rst', 31 | url='http://github.com/vpelletier/pprofile', 32 | license='GPL 2+', 33 | platforms=['any'], 34 | classifiers=[ 35 | 'Intended Audience :: Developers', 36 | 'License :: OSI Approved :: GNU General Public License v2 or later (GPLv2+)', 37 | 'Operating System :: OS Independent', 38 | 'Programming Language :: Python :: 2.7', 39 | 'Programming Language :: Python :: 3', 40 | 'Programming Language :: Python :: Implementation :: PyPy', 41 | 'Programming Language :: Python :: Implementation :: CPython', 42 | 'Programming Language :: Python :: Implementation :: IronPython', 43 | 'Topic :: Software Development', 44 | ], 45 | packages=['pprofile'], 46 | py_modules=['zpprofile'], 47 | entry_points={ 48 | 'console_scripts': [ 49 | 'pprofile=pprofile:main', 50 | ], 51 | }, 52 | zip_safe=True, 53 | cmdclass=versioneer.get_cmdclass(), 54 | ) 55 | -------------------------------------------------------------------------------- /test.sh: -------------------------------------------------------------------------------- 1 | #!/bin/sh 2 | set -eu 3 | repository="$(dirname "$0")" 4 | cd "$repository" 5 | for python in /usr/bin/python /usr/bin/python3 /usr/bin/pypy3; do 6 | echo "Testing with $python" 7 | testdir="$(mktemp --tmpdir --directory "pprofile_tests.XXXXXXXX")" 8 | trap 'rm -r "$testdir"' EXIT 9 | virtualenv -p "$python" "$testdir" 10 | "${testdir}/bin/pip" install "$repository" 11 | pprofile="${testdir}/bin/pprofile" 12 | 13 | "$pprofile" --include demo --threads 0 demo/threads.py 14 | "$pprofile" --include demo --format callgrind demo/threads.py 15 | "$pprofile" --include demo --statistic .01 demo/threads.py 16 | "${testdir}/bin/python" demo/embedded.py 17 | "$pprofile" --include demo demo/threads.py 18 | "$pprofile" --include demo demo/empty.py 19 | "$pprofile" --format callgrind demo/empty.py 20 | "$pprofile" --include demo --statistic .01 demo/empty.py 21 | "$pprofile" --format callgrind --zipfile "${testdir}/source_code.zip" demo/threads.py 22 | "$pprofile" --format callgrind --zipfile "${testdir}/source_code.zip" demo/empty.py 23 | "$pprofile" --exclude-syspath demo/threads.py 24 | "$pprofile" --exclude-syspath --statistic .01 demo/threads.py 25 | "$pprofile" --include demo demo/encoding.py 26 | LC_CTYPE=ISO-8859-15 "$pprofile" --include demo demo/encoding.py 27 | "$pprofile" --include demo demo/encoding.py > /dev/null 28 | "$pprofile" --include demo demo/empty.py -search 29 | "$pprofile" --include demo -- demo/empty.py -search 30 | "$pprofile" --include demo demo/recurse.py 31 | "$pprofile" --include demo demo/recurse2.py 32 | "$pprofile" --include demo demo/recurse3.py 33 | "$pprofile" --include demo demo/recurse4.py 34 | "$pprofile" --include demo demo/twocalls.py 35 | "$pprofile" --include demo demo/twocalls2.py 36 | "$pprofile" --include demo demo/the_main.py 37 | "$pprofile" --include demo demo/module_globals.py 38 | "$pprofile" --format callgrindzip --out "${testdir}/test_threads.zip" demo/threads.py && unzip -l "${testdir}/test_threads.zip" 39 | "$pprofile" --out "${testdir}/test_threads.zip" demo/threads.py && unzip -l "${testdir}/test_threads.zip" 40 | 41 | trap - EXIT 42 | rm -r "$testdir" 43 | done 44 | echo 'Success' 45 | -------------------------------------------------------------------------------- /zpprofile.py: -------------------------------------------------------------------------------- 1 | from pprofile.zope import * 2 | --------------------------------------------------------------------------------