├── .gitignore ├── LICENSE.txt ├── MANIFEST.in ├── README.md ├── README.rst ├── _errator.pyx ├── devbuild.bat ├── devbuild.bsh ├── docker_build.bsh ├── docs ├── building-howto.txt ├── demo.py ├── index.html ├── using_errator.html ├── using_errator.rst └── verbose.py ├── errator.py ├── manylinux_build.bsh ├── manylinux_build_py38.bsh ├── requirements.txt ├── setup.cfg ├── setup.py ├── tests.py ├── timing.py └── win_build_all.bat /.gitignore: -------------------------------------------------------------------------------- 1 | /wheels/errator-0.3.3-cp36-cp36m-manylinux1_x86_64.whl 2 | /wheels/errator-0.3.3-cp36-cp36m-manylinux2010_x86_64.whl 3 | /wheels/errator-0.3.3-cp37-cp37m-manylinux1_x86_64.whl 4 | /wheels/errator-0.3.3-cp37-cp37m-manylinux2010_x86_64.whl 5 | /wheels/errator-0.3.3-cp39-cp39-manylinux1_x86_64.whl 6 | /wheels/errator-0.3.3-cp39-cp39-manylinux2010_x86_64.whl 7 | /build/ 8 | /wheels/errator-0.3.3-cp38-cp38-manylinux1_x86_64.whl 9 | /_errator.cpython-38-x86_64-linux-gnu.so 10 | /_errator.c 11 | /wheels/ 12 | -------------------------------------------------------------------------------- /LICENSE.txt: -------------------------------------------------------------------------------- 1 | The MIT License (MIT) 2 | 3 | Copyright (c) 2016 Tom Carroll 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /MANIFEST.in: -------------------------------------------------------------------------------- 1 | global-include *.pyx 2 | include README.rst 3 | include docs/using_errator.rst 4 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | errator 2 | ======= 3 | 4 | Provide human-readable error narration of exception tracebacks with `errator`. 5 | 6 | 1. [What's new in 0.3](#what-s-new-in-0-3) 7 | 2. [Intro](#intro) 8 | 3. [How it works](#how-it-works) 9 | 4. [Requirements](#requirements) 10 | 5. [Installing](#installing) 11 | 6. [Quick Start](#quick-start) 12 | 7. [Building from source](#building-from-source) 13 | 14 | What's new in 0.3 15 | ----------------- 16 | 17 | This is largely a performance release for `errator`, as it implements the more costly part of the narration process in a [Cython](http://cython.org/) extension. Additionally, there has been a change in the way verbose narrations are performed, as the original approach entailed a lot of overhead even if verbose narrations weren't required. 18 | 19 | - `errator` now consists of a Python file and a Python extension module (built from a Cython source), which improves runtime performance 20 | - The 'verbose' keyword argument has been removed from the `get_narration()` function, and has been added to the `set_narration_options()` and `set_default_options()` functions. Now getting verbose narrations is a matter of setting the option either as a default or on a per-thread basis, hence this option must be set before collecting narrations. 21 | - Previously, detailed exception information (file name, line number) was always collected in case a user wanted to fetch verbose narration information with `get_narration()`. However, this proved to be a big performance hit, and now this information is only collected if the 'verbose' option has been set with `set_narration_options()` or `set_default_options()` function before the exception occurs. 22 | 23 | What's new in 0.2.1 24 | ------------------- 25 | 26 | This patch release addresses certain performance issues: 27 | 28 | - Patched a bug that caused too many object creations to occur 29 | - Removed some redundant resetting of state in narration fragment objects 30 | - Added caching of filenames for function objects so that lookups aren't always required 31 | - Sped up the resetting process 32 | 33 | What's new in 0.2 34 | ----------------- 35 | 36 | - The `get_narration()` now has a new keyword argument, "verbose" (default False), that when True returns expanded narration information that includes the line number, function, and file name of the point in the stack trace that the narration applies to. 37 | - To provide a more tidy display of stack information, `errator` now has analogs of the functions from the standard `traceback` module that filter out `errator`-based calls from the call stack, leaving only application calls in the display of stack traces. 38 | - Narration output formatting has been modified slightly. 39 | 40 | Intro 41 | ----- 42 | 43 | `errator` came as an idea on the back of trying to figure out what the semantics of an exception traceback are in non-trivial pieces of code. 44 | 45 | When an exception occurs deep inside a call stack within some generic utility function that is used in numerous contexts, reading the traceback is often not helpful in determining the source of the problem. Data values aren't obvious, and the initial starting conditions of the error can't easily been seen. 46 | 47 | Logging is a step in the right direction, but in general outputs too much information; often, there is lots of info regarding error-free processing in the log, making it hard to find the right log output that is associated with the error. 48 | 49 | `errator` is something of a marriage between logging and tracebacks: plain text messages that are associated directly with the call trail that led to an exception. `errator` works by providing tools that let you state intent of code in text, but only captures that text when an exception bubbles up the stack. You can then acquire this "error narration" and display it to your user in the most appropriate fashion. 50 | 51 | How it works 52 | ------------ 53 | 54 | `errator` uses decorators and context managers to maintain a stack of "narration fragments" behind the scenes. When code executes without exceptions, these fragments are created and thrown away as narrated code executes and returns. However, when an exception is raised, the narration fragments are retained and their content can be retrieved. Fragments can be automatically discarded (pruned) when an exception doesn't propagate any further up the stack, or can be discarded under user control, allowing more control over the content of "narration" provided for the exception. 55 | 56 | `errator` is thread-safe, allowing you to capture separate error narrations for independent threads of control through the same code. 57 | 58 | Requirements 59 | ------------ 60 | 61 | `errator` doesn't have any runtime external dependencies, and is compatible with Python 2.7 and 3.x. If you wish to build from source, you'll need to install Cython to build the extension module, and have a C compiler and the required Python header files. 62 | 63 | Installing 64 | ---------- 65 | 66 | As of 0.3, `errator` is comprised of a Python module and an extension wrapper. You can install it with `pip install errator`, or you can clone the Git project and build from source (more on this below). 67 | 68 | Quick Start 69 | ----------- 70 | 71 | The next section discusses `errator` with functions, but you can also use the decorators described with methods too. 72 | 73 | **Basic Use** 74 | 75 | Start by importing `errator` into the module that you want to narrate: 76 | 77 | ``` sourceCode 78 | from errator import * 79 | ``` 80 | 81 | Now, suppose you have a utility function that performs some specialized string formatting, but it is possible to pass in arguments that cause a exception to be raised. Your function is called all over the place for a variety of different reasons, often very deep down the call stack where it isn't obvious what the original functional intent was, or where the source of bad arguments may have been. 82 | 83 | To start building the narration to your function's execution, you can use the `narrate()` decorator to associate a bit of text with your utility function in order to provide easily understandable explanations about what's going on: 84 | 85 | ``` sourceCode 86 | @narrate("I'm trying to format a string") 87 | def special_formatter(fmt_string, **kwargs): 88 | # magic format code that sometimes raises an exception 89 | ``` 90 | 91 | The `narrate()` decorator knows to look for exceptions and doesn't impede their propagation, but captures that bit of text in an internal stack when an exception occurs. So if you write: 92 | 93 | ``` sourceCode 94 | try: 95 | s = special_formatter(fmt, **args) 96 | exception Exception: 97 | the_tale = get_narration() 98 | ``` 99 | 100 | ...and `special_formatter()` raises an exception, the exception will still bubble up the stack, but `get_narration()` will return a list of strings for all the `narrate()`-decorated functions down to the exception. If no exception is raised, there are no strings to fetch (unless you want there to be strings, but we'll get to that). 101 | 102 | **Getting more information** 103 | 104 | Maybe you'd like some insight as to the value of the arguments passed when an exception is raised, so you can better tell what's causing it. Instead of a string, you can supply the `narrate()` decorator with a callable that returns a string and that has the same signature as the function being decorated. This callable will only be invoked if the decorated function raises an exception, and gets invoked with the same arguments as the function: 105 | 106 | ``` sourceCode 107 | @narrate(lambda fs, **kw: "I'm trying to format a string with '%s' and args '%s'" % (fs, str(kw))) 108 | def special_formatter(fmt_string, **kwargs): 109 | # magic format code that sometimes raises an exception 110 | ``` 111 | 112 | The lambda passed to narrate() will only be called when `special_formatter()` raises an exception, otherwise it will go un-executed. 113 | 114 | **Finer details with contexts** 115 | 116 | Now, perhaps `special_formatter()` is a rather long function, and you'd like to be able to narrate it's operation in more detail to get better narrations when things go wrong. You can use the `narrate_cm()` context manager to create a narration fragment for a block of code. If everything goes well in the block, then the fragment is discarded, but the fragment will be retained if an exception occurs: 117 | 118 | ``` sourceCode 119 | def special_formatter(fmt_string, **kwargs): 120 | for format_token in parse_format(fmt_string): 121 | if format_token.type == float: 122 | with narrate_cm("I started processing a float format"): 123 | # do magic stuff for floats... 124 | elif format_token.type == int: 125 | with narrate_cm("I started processing an int format"): 126 | # do magic stuff for ints... 127 | ``` 128 | 129 | Narration fragments added with `narrate_cm()` are treated just like those created by the function decorator-- they are added to the stack, and silently removed if the context manager's code block exits normally. But exceptions raised in the context block are retained as the exception propagates back through the stack. 130 | 131 | Like `narrate()`, `narrate\_cm()` allows you to supply a callable instead of a string: 132 | 133 | ``` sourceCode 134 | with narrate_cm(lambda x: "I started processing an int with format %s" % x, format_token.format): 135 | # format code 136 | ``` 137 | 138 | ...and again, this callable will only be invoked if an exception is raised in the context. Unlike `narrate()`, however, you are free to define a callable with any signature, as long as you supply the arguments needed to invoke the callable if need be. 139 | 140 | Context managers may nest, and in fact any combination of function decorator and context manager will work as expected. 141 | 142 | **A larger example** 143 | 144 | Let's look at an example with more complex calling relationships. Suppose we have functions `A`, `B`, `C`, `D`, `E`, and `F`. They have the following calling relationships: 145 | 146 | - `A` calls `B` then `C` 147 | - `B` calls `D` 148 | - `C` calls `E` or `F` 149 | - `D` calls `F` 150 | 151 | We'll make it so that if we're unlucky enough to call `E`, we'll get an exception raised. This will happen only for input values of `A` greater than 10. 152 | 153 | So let's define these functions and narrate them-- paste these into an interactive Python session after you've imported `errator`: 154 | 155 | ``` sourceCode 156 | @narrate(lambda v: "I'm trying to A with %s as input" % v) 157 | def A(val): 158 | B(val / 2) 159 | C(val * 2) 160 | 161 | @narrate(lambda v: "I'm trying to B with %s as input" % v) 162 | def B(val): 163 | D(val * 10) 164 | 165 | @narrate(lambda v: "I'm trying to C with %s as input" % v) 166 | def C(val): 167 | if val > 20: 168 | E(val) 169 | else: 170 | F(val) 171 | 172 | @narrate(lambda v: "I'm trying to D with %s as input" % v) 173 | def D(val): 174 | F(val * 3) 175 | 176 | @narrate(lambda v: "I'm trying to E with %s as input" % v) 177 | def E(val): 178 | raise ValueError("how dare you call me with such a value?") 179 | 180 | @narrate(lambda v: "I'm trying to F with %s as input" % v) 181 | def F(val): 182 | print("very well") 183 | ``` 184 | 185 | Now run `A` with a value less than 11, and look for narration text: 186 | 187 | ``` sourceCode 188 | >>> A(3) 189 | very well 190 | very well 191 | >>> get_narration() 192 | [] 193 | >>> 194 | ``` 195 | 196 | Since there was no exception, there are no narrations. Now run `A` with a value greater than 10, which will cause an exception in E: 197 | 198 | ``` sourceCode 199 | >>> A(11) 200 | very well 201 | Traceback (most recent call last): 202 | File "", line 1, in 203 | File "errator.py", line 322, in callit 204 | _v = m(*args, **kwargs) 205 | File "", line 4, in A 206 | File "errator.py", line 322, in callit 207 | _v = m(*args, **kwargs) 208 | File "", line 4, in C 209 | File "errator.py", line 322, in callit 210 | _v = m(*args, **kwargs) 211 | File "", line 3, in E 212 | ValueError: how dare you call me with such a value? 213 | >>> 214 | ``` 215 | 216 | So far, it's as we'd expect, except perhaps for the inclusion of `errator` calls in the stack (`errator` includes tools that allow you to get stack traces that have been cleaned of `errator` calls). But now let's look at the narration: 217 | 218 | ``` sourceCode 219 | >>> for l in get_narration(): 220 | ... print(l) 221 | ... 222 | I'm trying to A with 11 as input 223 | I'm trying to C with 22 as input 224 | I'm trying to E with 22 as input, but exception type: ValueError, value: how dare you call me with such a value? was raised 225 | >>> 226 | ``` 227 | 228 | We have a narration for our recent exception. Now try the following: 229 | 230 | ``` sourceCode 231 | >>> A(8) 232 | very well 233 | very well 234 | >>> get_narration() 235 | ["I'm trying to A with 11 as input", "I'm trying to C with 22 as input", # etc... 236 | ``` 237 | 238 | Wait, this didn't have an exception; why is there still error narration? This is because *an error narration only gets cleared out if a decorated function does NOT have an exception bubble up*; the assumption is that the exception was caught and the narration was retrieved, so a decorated function that returns normally would remove the previous narration fragments. In our example, there is no function that is decorated with `narrate()` that catches the exception and returns normally, so the narration never clears out. 239 | 240 | There are a few ways to clear unwanted narrations: first is to manually clear the narration, and the other is to make sure you have a decorated function that catches the exception and returns normally, which will clear the narration automatically 241 | 242 | To manually clear narrations we call `reset_narration()`: 243 | 244 | ``` sourceCode 245 | >>> reset_narration() 246 | >>> get_narration() 247 | >>> [] 248 | ``` 249 | 250 | For the second, if we define a decorated function that calls A but which handles the exception and returns normally, the narration fragments will be cleaned up automatically: 251 | 252 | ``` sourceCode 253 | @narrate("Handler for A") 254 | def first(val): 255 | try: 256 | A(val) 257 | except: 258 | print("Got %d narration lines" % len(get_narration())) 259 | ``` 260 | 261 | This outermost function still can retrieve the narration, but as it returns normally, the narration is cleared out when it returns: 262 | 263 | ``` sourceCode 264 | >>> first(11) 265 | very well 266 | Got 4 narration lines 267 | >>> get_narration() 268 | [] 269 | >>> 270 | ``` 271 | 272 | `errator` provides various narration options and finer degrees of control for retriving the narration; these are covered in the detailed docs. See the `using_errator` file in the docs directory. 273 | 274 | Building from source 275 | -------------------- 276 | 277 | `errator` is built using Cython, however the source package contains the generated C file, so you only need: 278 | 279 | - A C compiler 280 | - Python header files for your version of Python 281 | 282 | For development, in the project root, run the following command: 283 | 284 | ``` sourceCode 285 | python setup.py build_ext --inplace 286 | ``` 287 | 288 | ...This will create the shared library that is used by `errator`. You can then do the normal `python setup.py install` dance to put the built distribution where you want it to go, or you can simply use it right from where you built it. 289 | 290 | If you want to build a wheel, the command is: 291 | 292 | ``` sourceCode 293 | python setup.py bdist_wheel 294 | ``` 295 | 296 | If you wish to build from the Cython pyx file, you'll need to grab the source from the Github repo and run the same commands as above; they will run Cython when appropriate. 297 | -------------------------------------------------------------------------------- /README.rst: -------------------------------------------------------------------------------- 1 | errator 2 | ======= 3 | 4 | Provide human-readable error narration of exception tracebacks with ``errator``. 5 | 6 | #. `What's new in 0.4 <#what-s-new-in-0-3>`__ 7 | #. `Intro <#intro>`__ 8 | #. `How it works <#how-it-works>`__ 9 | #. `Requirements <#requirements>`__ 10 | #. `Installing <#installing>`__ 11 | #. `Quick Start <#quick-start>`__ 12 | #. `Building from source <#building-from-source>`__ 13 | 14 | What's new in 0.4 15 | ----------------- 16 | This has largely been a maintenance release which focused on two aspects: 17 | 18 | - Get the 'manylinux' builds in order. 19 | 20 | - Add type annotations where it made sense. 21 | 22 | However, it also fixed a bug where if a fragment formatting function raised an exception 23 | it would not provide any data for that fragment (and possibly any others). 24 | 25 | There is also a new capability: you can now add a `tags` keyword argument to both 26 | `narrate()` and `narrate_cm()` to allow selective retrieval of narration fragments 27 | with the `get_narration()` function using the `with_tags` keyword argument to that 28 | function. In either case, the tags are a list of strings. If `narrate()` or `narrate_cm()` 29 | are used without tags, then their fragments are always included in the return from 30 | `get_narration()`. If `get_narration()` doesn't specify any tags, then every current 31 | fragement is returned regards of the absence or presence of tags. Otherwise, when using 32 | `get_narration()` with tags, a fragment must have one or more tags in common with those 33 | specified in `get_narration()` in order to be returned from that function. 34 | 35 | What's new in 0.3 36 | ----------------- 37 | This is largely a performance release for ``errator``, as it implements the more costly part of the narration process in a `Cython `__ extension. Additionally, there has been a change in the way verbose narrations are performed, as the original approach entailed a lot of overhead even if verbose narrations weren't required. 38 | 39 | - ``errator`` now consists of a Python file and a Python extension module (built from a Cython source), which improves runtime performance 40 | 41 | - The 'verbose' keyword argument has been removed from the ``get_narration()`` function, and has been added to the ``set_narration_options()`` and ``set_default_options()`` functions. Now getting verbose narrations is a matter of setting the option either as a default or on a per-thread basis, hence this option must be set `before` collecting narrations. 42 | 43 | - Previously, detailed exception information (file name, line number) was always collected in case a user wanted to fetch verbose narration information with ``get_narration()``. However, this proved to be a big performance hit, and now this information is `only` collected if the 'verbose' option has been set with ``set_narration_options()`` or ``set_default_options()`` function `before` the exception occurs. 44 | 45 | What's new in 0.2.1 46 | ------------------- 47 | 48 | This patch release addresses certain performance issues: 49 | 50 | - Patched a bug that caused too many object creations to occur 51 | 52 | - Removed some redundant resetting of state in narration fragment objects 53 | 54 | - Added caching of filenames for function objects so that lookups aren't always required 55 | 56 | - Sped up the resetting process 57 | 58 | 59 | What's new in 0.2 60 | ----------------- 61 | 62 | - The ``get_narration()`` now has a new keyword argument, "verbose" (default False), that when True returns expanded narration information that includes the line number, function, and file name of the point in the stack trace that the narration applies to. 63 | 64 | - To provide a more tidy display of stack information, ``errator`` now has analogs of the functions from the standard ``traceback`` module that filter out ``errator``-based calls from the call stack, leaving only application calls in the display of stack traces. 65 | 66 | - Narration output formatting has been modified slightly. 67 | 68 | Intro 69 | ----- 70 | 71 | ``errator`` came as an idea on the back of trying to figure out what the semantics of an exception traceback are in non-trivial pieces of code. 72 | 73 | When an exception occurs deep inside a call stack within some generic utility function that is used in numerous contexts, reading the traceback is often not helpful in determining the source of the problem. Data values aren't obvious, and the initial starting conditions of the error can't easily been seen. 74 | 75 | Logging is a step in the right direction, but in general outputs too much information; often, there is lots of info regarding error-free processing in the log, making it hard to find the right log output that is associated with the error. 76 | 77 | ``errator`` is something of a marriage between logging and tracebacks: plain text messages that are associated directly with the call trail that led to an exception. ``errator`` works by providing tools that let you state intent of code in text, but only captures that text when an exception bubbles up the stack. You can then acquire this "error narration" and display it to your user in the most appropriate fashion. 78 | 79 | How it works 80 | ------------ 81 | 82 | ``errator`` uses decorators and context managers to maintain a stack of "narration fragments" behind the scenes. When code executes without exceptions, these fragments are created and thrown away as narrated code executes and returns. However, when an exception is raised, the narration fragments are retained and their content can be retrieved. Fragments can be automatically discarded (pruned) when an exception doesn't propagate any further up the stack, or can be discarded under user control, allowing more control over the content of "narration" provided for the exception. 83 | 84 | ``errator`` is thread-safe, allowing you to capture separate error narrations for independent threads of control through the same code. 85 | 86 | Requirements 87 | ------------ 88 | 89 | ``errator`` doesn't have any runtime external dependencies, and is compatible with Python 2.7 and 3.x. If you wish to build from source, you'll need to install Cython to build the extension module, and have a C compiler and the required Python header files. 90 | 91 | Installing 92 | ---------- 93 | 94 | As of 0.3, ``errator`` is comprised of a Python module and an extension. You can install it with ``pip install errator``, or you can clone the Git project and build from source (more on this below). 95 | 96 | Quick Start 97 | ----------- 98 | 99 | The next section discusses ``errator`` with functions, but you can also use the decorators described with methods too. 100 | 101 | **Basic Use** 102 | 103 | Start by importing ``errator`` into the module that you want to narrate: 104 | 105 | .. code:: python 106 | 107 | from errator import * 108 | 109 | Now, suppose you have a utility function that performs some specialized string formatting, but it is possible to pass in arguments that cause a exception to be raised. Your function is called all over the place for a variety of different reasons, often very deep down the call stack where it isn't obvious what the original functional intent was, or where the source of bad arguments may have been. 110 | 111 | To start building the narration to your function's execution, you can use the ``narrate()`` decorator to associate a bit of text with your utility function in order to provide easily understandable explanations about what's going on: 112 | 113 | .. code:: python 114 | 115 | @narrate("I'm trying to format a string") 116 | def special_formatter(fmt_string, **kwargs): 117 | # magic format code that sometimes raises an exception 118 | 119 | The ``narrate()`` decorator knows to look for exceptions and doesn't impede their propagation, but captures that bit of text in an internal stack when an exception occurs. So if you write: 120 | 121 | .. code:: python 122 | 123 | try: 124 | s = special_formatter(fmt, **args) 125 | exception Exception: 126 | the_tale = get_narration() 127 | 128 | ...and ``special_formatter()`` raises an exception, the exception will still bubble up the stack, but ``get_narration()`` will return a list of strings for all the ``narrate()``-decorated functions down to the exception. If no exception is raised, there are no strings to fetch (unless you want there to be strings, but we'll get to that). 129 | 130 | **Getting more information** 131 | 132 | Maybe you'd like some insight as to the value of the arguments passed when an exception is raised, so you can better tell what's causing it. Instead of a string, you can supply the ``narrate()`` decorator with a callable that returns a string and that has the same signature as the function being decorated. This callable will `only be invoked if the decorated function raises an exception`, and gets invoked with the same arguments as the function: 133 | 134 | .. code:: python 135 | 136 | @narrate(lambda fs, **kw: "I'm trying to format a string with '%s' and args '%s'" % (fs, str(kw))) 137 | def special_formatter(fmt_string, **kwargs): 138 | # magic format code that sometimes raises an exception 139 | 140 | The lambda passed to narrate() will only be called when ``special_formatter()`` raises an exception, otherwise it will go un-executed. 141 | 142 | **Finer details with contexts** 143 | 144 | Now, perhaps ``special_formatter()`` is a rather long function, and you'd like to be able to narrate it's operation in more detail to get better narrations when things go wrong. You can use the ``narrate_cm()`` context manager to create a narration fragment for a block of code. If everything goes well in the block, then the fragment is discarded, but the fragment will be retained if an exception occurs: 145 | 146 | .. code:: python 147 | 148 | def special_formatter(fmt_string, **kwargs): 149 | for format_token in parse_format(fmt_string): 150 | if format_token.type == float: 151 | with narrate_cm("I started processing a float format"): 152 | # do magic stuff for floats... 153 | elif format_token.type == int: 154 | with narrate_cm("I started processing an int format"): 155 | # do magic stuff for ints... 156 | 157 | Narration fragments added with ``narrate_cm()`` are treated just like those created by the function decorator-- they are added to the stack, and silently removed if the context manager's code block exits normally. But exceptions raised in the context block are retained as the exception propagates back through the stack. 158 | 159 | Like ``narrate()``, ``narrate\_cm()`` allows you to supply a callable instead of 160 | a string: 161 | 162 | .. code:: python 163 | 164 | with narrate_cm(lambda x: "I started processing an int with format %s" % x, format_token.format): 165 | # format code 166 | 167 | ...and again, this callable will only be invoked if an exception is raised in the context. Unlike ``narrate()``, however, you are free to define a callable with any signature, as long as you supply the arguments needed to invoke the callable if need be. 168 | 169 | Context managers may nest, and in fact any combination of function decorator and context manager will work as expected. 170 | 171 | **A larger example** 172 | 173 | Let's look at an example with more complex calling relationships. Suppose we have functions ``A``, ``B``, ``C``, ``D``, ``E``, and ``F``. They have the following calling relationships: 174 | 175 | 176 | * ``A`` calls ``B`` then ``C`` 177 | * ``B`` calls ``D`` 178 | * ``C`` calls ``E`` or ``F`` 179 | * ``D`` calls ``F`` 180 | 181 | 182 | We'll make it so that if we're unlucky enough to call ``E``, we'll get an exception raised. This will happen only for input values of ``A`` greater than 10. 183 | 184 | So let's define these functions and narrate them-- paste these into an interactive Python session after you've imported ``errator``: 185 | 186 | .. code:: python 187 | 188 | @narrate(lambda v: "I'm trying to A with %s as input" % v) 189 | def A(val): 190 | B(val / 2) 191 | C(val * 2) 192 | 193 | @narrate(lambda v: "I'm trying to B with %s as input" % v) 194 | def B(val): 195 | D(val * 10) 196 | 197 | @narrate(lambda v: "I'm trying to C with %s as input" % v) 198 | def C(val): 199 | if val > 20: 200 | E(val) 201 | else: 202 | F(val) 203 | 204 | @narrate(lambda v: "I'm trying to D with %s as input" % v) 205 | def D(val): 206 | F(val * 3) 207 | 208 | @narrate(lambda v: "I'm trying to E with %s as input" % v) 209 | def E(val): 210 | raise ValueError("how dare you call me with such a value?") 211 | 212 | @narrate(lambda v: "I'm trying to F with %s as input" % v) 213 | def F(val): 214 | print("very well") 215 | 216 | Now run ``A`` with a value less than 11, and look for narration text: 217 | 218 | .. code:: python 219 | 220 | >>> A(3) 221 | very well 222 | very well 223 | >>> get_narration() 224 | [] 225 | >>> 226 | 227 | Since there was no exception, there are no narrations. Now run ``A`` with a value greater than 10, which will cause an exception in E: 228 | 229 | .. code:: python 230 | 231 | >>> A(11) 232 | very well 233 | Traceback (most recent call last): 234 | File "", line 1, in 235 | File "errator.py", line 322, in callit 236 | _v = m(*args, **kwargs) 237 | File "", line 4, in A 238 | File "errator.py", line 322, in callit 239 | _v = m(*args, **kwargs) 240 | File "", line 4, in C 241 | File "errator.py", line 322, in callit 242 | _v = m(*args, **kwargs) 243 | File "", line 3, in E 244 | ValueError: how dare you call me with such a value? 245 | >>> 246 | 247 | So far, it's as we'd expect, except perhaps for the inclusion of ``errator`` calls in the stack (``errator`` includes tools that allow you to get stack traces that have been cleaned of ``errator`` calls). But now let's look at the narration: 248 | 249 | .. code:: 250 | 251 | >>> for l in get_narration(): 252 | ... print(l) 253 | ... 254 | I'm trying to A with 11 as input 255 | I'm trying to C with 22 as input 256 | I'm trying to E with 22 as input, but exception type: ValueError, value: how dare you call me with such a value? was raised 257 | >>> 258 | 259 | We have a narration for our recent exception. Now try the following: 260 | 261 | .. code:: python 262 | 263 | >>> A(8) 264 | very well 265 | very well 266 | >>> get_narration() 267 | ["I'm trying to A with 11 as input", "I'm trying to C with 22 as input", # etc... 268 | 269 | Wait, this didn't have an exception; why is there still error narration? This is because *an error narration only gets cleared out if a decorated function does NOT have an exception bubble up*; the assumption is that the exception was caught and the narration was retrieved, so a decorated function that returns normally would remove the previous narration fragments. In our example, there is no function that is decorated with ``narrate()`` that catches the exception and returns normally, so the narration never clears out. 270 | 271 | There are a few ways to clear unwanted narrations: first is to manually clear the narration, and the other is to make sure you have a decorated function that catches the exception and returns normally, which will clear the narration automatically 272 | 273 | To manually clear narrations we call ``reset_narration()``: 274 | 275 | .. code:: python 276 | 277 | >>> reset_narration() 278 | >>> get_narration() 279 | >>> [] 280 | 281 | For the second, if we define a decorated function that calls A but which handles the exception and returns normally, the narration fragments will be cleaned up automatically: 282 | 283 | .. code:: python 284 | 285 | @narrate("Handler for A") 286 | def first(val): 287 | try: 288 | A(val) 289 | except: 290 | print("Got %d narration lines" % len(get_narration())) 291 | 292 | This outermost function still can retrieve the narration, but as it returns normally, the narration is cleared out when it returns: 293 | 294 | .. code:: python 295 | 296 | >>> first(11) 297 | very well 298 | Got 4 narration lines 299 | >>> get_narration() 300 | [] 301 | >>> 302 | 303 | ``errator`` provides various narration options and finer degrees of control for retriving the narration; these are covered in the detailed docs. See the ``using_errator`` file in the docs directory. 304 | 305 | Building from source 306 | -------------------- 307 | 308 | ``errator`` is built using Cython, however the source package contains the generated C file, so you only need: 309 | 310 | - A C compiler 311 | - Python header files for your version of Python 312 | 313 | For development, in the project root, run the following command: 314 | 315 | .. code:: 316 | 317 | python setup.py build_ext --inplace 318 | 319 | ...This will create the shared library that is used by ``errator``. You can then do the normal ``python setup.py install`` dance to put the built distribution where you want it to go, or you can simply use it right from where you built it. 320 | 321 | If you want to build a wheel, the command is: 322 | 323 | .. code:: 324 | 325 | python setup.py bdist_wheel 326 | 327 | If you wish to build from the Cython pyx file, you'll need to grab the source from the Github repo and run the same commands as above; they will run Cython when appropriate. 328 | -------------------------------------------------------------------------------- /_errator.pyx: -------------------------------------------------------------------------------- 1 | from collections import deque, defaultdict 2 | import inspect 3 | import sys 4 | from threading import Thread, current_thread 5 | from typing import Iterable 6 | import traceback 7 | 8 | _default_options = {"auto_prune": True, 9 | "check": False, 10 | "verbose": False} 11 | 12 | 13 | class ErratorException(Exception): 14 | pass 15 | 16 | 17 | class ErratorDeque(deque): 18 | 19 | def __init__(self, iterable: Iterable = (), auto_prune: bool = None, 20 | check: bool = None, verbose:bool = None): 21 | super(ErratorDeque, self).__init__(iterable=iterable) 22 | self.__dict__.update(_default_options) 23 | if auto_prune is not None: 24 | self.auto_prune = bool(auto_prune) 25 | 26 | if check is not None: 27 | self.check = bool(check) 28 | 29 | if verbose is not None: 30 | self.verbose = bool(verbose) 31 | 32 | def set_check(self, value): 33 | """ 34 | sets the check flag to the provided boolean value 35 | :param value: interpreted as a boolean value for self.check; if None don't change 36 | the value 37 | :return: self 38 | """ 39 | if value is not None: 40 | self.check = bool(value) 41 | return self 42 | 43 | def set_auto_prune(self, value): 44 | """ 45 | sets the auto_prune flag to the provided boolean value 46 | :param value: interpreted as a boolean value for self.auto_prune; if None don't 47 | change the value 48 | :return: self 49 | """ 50 | if value is not None: 51 | self.auto_prune = bool(value) 52 | return self 53 | 54 | def set_verbose(self, value): 55 | """ 56 | sets the verbose flag to the provided boolean value 57 | :param value: interpreted as a boolean value for self.verbose; if None don't 58 | change 59 | :return: self 60 | """ 61 | if value is not None: 62 | self.verbose = bool(value) 63 | return self 64 | 65 | def pop_until_true(self, f): 66 | """ 67 | Performs pop(right) from the deque up to and including the element for which f 68 | returns True 69 | 70 | This method tests the last element in the deque (right end) using the supplied 71 | function f. If f returns False for the element, the element is popped and the 72 | test repeated for new last element. If f returns True, that element is popped 73 | and the method returns. If f never returns True, then all elements will be 74 | popped from the list. 75 | 76 | :param f: callable of one argument, an item on the deque. Returns True if the 77 | item is the last one to pop from the deque, False otherwise. 78 | :return: None 79 | """ 80 | selfpop = self.pop 81 | while self and not f(self[-1]): 82 | inst = selfpop() 83 | inst.__class__.return_instance(inst) 84 | if self: 85 | inst = selfpop() 86 | inst.__class__.return_instance(inst) 87 | return 88 | 89 | 90 | # _thread_fragments is hashed by a thread's name and contains a deque NarrationFragment 91 | # for each frame in the thread's call path 92 | _thread_fragments = defaultdict(ErratorDeque) 93 | 94 | 95 | cdef class NarrationFragment(object): 96 | # CYTHON 97 | cdef public text_or_func 98 | cdef public tuple args 99 | cdef public dict kwargs 100 | cdef public str exception_text 101 | cdef public calling 102 | cdef public int status 103 | cdef public str func_name, source_file 104 | cdef public int lineno 105 | cdef public frozenset tags 106 | # CYTHON 107 | 108 | IN_PROCESS = 1 109 | RAISED_EXCEPTION = 2 110 | PASSEDTHRU_EXCEPTION = 3 111 | COMPLETED = 4 112 | 113 | _free_instances = deque() 114 | 115 | _callable_id_to_filename = {} 116 | 117 | _empty_set = frozenset() 118 | 119 | 120 | @classmethod 121 | def get_instance(cls, text_or_func, narrated_callable, *args, **kwargs): 122 | cdef NarrationFragment inst 123 | try: 124 | inst = cls._free_instances.pop() 125 | inst.__init__(text_or_func, narrated_callable, *args, **kwargs) 126 | except IndexError: 127 | inst = cls(text_or_func, narrated_callable, *args, **kwargs) 128 | return inst 129 | 130 | @classmethod 131 | def return_instance(cls, inst): 132 | cls._free_instances.append(inst) 133 | 134 | def __init__(self, text_or_func, narrated_callable, *args, **kwargs): 135 | """ 136 | Creates a new NarrationFragment that will report using the supplied text or func 137 | :param text_or_func: either a string or a callable with the same signature as the 138 | callable being decorated 139 | :param narrated_callable: the callable being decorated or None. If supplied, then 140 | the callable will be inspected and some metadata on it will be saved 141 | :param args: possibly empty sequence of additional arguments 142 | :param kwargs: possibly empty dictionary of keyword arguments 143 | """ 144 | cdef long ncid 145 | cdef str str 146 | self.text_or_func = text_or_func 147 | self.args = args 148 | self.kwargs = kwargs if kwargs else {} 149 | self.exception_text = None 150 | self.calling = None 151 | self.status = self.IN_PROCESS 152 | self.func_name = None 153 | self.source_file = None 154 | self.lineno = 0 155 | self.tags = self._empty_set 156 | 157 | cpdef set_tags(self, tags: frozenset): 158 | self.tags = tags 159 | 160 | cdef bint are_tags_disjoint(self, frozenset other_tags): 161 | return self.tags.isdisjoint(other_tags) 162 | 163 | cdef bint any_tags(self): 164 | return len(self.tags) != 0 165 | 166 | cpdef bint frame_describes_func(self, frame): 167 | """ 168 | returns True if the supplied tuple from inspect.stack/trace matches the 169 | function and file name for this fragment 170 | :param frame: 171 | :return: 172 | """ 173 | return self.func_name == frame[3] and self.source_file == frame[1] 174 | 175 | cpdef annotate_fragment(self, frame): 176 | """ 177 | Extract relevant info from supplied FrameInfo object 178 | :param frame: 179 | :return: 180 | """ 181 | self.lineno = frame[2] 182 | 183 | @classmethod 184 | def clone(cls, NarrationFragment src): 185 | cdef NarrationFragment new = cls(src.text_or_func, None, 186 | *src.args if src.args is not None else (), 187 | **src.kwargs if src.kwargs is not None else {}) 188 | new.exception_text = src.exception_text 189 | new.calling = src.calling 190 | new.func_name = src.func_name 191 | new.source_file = src.source_file 192 | new.lineno = src.lineno 193 | return new 194 | 195 | cpdef str format(self, bint verbose=False, bint best_effort_return=False): 196 | cdef str result 197 | cdef str tale 198 | 199 | try: 200 | tale = (self.text_or_func(*self.args, **self.kwargs) 201 | if callable(self.text_or_func) 202 | else self.text_or_func) 203 | 204 | self.args = self.kwargs = None 205 | 206 | if self.exception_text: 207 | tale = "{}, but {} was raised".format(tale, self.exception_text) 208 | self.exception_text = None 209 | self.text_or_func = tale 210 | 211 | if verbose and self.func_name: 212 | if self.lineno is not None: 213 | result = "\n".join([tale, " line %s in %s, %s" % 214 | (str(self.lineno), 215 | str(self.func_name), 216 | str(self.source_file))]) 217 | else: 218 | result = "\n".join([tale, "%s in %s" % (str(self.func_name), 219 | str(self.source_file))]) 220 | else: 221 | result = tale 222 | except Exception as _: 223 | if not best_effort_return: 224 | raise 225 | etype, val, tb = sys.exc_info() 226 | nested_result = list() 227 | prefix = "\t>>>> " 228 | nested_result.append(f"{prefix}EXCEPTION DURING ERRATOR " 229 | f"FRAGMENT FORMATTING for {self.func_name}") 230 | nested_result.append(f"{prefix}A fragment formatting callable raised " 231 | f"exception type {etype}, value '{val}' while errator " 232 | f"was processing another exception from" 233 | f" '{self.func_name}'") 234 | nested_result.append(f"{prefix}file {self.source_file}, line {self.lineno}") 235 | nested_result.append(f"{prefix}The details are:") 236 | for fs in traceback.extract_tb(tb): 237 | nested_result.append(f"{prefix} line {fs.lineno} in {fs.filename}:" 238 | f"\n{prefix} {fs.line}") 239 | nested_result.append(f"{prefix}Processing the outer exception " 240 | f"now continues") 241 | result = '\n'.join(nested_result) 242 | 243 | return result 244 | 245 | cpdef str tell(self, verbose=False): 246 | cdef str tale = self.format(verbose=verbose, best_effort_return=True) 247 | return tale 248 | 249 | cpdef fragment_exception_text(self, etype, text): 250 | self.exception_text = "exception type: {}, value: '{}'".format(etype.__name__, 251 | text) 252 | 253 | 254 | cdef inline bint _pop_until_found_calling(item): 255 | return item.calling == item 256 | 257 | 258 | cdef class NarrationFragmentContextManager(NarrationFragment): 259 | _free_instances = deque() 260 | 261 | def __init__(self, *args, **kwargs): 262 | super(NarrationFragmentContextManager, self).__init__(*args, **kwargs) 263 | if _thread_fragments[current_thread().name].verbose: 264 | calling_frame = inspect.stack()[3] 265 | self.func_name = calling_frame[3] 266 | self.source_file = calling_frame[1] 267 | 268 | cpdef str format(self, bint verbose=False, bint best_effort_return=False): 269 | cdef str tale = super(NarrationFragmentContextManager, 270 | self).format(verbose=verbose, 271 | best_effort_return=best_effort_return) 272 | cdef list parts = tale.split("\n") 273 | parts = [" " * (i + 2) + parts[i] for i in range(len(parts))] 274 | return "\n".join(parts) 275 | 276 | def __enter__(self): 277 | cdef str tname = current_thread().name 278 | _thread_fragments[tname].append(self) 279 | self.calling = self 280 | return self 281 | 282 | def __exit__(self, exc_type, exc_val, _): 283 | cdef str tname = current_thread().name 284 | 285 | d = _thread_fragments[tname] 286 | if exc_type is None: 287 | # then all went well; pop ourselves off the end 288 | self.status = self.COMPLETED 289 | if d.check: 290 | try: 291 | _ = self.format() 292 | except Exception as e: 293 | ctx_frame = inspect.getouterframes(inspect.currentframe())[1] 294 | frame, fname, lineno, function, _, _ = ctx_frame 295 | del frame, function, ctx_frame 296 | raise ErratorException("Failed formatting fragment in context; " 297 | "got exception {}, '{}'; {}:{} is the last " 298 | "line of the context".format(type(e), str(e), 299 | fname, lineno)) 300 | 301 | if d and d.auto_prune: 302 | d.pop_until_true(_pop_until_found_calling) 303 | self.calling = None # break ref cycle 304 | else: 305 | if d[-1] is self: 306 | # this is where the exception was raised 307 | self.fragment_exception_text(exc_type, str(exc_val)) 308 | self.status = self.RAISED_EXCEPTION 309 | # the following code annotates fragments with stack trace information 310 | # so if verbose output is requested it can be included 311 | if d.verbose: 312 | tb = inspect.trace() 313 | stack = inspect.stack() 314 | stack.reverse() 315 | # NOTE: slightly different than for func decorators! 316 | sc = deque(stack + tb) 317 | scpop = sc.pop 318 | deck = deque(d) 319 | deckpop = deck.pop 320 | while deck and sc: 321 | while sc and not deck[-1].frame_describes_func(sc[-1]): 322 | scpop() 323 | if sc: 324 | deck[-1].annotate_fragment(sc[-1]) 325 | deckpop() 326 | else: 327 | self.status = self.PASSEDTHRU_EXCEPTION 328 | try: 329 | _ = self.format() 330 | except Exception as e: 331 | ctx_frame = inspect.getouterframes(inspect.currentframe())[1] 332 | frame, fname, lineno, function, _, _ = ctx_frame 333 | del frame, function, ctx_frame 334 | raise ErratorException("Failed formatting fragment in context; got " 335 | "exception {}, '{}'; {}:{} is the last line of " 336 | "the context".format(type(e), str(e), 337 | fname, lineno)) 338 | 339 | 340 | def narrate(str_or_func, tags: Iterable[str] = None): 341 | """ 342 | Decorator for functions or methods that add narration that can be recovered if the 343 | method raises an exception 344 | 345 | :param str_or_func: either a string that will be captured and rendered if the function 346 | fails, or else a callable with the same signature as the function/method that is 347 | being decorated that will only be called if the function/method raises an 348 | exception; in this case, the callable will be invoked with the (possibly 349 | modified) arguments that were passed to the function. The callable must return 350 | a string, and that will be used for the string that describes the execution of 351 | the function/method 352 | :param tags: optional, iterable of strings. If supplied, then the fragment for 353 | this narration can be optionally retrieved using get_narration() by the caller 354 | of that function supplying one or more of the same string tags that appear in 355 | the 'tags' argument of this application of the decorator. If tags aren't supplied, 356 | then this narration fragment appears in any list of strings returned by 357 | get_narration(), regardless if tags are supplied in that call or not. 358 | 359 | NOTE: if a callable is passed in, it will only be called with the decorated 360 | function's arguments if the decorated function raises an exception during 361 | execution. This way no time is spent formatting a string that may not be needed. 362 | However, if the decorated function has changed the value of any of the arguments 363 | and these are in turn used in formatting the narration string, be aware that these 364 | may not be the values that were actually passed into the decorated function. 365 | """ 366 | def capture_stanza(m): 367 | cdef str func_name = m.__name__, source_file = inspect.getsourcefile(m) 368 | cdef frozenset the_tags = None 369 | 370 | if tags is not None: 371 | the_tags = frozenset(tags) 372 | 373 | def narrate_it(*args, **kwargs): 374 | global current_thread 375 | cdef NarrationFragment fragment = NarrationFragment.get_instance(str_or_func, 376 | m, *args, 377 | **kwargs) 378 | if the_tags is not None: 379 | fragment.set_tags(the_tags) 380 | fragment.func_name = func_name 381 | fragment.source_file = source_file 382 | fragment.calling = m 383 | frag_deque = _thread_fragments[current_thread().name] 384 | frag_deque.append(fragment) 385 | try: 386 | _v = m(*args, **kwargs) 387 | fragment.status = fragment.COMPLETED 388 | if frag_deque.check: 389 | try: 390 | _ = fragment.format() 391 | except Exception as e: 392 | raise ErratorException("Failed formatting the fragment for " 393 | "function {}; received exception " 394 | "{}, '{}'".format(m, type(e), str(e))) 395 | if frag_deque and frag_deque.auto_prune: 396 | frag_deque.pop_until_true(lambda item: item.calling == m) 397 | fragment = None 398 | return _v 399 | except Exception as e: 400 | if fragment is frag_deque[-1]: 401 | # only grab the exception text if this is the last fragment 402 | # on the call chain 403 | fragment.fragment_exception_text(e.__class__, str(e)) 404 | fragment.status = fragment.RAISED_EXCEPTION 405 | # the following code annotates fragments with stack trace information 406 | # so if verbose output is requested it can be included 407 | if frag_deque.verbose: 408 | tb = inspect.trace() 409 | stack = inspect.stack() 410 | stack.reverse() 411 | sc = deque(stack + tb[1:]) 412 | scpop = sc.pop 413 | deck = deque(frag_deque) 414 | deckpop = deck.pop 415 | while deck and sc: 416 | while sc and not deck[-1].frame_describes_func(sc[-1]): 417 | scpop() 418 | if sc: 419 | deck[-1].annotate_fragment(sc[-1]) 420 | deckpop() 421 | else: 422 | fragment.status = fragment.PASSEDTHRU_EXCEPTION 423 | try: 424 | _ = fragment.format() # get the formatted fragment right now! 425 | except Exception as e: 426 | raise ErratorException("Failed formatting the fragment for " 427 | "function {}; received exception {}, '{}'". 428 | format(m, type(e), str(e))) 429 | raise 430 | 431 | narrate_it.__name__ = m.__name__ 432 | narrate_it.__doc__ = m.__doc__ 433 | narrate_it.__dict__.update(m.__dict__) 434 | return narrate_it 435 | return capture_stanza 436 | 437 | 438 | cpdef list get_narration(thread: Thread=None, bint from_here=False, 439 | with_tags: Iterable=None): 440 | """ 441 | Return a list of strings, each one a narration fragment in the function call path. 442 | 443 | This method tells the tale of an exception; it returns a list of strings that are the 444 | narration fragments from each function/method call or context where narration has 445 | been captured. It starts at the most global level and goes to the level where the 446 | exception was raised. 447 | 448 | :param thread: instance of Thread. If not supplied, the current thread is used. 449 | :param from_here: boolean, optional, default False. If True, then the list of strings 450 | returned is from the narration fragment nearest the active stack frame and down to 451 | the exception origin, not from the most global level to the exception. This is 452 | useful from where the exception is actually caught, as it provides a way to 453 | only show the narration from this point forward. However, not showing all the 454 | fragments may actually hide important aspects of the narration, so bear this in 455 | mind when using this to prune the narration. Use in conjuction with the 456 | auto_prune option set to False to allow several stack frames to return before 457 | collecting the narration (be sure to manually clean up the narration when 458 | auto_prune is False). 459 | :param with_tags: iterable, optional, default None. If supplied, will only return 460 | narration fragments where the fragment was given one or more of the supplied 461 | tags from with_tags. Narrations with no tags at all will always be included 462 | regardless of the tags supplied. Likewise, if no tags are supplied, then all 463 | narration fragments are returned. However, if an empty tag list is supplied, 464 | then no fragments will be returned. 465 | :return: list of formatted strings. 466 | """ 467 | cdef list l 468 | cdef bint verbose 469 | cdef frozenset tags = None 470 | cdef NarrationFragment nf 471 | 472 | if with_tags is not None: 473 | tags = frozenset(with_tags) 474 | if thread is None: 475 | thread = current_thread() 476 | elif not isinstance(thread, Thread): 477 | raise ErratorException("the 'thread' argument isn't an instance " 478 | "of Thread: {}".format(thread)) 479 | d = _thread_fragments.get(thread.name) 480 | if not d: 481 | l = list() 482 | else: 483 | verbose = d.verbose 484 | if not from_here: 485 | l = [nf.tell(verbose=verbose) for nf in d 486 | if tags is None or not nf.any_tags() or not nf.are_tags_disjoint(tags)] 487 | else: 488 | # collect from the last IN_PROCESS fragment to the exception 489 | l = list() 490 | lappend = l.append 491 | for i in range(-1, -1 * len(d) - 1, -1): 492 | if d[i].status == NarrationFragment.IN_PROCESS: 493 | for j in range(i, 0, 1): 494 | nf = d[j] 495 | if (tags is None or not nf.any_tags() or 496 | not nf.are_tags_disjoint(tags)): 497 | lappend(nf.tell(verbose=verbose)) 498 | break 499 | return l 500 | -------------------------------------------------------------------------------- /devbuild.bat: -------------------------------------------------------------------------------- 1 | python setup.py build_ext --inplace 2 | -------------------------------------------------------------------------------- /devbuild.bsh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | python setup.py build_ext --inplace 4 | -------------------------------------------------------------------------------- /docker_build.bsh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | docker pull quay.io/pypa/manylinux2010_x86_64:latest 4 | docker run -v `pwd`:/io quay.io/pypa/manylinux2010_x86_64:latest /io/manylinux_build.bsh 5 | docker pull quay.io/pypa/manylinux1_x86_64:latest 6 | docker run -v `pwd`:/io quay.io/pypa/manylinux2014_x86_64:latest /io/manylinux_build_py38.bsh 7 | -------------------------------------------------------------------------------- /docs/building-howto.txt: -------------------------------------------------------------------------------- 1 | Various build tools for errator: 2 | ================================ 3 | 4 | General 5 | ------- 6 | All builds are now mediated through 'pip wheel' and 'setup.py', the former relying on the 7 | latter. There are different build requirements for development and final builds for 8 | publishing on PyPI; these are covered below. 9 | 10 | 11 | Linux Builds 12 | ------------ 13 | 14 | --General-- 15 | 16 | Relevant files: setup.py, setup.cfg, MANIFEST.in, requirements.txt 17 | 18 | Regardless of whether the build is for dev or publishing on PyPI, you require the 19 | following resources on your base Linux platform: 20 | 21 | - gcc 22 | - Python development package (header files) 23 | 24 | Install these via your platform's package manager. 25 | 26 | Additionally, for whatever Python environment you intend to develop in, whether the 27 | platform's native Python install or a virtualenv, you'll need to install the additional 28 | requirements for development and testing. These are in the requirements.txt file in the 29 | root of the project, and can be installed as follows: 30 | 31 | pip install -r requirements.txt 32 | 33 | This will add Cython and pytest to your environment, which are only used to build and test 34 | new versions. 35 | 36 | --Dev builds-- 37 | 38 | Relevant files: devbuild.bsh 39 | 40 | Development builds are created using the 'devbuild.bsh' script file found in the root 41 | of the project. This will create a shared library that you can load from within Python 42 | with a line that reads: 43 | 44 | import errator 45 | 46 | You may have to set execute permissions on the script file. The shared library is built 47 | right in the project root, but some additional files are created in a 'build' directory 48 | in the root, which is created if it doesn't exist. 49 | 50 | --PyPI (manylinux) builds-- 51 | 52 | Relevant files: docker_build.bsh, manylinux_build.bsh, manylinux_build_py38.bsh 53 | 54 | To build a set of releases for publishing on PyPI, some additional tooling is required. 55 | You must create a so-called 'manylinux' build to be able to build to publish to PyPI, 56 | and the simplest way to do that is to utilize the manylinux docker containers to perform 57 | the build. 58 | 59 | You will need to install 'docker' on your build machine using your package manager, and 60 | then when connected to the internet, you run: 61 | 62 | docker_build.bsh 63 | 64 | Which can be found in the root of the project (you may need to set execute permissions). 65 | This script tells docker to pull down one or more manylinux images (these are the latest 66 | images, but are cached by docker so they are only pulled down once when new), and then 67 | a container is started using each image and running one of the manylinux_build*.bsh 68 | scripts from withing the container. The script creates a 'wheels' directory in the project 69 | root, and as the stages of the build are performed, the shared libraries and then 70 | the wheels are written there. Finally, the built wheels are installed into the container's 71 | Python system and the test program is run to ensure that everything works properly. 72 | 73 | NOTE: Currently, there are two different containers run to build for all versions of 74 | Python supported. This is covered below. 75 | 76 | manylinux_build.bsh: 77 | This builds wheels for Python 3.6, 3.7, and 3.9, using the manylinux2010 image. This 78 | yields wheels for manylinux1 and manylinux2010. 79 | 80 | manylinux_build_py38.bsh: 81 | Using the manylinux2010 image to build a wheel for Python 3.8 yields errors involving the 82 | version of glibc, and so to build a wheel for Python 3.8 we use the manylinux1 image. 83 | This is run after the general manylinux build and so kind of expects any initial cleanup 84 | to have been done by that script. 85 | 86 | When docker_build.sh is done, it will leave working wheels in the 'wheels' directory in 87 | the project root. Unfortunately these will be owned by root, so to remove them you'll 88 | need root permissions, either on the development machine or else from mounting the 89 | directory into a container running a bash session where you can delete them from there. 90 | 91 | 92 | Windows builds 93 | -------------- 94 | 95 | --General-- 96 | 97 | Pip has materially simplified building on Windows, both for dev and for PyPI, but there 98 | are still a couple of pre-requisites that must be addressed. 99 | 100 | Foremost is that the free Microsoft community edition C++ compiler must be downloaded 101 | from Microsoft and installed. A quick Google search usually takes you right to it. You 102 | can alternatively install the free Visual Studio package which includes the compiler. 103 | 104 | Additionally, for whatever Python environment you intend to develop in, whether the 105 | platform's native Python install or a virtualenv, you'll need to install the additional 106 | requirements for development and testing. These are in the requirements.txt file in the 107 | root of the project, and can be installed as follows: 108 | 109 | pip install -r requirements.txt 110 | 111 | This will add Cython and pytest to your environment, which are only used to build and test 112 | new versions. 113 | 114 | --Dev builds-- 115 | 116 | Relevant files: devbuild.bat 117 | 118 | You can build and in-place shared library using the 'devbuild.bat' file from the Windows 119 | command line. This will properly invoke the MS C++ compiler and yield a shared library 120 | that you can load directly into Python with: 121 | 122 | import errator 123 | 124 | --PyPI builds-- 125 | 126 | Relevant files: win_build_all.bat 127 | 128 | When you're ready to create wheels for pushing to PyPI, run 'win_build_all.bat'. This 129 | will create a wheels directory into which the wheels will be written. 130 | 131 | I haven't cracked a containerized Windows build yet, so instead the windows build is based 132 | on having multiple versions of Python installed on the build machine. The assumptions here 133 | are: 134 | 135 | - The available Python verions are in c:\py with names like py36, py37, etc (although 136 | this is easily tweaked). 137 | - There is a c:\vpy for the creation of virtual build and test environments (this can 138 | also be tweaked). 139 | - The build machine can reach the internet. 140 | 141 | The main file performs a loop in which, for each release of Python: 142 | 143 | - Previous build/test virtualenvs are removed. 144 | - A new build virtualenv is created and conditioned. 145 | - The wheel for this Python version is built. 146 | - A new test virtualenv is created. 147 | - The new wheel is installed into the test virtualenv 148 | - pytest is run using the test virtualenv and the installed wheel 149 | 150 | If all tests pass, then we call this build good. 151 | 152 | 153 | Uploading to PyPI using twine 154 | ----------------------------- 155 | 156 | To upload to test PyPI using twine: 157 | 158 | twine upload --repository-url https://test.pypi.org/legacy/ wheels/* 159 | 160 | Run this from the project root directory. This will upload all the created distributions. 161 | This should include any distros from windows 162 | 163 | To upload to the real PyPI using twine: 164 | 165 | twine upload wheels/* 166 | 167 | Run this from the project root directory. This will upload all the created distributions. 168 | This should include any distros from windows/linux. 169 | 170 | 171 | 172 | -------------------------------------------------------------------------------- /docs/demo.py: -------------------------------------------------------------------------------- 1 | """ 2 | Demonstrate errator in multi-threaded apps 3 | 4 | This example just shows how to acquire an error's narration and display it. It also shows how 5 | a separate narration is kept for each thread, and how to display data passed to a function 6 | or method in the case of an exception. 7 | """ 8 | import threading 9 | from errator import narrate_cm, narrate, get_narration 10 | 11 | 12 | # narrate the call to nf2, using a lambda to show the arguments passed when an exception occurs 13 | @narrate(lambda a3, a4: 14 | "...I was subsequently asked to nf2 with {} and {}".format(a3, a4)) 15 | def f2(arg3, arg4): 16 | # use a narration context manager to wrap a block of code with narration 17 | with narrate_cm("...so I first started to do 'this'"): 18 | # all of my 'this' activities 19 | # which create variables x and y 20 | x = arg3 * 2 21 | y = arg4 * 3 22 | 23 | with narrate_cm(lambda: "...and then went on to do 'that' with x={} and y={}".format(x, y)): 24 | # all of my 'that' activities 25 | raise Exception("ruh-roh") 26 | 27 | 28 | # narrate the call to nf1, using a lambda to show the arguments passed when an exception occurs 29 | @narrate(lambda a1, a2: "I was asked to nf1 with {} and {}".format(a1, a2)) 30 | def f1(arg1, arg2): 31 | # do some things, then 32 | try: 33 | f2(arg1+1, arg2+1) 34 | except Exception as e: 35 | lines = ["My thread {}'s story:".format( 36 | threading.current_thread().name)] 37 | lines.extend([s for s in get_narration()]) 38 | print("\n".join(lines)) 39 | print("") 40 | 41 | t1 = threading.Thread(target=f1, args=(1, 2), name="t1") 42 | t2 = threading.Thread(target=f1, args=(10, 20), name="t2") 43 | t1.start() 44 | t2.start() 45 | -------------------------------------------------------------------------------- /docs/index.html: -------------------------------------------------------------------------------- 1 |

Errator

2 |

Provide human-readable error narration of exception tracebacks with Errator.

3 |
    4 |
  1. What's new in 0.2
    2. 5 |
  2. Intro
    3. 6 |
  3. How it works
    4. 7 |
  4. Requirements
    5. 8 |
  5. Installing
    6. 9 |
  6. Quick Tutorial
    7. 10 |
11 |

What's new in 0.2

12 |
    13 |
  • The get_narration() now has a new keyword argument, "verbose" (default False), that when True returns expanded narration information that includes the line number, function, and file name of the point in the stack trace that the narration applies to.
    • 14 |
  • To provide a more tidy display of stack information, errator now has analogs of the functions from the standard traceback module that filter out errator-based calls from the call stack, leaving only application calls in the display of stack traces.
    • 15 |
  • Narration output formatting has been modified slightly.
    • 16 |
17 |

Intro

18 |

Errator came as an idea on the back of trying to figure out what the semantics of an exception traceback are in non-trivial pieces of code.

19 |

When an exception occurs deep inside a call stack within some generic utility function that is used in numerous contexts, reading the traceback is often not helpful in determine the source of the problem. Data values aren't obvious, and the initial starting conditions of the error can't easily been seen.

20 |

Logging is a step in the right direction, but in general outputs too much information; often, there is lots of info regarding error-free processing in the log, making it hard to find the right log output that is associated with the error.

21 |

Errator is something of a marriage between logging and tracebacks: plain text messages that are associated directly with the call trail that led to an exception. Errator works by providing tools that let you state intent of code in text, but only captures that text when an exception bubbles up the stack. You can then acquire this "error narration" and display it to your user in the most appropriate fashion.

22 |

How it works

23 |

Errator uses decorators and context managers to maintain a stack of "narration fragments" behind the scenes. When code executes without exceptions, these fragments are created and thrown away as narrated code executes and returns. However, when an exception is raised, the narration fragments are retained and their content can be retrieved. Fragments can be automatically discarded (pruned) when an exception doesn't propagate any further up the stack, or can be discarded under user control, allowing more control over the content of "narration" provided for the exception.

24 |

Errator is thread-safe, allowing you to capture separate error narrations for independent threads of control through the same code.

25 |

Requirements

26 |

Errator doesn't have any external dependencies. It is compatible with Python 2.7 and 3.x.

27 |

Installing

28 |

Errator is a single file, and can be installed either with pip or running 'python setup.py install' after pulling the Git project.

29 |

Quick Tutorial

30 |

The next section discusses Errator with functions, but you can also use the decorators described with methods too.

31 |

Start with pulling errator into your module that you want to narrate:

32 | 
from errator import *
33 |

Now, suppose you have a utility function that performs some specialized string formatting, but it is possible to pass in arguments that cause a exception to be raised. Your function is called all over the place for a variety of different reasons, often very deep down the call stack where it isn't obvious what the original functional intent was, or where the source of bad arguments may have been.

34 |

To start building the narration to your function's execution, you can use the narrate() decorator to associate a bit of text with your utility function to provide easily understandable explanations about what's going on:

35 | 
@narrate("I'm trying to format a string")
 36 | def special_formatter(fmt_string, **kwargs):
 37 |     # magic format code that sometimes raises an exception
38 |

The narrate() decorator knows to look for exceptions and doesn't impede their propagation, but captures that bit of text in an internal stack when an exception occurs. So if you write:

39 | 
try:
 40 |     s = special_formatter(fmt, **args)
 41 | exception Exception:
 42 |     the_tale = get_narration()
43 |

...and special_formatter() raises an exception, the exception will still bubble up the stack, but get_narration() will return a list of strings for all the narrate()-decorated functions down to the exception. If no exception is raised, there are no strings (well, it's a little more complicated than that, but we'll get to that).

44 |

Maybe you'd like some insight as to the arguments present when an exception is raised so you can better tell what's causing it. Instead of a string, you can supply the narrate() decorator with a callable that has the same signature as the function being decorated. This callable will be invoked only if the decorated function raises an exception, and gets invoked with the same arguments that the function was:

45 | 
@narrate(lambda fs, **kw: "I'm trying to format a string with '%s' and args '%s'" % (fs, str(kw)))
 46 | def special_formatter(fmt_string, **kwargs):
 47 |     # magic format code that sometimes raises an exception
48 |

The lambda passed to narrate() will only be called when special_formatter() raises an exception, otherwise it will go un-executed.

49 |

Now, perhaps special_formatter() is a rather long function, and you'd like to be able to narrate it's operation in more detail to get better narrations when things go wrong. You can use the narrate_cm() context manager to create a narration fragment for a block of code. If everything goes well in the block, then the fragment is discarded, but the fragment will be retained if an exception occurs:

50 | 
def special_formatter(fmt_string, **kwargs):
 51 |     for format_token in parse_format(fmt_string):
 52 |         if format_token.type == float:
 53 |             with narrate_cm("I started processing a float format"):
 54 |                 # do magic stuff for floats...
 55 |         elif format_token.type == int:
 56 |             with narrate_cm("I started processing an int format"):
 57 |                 # do magic stuff for ints...
58 |

Narration fragments added with narrate_cm() are treated just like those created by the function decorator-- they are added to the stack, and silently removed if the context manager's code block exits normally. But exceptions raised in the context block are retained as the exception propagates back through the stack.

59 |

Like narrate(), narrate\_cm() allows you to supply a callable instead of a string:

60 | 
with narrate_cm(lambda x: "I started processing an int with format %s" % x, format_token.format):
 61 |     # format code
62 |

...and again, this callable will only be invoked if an exception is raised in the context. Unlike narrate(), however, you are free to define a callable with any signature, as long as you supply the arguments needed as well to invoke the callable if need be.

63 |

Context managers may nest, and in fact any combination of function decorator and context manager will work as expected.

64 |

Let's look at an example with more complex calling relationships. Suppose we have functions A, B, C, D, E, and F. They have the following calling relationships:

65 |
    66 |
  • A calls B then C
    • 67 |
  • B calls D
    • 68 |
  • C calls E or F
    • 69 |
  • D calls F
    • 70 |
71 |

We'll make it so that if we're unlucky enough to call E, we'll get an exception raised. This will happen only for input values of A greater than 10.

72 |

So let's define these functions and narrate them-- paste these into an interactive Python session after you've imported errator:

73 | 
@narrate(lambda v: "I'm trying to A with %s as input" % v)
 74 | def A(val):
 75 |     B(val / 2)
 76 |     C(val * 2)
 77 | 
 78 | @narrate(lambda v: "I'm trying to B with %s as input" % v)
 79 | def B(val):
 80 |     D(val * 10)
 81 | 
 82 | @narrate(lambda v: "I'm trying to C with %s as input" % v)
 83 | def C(val):
 84 |     if val > 20:
 85 |         E(val)
 86 |     else:
 87 |         F(val)
 88 | 
 89 | @narrate(lambda v: "I'm trying to D with %s as input" % v)
 90 | def D(val):
 91 |     F(val * 3)
 92 | 
 93 | @narrate(lambda v: "I'm trying to E with %s as input" % v)
 94 | def E(val):
 95 |     raise ValueError("how dare you call me with such a value?")
 96 | 
 97 | @narrate(lambda v: "I'm trying to F with %s as input" % v)
 98 | def F(val):
 99 |     print("very well")
100 |

Now run A with a value less than 11, and look for narration text:

101 | 
>>> A(3)
102 | very well
103 | very well
104 | >>> get_narration()
105 | []
106 | >>>
107 |

Now run A with a value greater than 10:

108 | 
>>> A(11)
109 | very well
110 | Traceback (most recent call last):
111 |   File "<stdin>", line 1, in <module>
112 |   File "errator.py", line 322, in callit
113 |     _v = m(*args, **kwargs)
114 |   File "<stdin>", line 4, in A
115 |   File "errator.py", line 322, in callit
116 |     _v = m(*args, **kwargs)
117 |   File "<stdin>", line 4, in C
118 |   File "errator.py", line 322, in callit
119 |     _v = m(*args, **kwargs)
120 |   File "<stdin>", line 3, in E
121 | ValueError: how dare you call me with such a value?
122 | >>>
123 |

So far, it's as we'd expect, except perhaps for the inclusion of errator calls in the stack. But now let's look at the narration:

124 | 
>>> for l in get_narration():
125 | ...     print(l)
126 | ... 
127 | I'm trying to A with 11 as input
128 | I'm trying to C with 22 as input
129 | I'm trying to E with 22 as input, but exception type: ValueError, value: how dare you call me with such a value? was raised
130 | >>>
131 |

We have a narration for our recent exception. Now try the following:

132 | 
>>> A(8)
133 | very well
134 | very well
135 | >>> get_narration()
136 | ["I'm trying to A with 11 as input", "I'm trying to C with 22 as input", # etc...
137 |

Wait, this didn't have an exception; why is there still narration? This is because an error narration only gets cleared out if a decorated function does NOT have an exception bubble up; the assumption is that the exception was caught and the narration was retrieved, so a decorated function that returns normally would remove the previous narration fragments. In our example, there is no function that is decorated with narrate() that catches the exception and returns normally, so the narration never clears out.

138 |

There are a few ways to clear unwanted narrations: first is to manually clear the narration, and the other is to make sure you have a decorated function that catches the exception and returns normally, which will clear the narration automatically

139 |

To manually clear narrations we call reset_narration():

140 | 
>>> reset_narration()
141 | >>> get_narration()
142 | >>> []
143 |

For the second, if we define a decorated function that calls A but which handles the exception and returns normally, the narration fragments will be cleaned up automatically:

144 | 
@narrate("Handler for A")
145 | def first(val):
146 |     try:
147 |         A(val)
148 |     except:
149 |         print("Got %d narration lines" % len(get_narration()))
150 |

This outermost function still can retrieve the narration, but as it returns normally, the narration is cleared out when it returns:

151 | 
>>> first(11)
152 | very well
153 | Got 4 narration lines
154 | >>> get_narration()
155 | []
156 | >>>
157 |

Errator provides finer degrees of control for getting the narration; these are covered in the detailed docs.

158 | -------------------------------------------------------------------------------- /docs/using_errator.rst: -------------------------------------------------------------------------------- 1 | ############# 2 | Using errator 3 | ############# 4 | 5 | #. `If you don't read anything else, READ THIS <#if-you-don-t-read-anything-else-read-this>`__ 6 | #. `errator's operation <#errator-s-operation>`__ 7 | #. `Capturing the narration <#capturing-the-narration>`__ 8 | #. `Skipping decorating functions <#skipping-decorating-functions>`__ 9 | #. `Customizing the narration <#customizing-the-narration>`__ 10 | #. `Getting more details with contexts <#getting-more-details-with-contexts>`__ 11 | #. `Advanced fragment access <#advanced-fragment-access>`__ 12 | #. `Verbose narrations <#verbose-narrations>`__ 13 | #. `Testing and debugging <#testing-and-debugging>`__ 14 | #. `Tidying up stack traces <#tidying-up-stack-traces>`__ 15 | #. `Overhead <#overhead>`__ 16 | #. `Usage tips <#usage-tips>`__ 17 | 18 | ``errator`` is a fairly small library that's easy to wrap your head around. While basic 19 | usage is fairly simple, ``errator`` also allows you more sophisticated uses in multi-threaded 20 | programs where each thread can have its own exception narration, as well as being able to 21 | manage partial narrations. 22 | 23 | There are a couple of anti-patterns in ``errator's`` use which are important to understand, so 24 | we'll lead off with addressing those before launching into a more general discussion of using 25 | ``errator``. 26 | 27 | .. note:: 28 | 29 | The documentation generally discusses decorating functions with ``errator``, but ``errator's`` decorators can also be used to decorate methods. For brevity, when 'function' is used it should be assumed to mean 'function or method'. 30 | 31 | If you don't read anything else, READ THIS 32 | ------------------------------------------ 33 | 34 | ``errator`` decorators, context managers, and narration management functions work together to 35 | manage a set of per-thread stacks of "narration fragments". In "normally" operating code (that is, with no exceptions), these fragments are created at the start of a function or context, and discarded 36 | when the function or context completes without an exception (push on call, pop on return). 37 | 38 | But when an exception occurs, the fragment is retained, and as the exception passes un-caught up the stack through other ``errator`` managed functions or contexts additional fragments may also be retained, until the exception is caught and ``errator`` is told that it may finally discard the fragments. This discarding may be done automatically or under programmatic control, depending on how ``errator`` is to be used, but the key is that unless ``errator`` discards the fragments, they will simply keep growing in number and may cause memory issues if code experiences numerous errors without disposing of the fragments, not to mention yielding confusing narrations of exceptions. 39 | 40 | There are two anti-patterns that can lead to this situation to be aware of. 41 | 42 | -------------------------------------------------------------------------------------------- 43 | Anti-pattern #1-- catching the exception outside of ``errator's`` view 44 | -------------------------------------------------------------------------------------------- 45 | 46 | If you catch an exception in a function that hasn't been decorated with ``errator`` decorators (and there are no more ``errator``-decorated functions or contexts at a more global level in the call stack), it will leak narration fragments and the narration will grow, making it useless: 47 | 48 | .. code-block:: python 49 | 50 | def nf1(): 51 | "NOTE: not decorated with 'narrate()'" 52 | try: 53 | nf2() 54 | except Exception as e: 55 | story = get_narration() 56 | # handle the exception 57 | 58 | @narrate("I starting to 'nf2'") 59 | def nf2(): 60 | nf3() 61 | 62 | @narrate("I've been asked to 'nf3'") 63 | def nf3(): 64 | raise Exception("catch me!") 65 | 66 | # some time later... 67 | nf1() 68 | 69 | The problem is that nf1() isn't decorated with ``narrate()``, and hence ``errator`` doesn't know that 70 | the exception was handled. Try it-- enter the above code and call nf1() twice, and then look at the 71 | returned narration from ``get_narration()``. **Remember**: this isn't a problem if there is an ``errator`` decorated function or context at a more global level in the call stack. 72 | 73 | You can fix this a couple of ways: 74 | 75 | **Approach #1:** 76 | 77 | .. code-block:: python 78 | 79 | # this approach will cause ``errator`` to automatically clean fragments: 80 | 81 | @narrate("I'm starting nf1") # we added decoration to the ``nf1()`` function 82 | def nf1(): 83 | "NOTE: NOW decorated with 'narrate()'" 84 | try: 85 | nf2() 86 | except Exception as e: 87 | story = get_narration() 88 | # handle the exception 89 | 90 | @narrate("I starting to 'nf2'") 91 | def nf2(): 92 | nf3() 93 | 94 | @narrate("I've been asked to 'nf3'") 95 | def nf3(): 96 | raise Exception("catch me!") 97 | 98 | # some time later... 99 | nf1() 100 | 101 | **Approach #2** 102 | 103 | .. code-block:: python 104 | 105 | # in this approach, you manually clear out the narration fragments 106 | 107 | def nf1(): 108 | "NOTE: no decoration, but we clean up in the exception clause" 109 | try: 110 | nf2() 111 | except Exception as e: 112 | story = get_narration() 113 | reset_narration() # CLEANS UP FRAGMENTS 114 | # handle the exception 115 | 116 | @narrate("I starting to 'nf2'") 117 | def nf2(): 118 | nf3() 119 | 120 | @narrate("I've been asked to 'nf3'") 121 | def nf3(): 122 | raise Exception("catch me!") 123 | 124 | # some time later... 125 | nf1() 126 | 127 | ----------------------------------------------------------------------------- 128 | Anti-pattern #2: Shutting off automatic cleanup but not clearing up fragments 129 | ----------------------------------------------------------------------------- 130 | 131 | For more complex uses of ``errator``, you can turn off automatic fragment cleanup, but if 132 | you do so then you **must** handle cleanup yourself. The following will suffer from the same 133 | leakage/growing narration as the first anti-pattern: 134 | 135 | .. code-block:: python 136 | 137 | @narrate("Look out-- I'm about to nf1()!") 138 | def nf1(): 139 | "we've got nf1 decorated" 140 | try: 141 | nf2() 142 | except Exception as e: 143 | story = get_narration() 144 | # handle the exception 145 | 146 | @narrate("I starting to 'nf2'") 147 | def nf2(): 148 | nf3() 149 | 150 | @narrate("I've been asked to 'nf3'") 151 | def nf3(): 152 | raise Exception("catch me!") 153 | 154 | set_narration_options(auto_prune=False) 155 | 156 | # later, in the same thread: 157 | nf1() 158 | 159 | In this example, even though all functions in the call chain are decorated with ``narrate()``, 160 | we'll still leak fragements and allow the narration to grow. This is because 161 | ``set_narration_options()`` was used to turn off "auto_prune", which makes ``errator`` not discard 162 | fragments when exceptions have been handled. Note that this has to happen in the same thread; 163 | each thread can have different narration options. 164 | 165 | If you want to have auto_prune off (and there are cases where you might want to do this), fixing 166 | this is like the second solution to the first anti-pattern: 167 | 168 | .. code-block:: python 169 | 170 | @narrate("Look out-- I'm about to nf1()!") 171 | def nf1(): 172 | "we've got nf1 decorated" 173 | try: 174 | nf2() 175 | except Exception as e: 176 | story = get_narration() 177 | reset_narration() #CLEANS UP THE FRAGMENTS 178 | # handle the exception 179 | 180 | @narrate("I starting to 'nf2'") 181 | def nf2(): 182 | nf3() 183 | 184 | @narrate("I've been asked to 'nf3'") 185 | def nf3(): 186 | raise Exception("catch me!") 187 | 188 | set_narration_options(auto_prune=False) 189 | 190 | # later, in the same thread: 191 | nf1() 192 | 193 | Here, we've simply called ``reset_narration()`` after the narration text has been acquired, and 194 | this gets rid of all fragments for the thread. 195 | 196 | ``errator's`` Operation 197 | ----------------------- 198 | 199 | Let's look at an example of a set of functions that can be decorated with ``errator's`` ``narrate()`` decorator. Let's suppose we have a set of functions ``nf1`` through ``nf6``, where ``nf1`` calls ``nf2``, ``nf2`` calls ``nf3``, and so forth. If we stopped in the debugger in ``nf6``, Python would report the stack like so: 200 | 201 | +-------+------------------+ 202 | | func | execution point | 203 | +=======+==================+ 204 | | nf1 | | 205 | +-------+------------------+ 206 | | nf2 | | 207 | +-------+------------------+ 208 | | nf3 | | 209 | +-------+------------------+ 210 | | nf4 | | 211 | +-------+------------------+ 212 | | nf5 | | 213 | +-------+------------------+ 214 | | nf6 | <-- current frame| 215 | +-------+------------------+ 216 | 217 | When we decorate functions with ``narrate()``, additional stack frames are added to the trace; we won't show those here, but instead will show what fragments are managed as the execution progresses. Here's the retained narration fragments if ``nf1..nf6`` are all decorated with ``narrate()`` and the current function is ``nf4``: 218 | 219 | +-------+------------------+---------------------+ 220 | | func | execution point | fragments for funcs | 221 | +=======+==================+=====================+ 222 | | nf1 | | | 223 | +-------+------------------+---------------------+ 224 | | nf2 | | | 225 | +-------+------------------+---------------------+ 226 | | nf3 | | | 227 | +-------+------------------+---------------------+ 228 | | nf4 | <-- current frame| nf1, nf2, nf3, nf4 | 229 | +-------+------------------+---------------------+ 230 | | nf5 | | | 231 | +-------+------------------+---------------------+ 232 | | nf6 | | | 233 | +-------+------------------+---------------------+ 234 | 235 | When ``nf4`` returns, the fragments are: 236 | 237 | +-------+------------------+---------------------+ 238 | | func | execution point | fragments for funcs | 239 | +=======+==================+=====================+ 240 | | nf1 | | | 241 | +-------+------------------+---------------------+ 242 | | nf2 | | | 243 | +-------+------------------+---------------------+ 244 | | nf3 | <-- current frame| nf1, nf2, nf3 | 245 | +-------+------------------+---------------------+ 246 | | nf4 | | | 247 | +-------+------------------+---------------------+ 248 | | nf5 | | | 249 | +-------+------------------+---------------------+ 250 | | nf6 | | | 251 | +-------+------------------+---------------------+ 252 | 253 | Note that the fragment for ``nf4`` is removed. 254 | 255 | Now suppose that we have an exception in ``nf6``, but the exception isn't captured until ``nf3``, at which point the exception is caught and doesn't propagate up the stack any further. This next table shows the fragments present as the functions either return and the exception propagates upward: 256 | 257 | +-------+------------------+-------------------------+ 258 | | func | execution point | fragments for funcs | 259 | +=======+==================+=========================+ 260 | | nf1 | normal return | nf1 | 261 | +-------+------------------+-------------------------+ 262 | | nf2 | normal return | nf1,nf2 | 263 | +-------+------------------+-------------------------+ 264 | | nf3 | exc handled | nf1,nf2,nf3,nf4,nf5,nf6 | 265 | +-------+------------------+-------------------------+ 266 | | nf4 | exc passes thru | nf1,nf2,nf3,nf4,nf5,nf6 | 267 | +-------+------------------+-------------------------+ 268 | | nf5 | exc passes thru | nf1,nf2,nf3,nf4,nf5,nf6 | 269 | +-------+------------------+-------------------------+ 270 | | nf6 | exception raised | nf1,nf2,nf3,nf4,nf5,nf6 | 271 | +-------+------------------+-------------------------+ 272 | 273 | Notice that in ``nf3`` where the exception is handled we still have all the fragments for all stack frames between the exception origin and the handler, but once the handler returns and ``errator`` sees that the exception isn't propagating further it removes the fragments that are no longer useful in narrating an exception (this makes ``nf3`` a good place to acquire the narration for the exception; more on that later). 274 | 275 | Capturing the narration 276 | ----------------------- 277 | 278 | Let's repeat the example from earlier, where we said that an exception was caught and processed in ``nf3``: 279 | 280 | +-------+------------------+-------------------------+ 281 | | func | execution point | fragments for funcs | 282 | +=======+==================+=========================+ 283 | | nf1 | normal return | nf1 | 284 | +-------+------------------+-------------------------+ 285 | | nf2 | normal return | nf1,nf2 | 286 | +-------+------------------+-------------------------+ 287 | | nf3 | exc handled | nf1,nf2,nf3,nf4,nf5,nf6 | 288 | +-------+------------------+-------------------------+ 289 | | nf4 | exc passes thru | nf1,nf2,nf3,nf4,nf5,nf6 | 290 | +-------+------------------+-------------------------+ 291 | | nf5 | exc passes thru | nf1,nf2,nf3,nf4,nf5,nf6 | 292 | +-------+------------------+-------------------------+ 293 | | nf6 | exception raised | nf1,nf2,nf3,nf4,nf5,nf6 | 294 | +-------+------------------+-------------------------+ 295 | 296 | If ``nf3`` catches the exception, it's probably a good place to grab the exception narration 297 | (this isn't required, but it may be a natural place). Suppose ``nf3()`` looks like the following: 298 | 299 | .. code-block:: python 300 | 301 | @narrate("While I was running nf3") 302 | def nf3(): 303 | try: 304 | nf4() 305 | except MyException: 306 | story = get_narration() 307 | 308 | In the ``except`` clause, we call ``get_narration()`` to acquire a list of strings that are the narration for the exception. This will return the entire narration that exists for this call stack; that is, it will give a list of narration fragment strings for ``nf1()`` through ``nf6()``. 309 | 310 | But perhaps the whole narration isn't wanted; perhaps all that's desired is the narration for 311 | ``nf3()`` through ``nf6()``, as the the narrations before this point actually make the exception narration less clear. You can trim your narration down with by calling ``get_narration()`` with the keyword argument ``from_here`` set to True: 312 | 313 | .. code-block:: python 314 | 315 | @narrate("While I was running nf3...") 316 | def nf3(): 317 | try: 318 | nf4() 319 | except MyException: 320 | story = get_narration(from_here=True) 321 | 322 | This will only return the narration strings from the current function to the function that's the source of the exception, in this case ``nf3()`` through ``nf6()``. The ``from_here`` argument allows you to control how much narration is returned from ``get_narration()``. It defaults to False, meaning to return the entire narration. 323 | 324 | Skipping decorating functions 325 | ----------------------------- 326 | 327 | What happens if you skip decorating some functions in a calling sequence? Nothing much; ``errator`` simply won't have anything in it's narration for that function. Below, we indicate a decorated function with an ``(e)`` before the function name, and skip decoration of some functions. When we get to ``nf5``, the captured fragments are as shown: 328 | 329 | +--------+------------------+---------------------+ 330 | | func | execution point | fragments for funcs | 331 | +========+==================+=====================+ 332 | | (e)nf1 | | nf1 | 333 | +--------+------------------+---------------------+ 334 | | (e)nf2 | | nf1,nf2 | 335 | +--------+------------------+---------------------+ 336 | | nf3 | | nf1,nf2 | 337 | +--------+------------------+---------------------+ 338 | | (e)nf4 | | nf1,nf2,nf4 | 339 | +--------+------------------+---------------------+ 340 | | nf5 | <-- current frame| nf1,nf2,nf4 | 341 | +--------+------------------+---------------------+ 342 | | nf6 | | | 343 | +--------+------------------+---------------------+ 344 | 345 | You can see that there's no narration fragment for function ``nf3``. 346 | 347 | Customizing the narration 348 | ------------------------- 349 | 350 | Suppose you have a function of several variables: 351 | 352 | .. code-block:: python 353 | 354 | @narrate("While I was calling f...") 355 | def f(x, y): 356 | # do stuff 357 | 358 | And a narration with a fixed string doesn't give you enough information as to how the function was called if there was an exception. The ``narrate()`` function allows you to supply it with a callable object instead of a string; this callable will be passed all the arguments that were passed to the function `and must return a string`, which will then be used as the descriptive string for the narration fragment. This function is **only** invoked if the decorated function raises an exception, otherwise it goes uncalled. 359 | 360 | Lambdas provide a nice way to specify a function that yields a string: 361 | 362 | .. code-block:: python 363 | 364 | @narrate(lambda a, b: "While I was calling f with x={} and y={}...".format(a, b)) 365 | def f(x, y): 366 | # do stuff 367 | 368 | But you can supply any callable that can cope with the argument list to the decorated function. This allows your narrations to provide more details regarding the calling context of a particular function, since actual argument values can become part of the narration. 369 | 370 | Additionally, you can add tags to your uses of `narrate()` in order to provide a way to 371 | select only certain narration fragments when you retrieve a narration with 372 | `get_narration()`. Tags are provided using lists of strings like so: 373 | 374 | .. code-block:: python 375 | 376 | @narrate('Some comment', tags=["verbose1", "common"]) 377 | def some_function(): 378 | pass 379 | 380 | And you can then use the `with_tags` keyword argument to `get_narration()` to only 381 | retrieve fragments with the tags you specify. So: 382 | 383 | .. code-block:: python 384 | get_narration(with_tags=["verbose1"]) 385 | 386 | would get the above fragment, while: 387 | 388 | .. code-block:: python 389 | get_narration(with_tags=["wibble"]) 390 | 391 | would not. 392 | 393 | Note that calling `get_narration()` with no tags retrieves every narration fragment 394 | regardless of the tags (or absence thereof), and any use of `narrate()` with no 395 | tags will return that fragment regardless of what tags have been specified in 396 | `get_narration()`. 397 | 398 | Getting more details with contexts 399 | ---------------------------------- 400 | 401 | It may be the case that narration at the function level isn't granular enough. You may have a lengthy function or one that calls out to other libraries, each of which can raise exceptions of their own. It might be helpful to have narration capabilities at a more granular level to address this. 402 | 403 | To support more granular narration, ``errator`` provides a context manager that is created with 404 | a call to ``narrate_cm()``. This context manager acts similarly to the ``narrate()`` 405 | decorator. First, a narration fragment is captured when the context is entered. If the context 406 | exits "normally" the fragment is discarded. However, if an exception is raised during the 407 | context, the fragment is retained as the exception propagates upward. 408 | 409 | Suppose we have a function that does two web service calls during its execution, and we'd like to know narration details around each of these activities if any fails in our function. We can use ``narrate_cm()`` to achieve this: 410 | 411 | .. code-block:: python 412 | 413 | @narrate(lambda a, b:"So call_em was invoked with x={} and y={}".format(a, b)) 414 | def call_em(x, y): 415 | # do some stuff to form the first WS call 416 | with narrate_cm("...and I started the first web service call..."): 417 | # do the web service call 418 | 419 | # extract data and do the second call, computing a string named ws2_req 420 | with narrate_cm(lambda req: "...I started WS call #2 call with {}".format(req), 421 | ws2_req): 422 | # do the second web service call 423 | 424 | # and whatever else... 425 | 426 | This example was constructed to illustrate a couple of uses. Similarly to ``narrate()``, ``narrate_cm()`` can be called either with a fixed string, or a callable that returns a string which will be invoked only if there's an exception raised in the context. 427 | 428 | The first use of ``narrate_cm()`` simply passes a fixed string. If there's an exception during the first web service call, the string is retained, but when reported the string will be indented a few spaces to show that the narration fragment is within the scope of the function's narration. 429 | 430 | The second use of ``narrate_cm()`` passes a lambda as its callable. But unlike passing a callable to ``narrate()``, you must also supply the arguments to give the callable to ``narrate_cm()``, in this case the local variable *ws2_req*. This is because the context manager doesn't know what is important relative to the context-- the function arguments or the local variables. You may pass both postional and keyword arguments to ``narrate_cm()``. 431 | 432 | Similarly to `narrate()`, you can supply a `tags` keyword argument to `narrate_cm()` so 433 | that this narration fragment can be selectively retrieved using `get_narration()`. 434 | The rules governing retrieval of a fragment for `get_narration()` apply here as well. 435 | 436 | Advanced fragment access 437 | ------------------------ 438 | 439 | ``errator`` provides a way to get copies of the actual objects where narration fragments are stored. There are a number of situations where this is useful: 440 | 441 | - if more control over fragment formatting is required 442 | - if retention of the details of an error narration is required 443 | - you're just that way 444 | 445 | You can get these objects by using the ``copy_narration()`` function. Instead of returning a list of strings like ``get_narration()`` does, this function returns a list of ``NarrationFragment`` 446 | objects which are copies of the objects managed by ``errator`` itself. The ``copy_narration()`` function takes the same ``thread`` and ``from_here`` arguments as does ``get_narration()``, so you can control what objects are returned in the same manner. Useful methods on NarrationFragment objects are: 447 | 448 | - ``tell()``, which returns a string that is the fragment's part of the overall narration 449 | - ``fragment_exception_text()``, which returns a string that describes the actual exception; really only useful on the last fragment in the call chain 450 | 451 | Being a lower-level object, you should expect the rest of NarrationFragment's interface to be a bit more volatile, and should stick with calling ``tell()`` if you wish to be isolated from change. 452 | 453 | Verbose narrations 454 | ------------------ 455 | 456 | **NOTE**: Turning on verbose functionality has a material impact on ``errator's`` performance, as the ``inspect`` Python module is consulted to acquire file name and line number information. However, these costs are only incurred when there is an exception. 457 | 458 | The story ``errator`` tells is meant to be user-focused; that is, from the perspective of a program's semantics rather than from that of a stack trace. However, there may be circumstances where it would be helpful to have some of the information in a stack trace merged into the rendered narration. ``errator`` supports this with the ``verbose`` keyword on the ``set_narration_options()`` function. It defaults to ``False``, but if set to ``True``, then each retrieved narration line will be followed by a line that reports the line number, function, and source file associated with the narration fragment. 459 | 460 | Consider this narrated program in a file named verbose.py: 461 | 462 | .. code-block:: python 463 | 464 | from ``errator`` import narrate_cm, narrate, get_narration, set_narration_options 465 | 466 | @narrate("So I started to 'nf1'...") 467 | def nf1(): 468 | nf2() 469 | 470 | @narrate("...which occasioned me to 'nf2'") 471 | def nf2(): 472 | with narrate_cm("during which I started a narration context..."): 473 | nf3() 474 | 475 | @narrate("...and that led me to finally 'nf3'") 476 | def nf3(): 477 | raise Exception("oops") 478 | 479 | if __name__ == "__main__": 480 | set_narration_options(verbose=False) 481 | try: 482 | nf1() 483 | except: 484 | for l in get_narration(): 485 | print(l) 486 | 487 | Which yields the following output when run: 488 | 489 | .. code-block:: 490 | 491 | So I started to 'nf1'... 492 | ...which occasioned me to 'nf2' 493 | during which I started a narration context... 494 | ...and that led me to finally 'nf3', but exception type: Exception, value: 'oops' was raised 495 | 496 | If we set ``verbose=True`` in the ``set_narration_options()`` call, then the output looks like the following: 497 | 498 | .. code-block:: 499 | 500 | So I started to 'nf1'... 501 | line 5 in nf1, /home/tom/errator/docs/verbose.py 502 | ...which occasioned me to 'nf2' 503 | line 10 in nf2, /home/tom/errator/docs/verbose.py 504 | during which I started a narration context... 505 | line 10 in nf2, /home/tom/errator/docs/verbose.py 506 | ...and that led me to finally 'nf3', but exception type: Exception, value: 'oops' was raised 507 | line 14 in nf3, /home/tom/errator/docs/verbose.py 508 | 509 | ...thus letting you see the actual lines being executed when the exception is raised. 510 | 511 | Testing and debugging 512 | --------------------- 513 | 514 | As ``errator`` is meant to help you make sense when something goes wrong, it would be a shame if something 515 | went wrong while ``errator`` was doing its thing. But since ``errator`` users can supply a callable to ``narrate()`` and ``narrate_cm()``, there's the possibility that an error lurks in the callable itself, and ``errator`` could raise an exception in trying to tell you about an exception. Worse, if there is a bug in a callable, you'd only know about it if an exception is raised, which may be difficult to force in testing, or may escape testing and only show up in production. 516 | 517 | To help you find problems earlier, ``errator`` provides an option that changes the rules regarding when fragments, and hence callables, are formatted. By adding: 518 | 519 | .. code-block:: python 520 | 521 | set_default_options(check=True) 522 | 523 | Before entering an ``errator`` decorated function or managed context, you inform ``errator`` that you wish to check the generation of every narration fragment, whether there's been an exception raised or not. You can also set the 'check' option on an existing narration's thread with: 524 | 525 | .. code-block:: python 526 | 527 | set_narration_options(check=True) 528 | 529 | which will set fragment checking only for the current thread's narration (or the thread named with the ``thread=`` argument; see the documentation for ``set_narration_options()`` for details). 530 | 531 | When the ``check`` option is True, every time a decorated function returns or a managed context exits, ``errator`` formats the narration fragment, including calling any callable supplied to exercise the code it refers to. By setting check to True in your testing code, you can be sure that every narration fragment is generated, and hence every callable for a fragment is invoked. This helps you ensure that you have the correct number of arguments to your callable and raises confidence that the callable will operate correctly in a real exception situation (this isn't a guarantee, however, as the conditions that raise an exception my be different from those in testing). 532 | 533 | .. note:: 534 | 535 | You don't want to run production code with ``check`` set to True (it defaults to False). This is because doing so incurs the execution time of every callable where the check==True applies, which can have significant performance impact on your code. ``errator`` normally only invokes the callable if there's an exception, thus sparing your code from the call overhead and extra execution time. So be sure not have the check option set True in production. 536 | 537 | Tidying up stack traces 538 | ----------------------- 539 | 540 | ``errator's`` ``narrate()`` decorator wraps the function being decorated, which means that if you use the various stack and traceback reporting functions in the standard ``traceback`` module, you can get materially longer traces than you'd otherwise like. If you'd rather not see these, ``errator`` supplies a set of wrapper functions that are analogs of the functions in ``traceback`` that strip out the ``errator`` calls from returned objects or printed stack traces. These functions are all argument-compatible with the functions in ``traceback``. Specifically, ``errator`` provides analogs to: 541 | 542 | - extract_tb 543 | - extract_stack 544 | - format_tb 545 | - format_stack 546 | - format_exception_only 547 | - format_exception 548 | - print_tb 549 | - print_exception 550 | - print_exc 551 | - format_exc 552 | - print_last 553 | - print_stack 554 | 555 | ...all of which remove traces of ``errator`` from the output. 556 | 557 | Overhead 558 | -------- 559 | 560 | While every effort is made to do the minimal amount of work required to provide ``errator's`` functionality, there is some unavoidable performance impact over non-'narrated' functions. The amount of impact is dependent on a number of factors, including the version of Python, the narration options activated, and the nature of any narration functions provided to `narrate()`. 561 | 562 | ``errator's`` initial implementation was in pure Python, which introduced significant overhead to the decorated functions. Starting with the 0.3 version of ``errator``, the frequently-executed code has been moved into a C extension generated by Cython, and performance has increased significantly. 563 | 564 | Starting with ``errator`` 0.3, the source repository contains a timing test file, `timing.py`, that illustrates the differences in a variety of different usage scenarios between plain Python functions and narrated functions. Three different types of tests are executed from this file, both in a narrated variety as well as with just plain Python functions: 565 | 566 | * A stack of 10 functions call each other, and an exception is raised and caught in different places in the stack. In the narration variant, when an exception is caught the narration is retrieved but discarded. 567 | * The same stack of 10 functions call each other, but no exceptions are ever raised. 568 | * A simple function is called repeatedly. 569 | 570 | Consider running this test on your target platform if there are performance concerns in the use of ``errator``, as your results may inform what functions that you want to narrate. 571 | 572 | Note that the addition of tags to your calls to `narrate()` and `narrate_cm()` add 573 | overhead, with `narrate_cm()` being the more expensive of the two. 574 | 575 | Usage tips 576 | ---------- 577 | 578 | * When decorating a method with ``narrate()`` and supplying a callable, don't forget to include the ``self`` argument in the callable's argument list. 579 | 580 | * Decorating generator functions gives unexpected results; the function will return immediately with the generator as the value, hence the narration fragment will not be retained. If you wish to get narration for generator functions, you need to use the ``narrate_cm()`` context manager within the generator to accomplish this. 581 | 582 | * At the moment, behavior with coroutines has not been investigated, but almost certainly the current release will do surprising things. This will need further investigation. 583 | -------------------------------------------------------------------------------- /docs/verbose.py: -------------------------------------------------------------------------------- 1 | from errator import narrate_cm, narrate, get_narration, set_narration_options 2 | 3 | 4 | @narrate("So I started to 'nf1'...") 5 | def f1(): 6 | f2() 7 | 8 | 9 | @narrate("...which occasioned me to 'nf2'") 10 | def f2(): 11 | with narrate_cm("during which I started a narration context..."): 12 | f3() 13 | 14 | 15 | @narrate("...and that led me to finally 'nf3'") 16 | def f3(): 17 | raise Exception("oops") 18 | 19 | 20 | if __name__ == "__main__": 21 | set_narration_options(verbose=True) 22 | try: 23 | f1() 24 | except: 25 | for l in get_narration(): 26 | print(l) 27 | -------------------------------------------------------------------------------- /errator.py: -------------------------------------------------------------------------------- 1 | from threading import current_thread, Thread 2 | import traceback 3 | import sys 4 | from io import StringIO 5 | from typing import List, Union, Callable, Iterable 6 | 7 | from _errator import (ErratorException, _default_options, ErratorDeque, _thread_fragments, 8 | NarrationFragment, NarrationFragmentContextManager, narrate, 9 | get_narration) 10 | 11 | __version__ = "0.4" 12 | 13 | 14 | def set_default_options(auto_prune: bool = None, check: bool = None, 15 | verbose: bool = None) -> dict: 16 | """ 17 | Sets default options that are applied to each per-narration thread 18 | :param auto_prune: optional, boolean, defaults to True. If not specified, then don't 19 | change the existing value of the auto_prune default option. Otherwise, set the 20 | default value to the boolean interpretation of auto_prune. 21 | 22 | auto_prune tells errator whether or not to remove narration fragments upon 23 | successful returns from a function/method or exits from a context. If set to 24 | False, fragments are retained on returns/exits, and it is up to the user entirely 25 | to manage the fragment stack using reset_narration(). 26 | :param check: optional, boolean, defaults to False. If not specified, then don't 27 | change the existing value. Otherwise, set the default value to the boolean 28 | interpretation of check. 29 | 30 | The check option changes the logic around fragment text generation. Normally, 31 | fragments only get their text generated in the case of an exception in a 32 | decorated function/method or context. When check is True, the fragment's text is 33 | always generated when the function/method or context finishes, regardless if 34 | there's an exception. This is particularly handy in testing for the cases where 35 | narrate() or narrate_cm() have been given a callable instead of a string-- the 36 | callable will be invoked every time instead of just when there's an exception, 37 | allowing you to make sure that the callable operates properly and itself won't be 38 | a source of errors (which may manifest themselves as exceptions raised within 39 | errator itself). The check option should normally be False, as there's a 40 | performance penalty to pay for always generating fragment text. 41 | :param verbose: boolean, optional, default False. If True, then the returned list of 42 | strings will include information on file, function, and line number. These more 43 | verbose strings will have an embedded \n to split the lines into two. 44 | :return: dict of default options. 45 | """ 46 | if auto_prune is not None: 47 | _default_options["auto_prune"] = bool(auto_prune) 48 | if check is not None: 49 | _default_options["check"] = bool(check) 50 | if verbose is not None: 51 | _default_options["verbose"] = bool(verbose) 52 | 53 | return dict(_default_options) 54 | 55 | 56 | def reset_all_narrations() -> None: 57 | """ 58 | Clears out all narration fragments for all threads 59 | 60 | This function simply removes all narration fragments from tracking. Any options 61 | set on a per-thread narration capture basis are retained. 62 | """ 63 | for k in list(_thread_fragments.keys()): 64 | _thread_fragments[k].clear() 65 | 66 | 67 | def reset_narration(thread: Thread = None, from_here: bool = False) -> None: 68 | """ 69 | Clears out narration fragments for the specified thread 70 | 71 | This function removes narration fragments from the named thread. It can clear them all 72 | out or a subset based on the current execution point of the code. 73 | :param thread: a Thread object (optional). Indicates which thread's narration 74 | fragments are to be cleared. If not specified, the calling thread's narration is 75 | cleared. 76 | :param from_here: boolean, optional, default False. If True, then only clear out the 77 | fragments from the fragment nearest the current stack frame to the fragment where 78 | the exception occurred. This is useful if you have auto_prune set to False for 79 | this thread's narration and you want to clean up the fragments for which you may 80 | have previously retrieved the narration using get_narration(). 81 | """ 82 | if thread is None: 83 | thread = current_thread() 84 | elif not isinstance(thread, Thread): 85 | raise ErratorException("the 'thread' argument isn't an instance " 86 | "of Thread: {}".format(thread)) 87 | d = _thread_fragments.get(thread.name) 88 | if d: 89 | assert isinstance(d, ErratorDeque) 90 | if not from_here: 91 | d.clear() 92 | else: 93 | for i in range(-1, -1 * len(d) - 1, -1): 94 | if d[i].status == NarrationFragment.IN_PROCESS: 95 | if d.auto_prune: 96 | if i != -1: 97 | target = d[i+1] 98 | d.pop_until_true(lambda x: x is target) 99 | else: 100 | target = d[i] 101 | d.pop_until_true(lambda x: x is target) 102 | break 103 | else: 104 | # in this case, nothing was IN_PROCESS, so we should clear all 105 | d.clear() 106 | 107 | 108 | def set_narration_options(thread: Thread = None, auto_prune: bool = None, 109 | check: bool = None, verbose: bool = None) -> None: 110 | """ 111 | Set options for capturing narration for the current thread. 112 | 113 | :param thread: Thread object. If not supplied, the current thread is used. 114 | Identifies the thread whose narration will be impacted by the options. 115 | :param auto_prune: optional, boolean, defaults to True. If not specified, then don't 116 | change the existing value of the auto_prune option. Otherwise, set the 117 | value to the boolean interpretation of auto_prune. 118 | 119 | auto_prune tells errator whether or not to remove narration fragments upon 120 | successful returns from a function/method or exits from a context. If set to 121 | False, fragments are retained on returns/exits, and it is up to the user entirely 122 | to manage the fragment stack using reset_narration(). 123 | :param check: optional, boolean, defaults to False. If not specified, then don't 124 | change the existing value. Otherwise, set the value to the boolean interpretation 125 | of check. 126 | 127 | The check option changes the logic around fragment text generation. Normally, 128 | fragments only get their text generated in the case of an exception in a decorated 129 | function/method or context. When check is True, the fragment's text is always 130 | generated when the function/ method or context finishes, regardless if there's an 131 | exception. This is particularly handy in testing for the cases where narrate() or 132 | narrate_cm() have been given a callable instead of a string-- the callable will be 133 | invoked every time instead of just when there's an exception, allowing you to make 134 | sure that the callable operates properly and itself won't be a source of errors 135 | (which may manifest themselves as exceptions raised within errator itself). The 136 | check option should normally be False, as there's a performance penalty to pay for 137 | always generating fragment text. 138 | :param verbose: boolean, optional, default False. If True, then the returned list of 139 | strings will include information on file, function, and line number. These more 140 | verbose strings will have an embedded \n to split the lines into two. 141 | """ 142 | if thread is None: 143 | thread = current_thread() 144 | elif not isinstance(thread, Thread): 145 | raise ErratorException("the 'thread' argument isn't an instance " 146 | "of Thread: {}".format(thread)) 147 | try: 148 | d = _thread_fragments[thread.name] 149 | d.set_auto_prune(auto_prune).set_check(check).set_verbose(verbose) 150 | except KeyError: 151 | # this should never happen now that _thread_fragments is a defaultdict 152 | _thread_fragments[thread.name] = ErratorDeque(auto_prune=bool(auto_prune) 153 | if auto_prune is not None 154 | else None, 155 | check=bool(check) 156 | if check is not None 157 | else None) 158 | 159 | 160 | def copy_narration(thread: Thread = None, 161 | from_here: bool = False) -> List[NarrationFragment]: 162 | """ 163 | Acquire copies of the NarrationFragment objects for the current exception 164 | narration. 165 | 166 | This method returns a list of NarrationFragment objects that capture all the narration 167 | fragments for the current narration for a specific thread. The actual narration can 168 | then be cleared, but this list will be unaffected. 169 | :param thread: optional, instance of Thread. If unspecified, the current thread is 170 | used. 171 | :param from_here: boolean, optional, default False. If True, then the list of 172 | fragments returned is from the narration fragment nearest the active stack frame 173 | and down to the exception origin, not from the most global level to the exception. 174 | This is useful from where the exception is actually caught, as it provides a way 175 | to only show the narration from this point forward. However, not showing all the 176 | fragments may actually hide important aspects of the narration, so bear this in 177 | mind when using this to prune the narration. Use in conjuction with the auto_prune 178 | option set to False to allow several stack frames to return before collecting the 179 | narration (be sure to manually clean up the narration when auto_prune is False). 180 | :return: a list of NarrationFragment objects. The first item is the most global in the 181 | narration. 182 | """ 183 | if thread is None: 184 | thread = current_thread() 185 | elif not isinstance(thread, Thread): 186 | raise ErratorException("the 'thread' argument isn't an instance " 187 | "of Thread: {}".format(thread)) 188 | d = _thread_fragments.get(thread.name) 189 | if not d: 190 | l = [] 191 | elif not from_here: 192 | l = [o.__class__.clone(o) for o in d] 193 | else: 194 | l = [] 195 | for i in range(-1, -1 * len(d) - 1, -1): 196 | if d[i].status == NarrationFragment.IN_PROCESS: 197 | for j in range(i, 0, 1): 198 | l.append(NarrationFragment.clone(d[j])) 199 | break 200 | return l 201 | 202 | 203 | _magic_name = "narrate_it" 204 | 205 | 206 | def narrate_cm(text_or_func: Union[Callable, str], *args, 207 | tags: Iterable[str] = None, **kwargs): 208 | """ 209 | Create a context manager that captures some narration of the operations being done 210 | within it 211 | 212 | This function returns an object that is a context manager which captures the narration 213 | of the code executing within the context. It has similar behaviour to narrate() in 214 | that either a fixed string can be provided or a callable that will be invoked if there 215 | is an exception raised during the execution of the context. 216 | 217 | :param text_or_func: either a string that will be captured and rendered if the 218 | function fails, or else a callable with the same signature as the function/method 219 | that is being decorated that will only be called if the function/method raises and 220 | exception; in this case, the callable will be invoked with the (possibly modified) 221 | arguments that were passed to the function. The callable must return a string, and 222 | that will be used for the string that describes the execution of the 223 | function/method. 224 | :param tags: optional, iterable of strings. If supplied, then the fragment for 225 | this narration can be optionally retrieved using get_narration() by the caller 226 | of that function supplying one or more of the same string tags that appear in 227 | the 'tags' argument of this use of the context manager. If tags aren't supplied, 228 | then this narration fragment appears in any list of strings returned by 229 | get_narration(), regardless if tags are supplied in that call or not. 230 | :param args: sequence of positional arguments; if str_or_func is a callable, these 231 | will be the positional arguments passed to the callable. If str_or_func is a 232 | string itself, positional arguments are ignored. 233 | :param kwargs: keyword arguments; if str_or_func is a callable, then these will be the 234 | keyword arguments passed to the callable. If str_or_func is a string itself, 235 | keyword arguments are ignored. 236 | :return: An errator context manager (NarrationFragmentContextManager) 237 | """ 238 | ifsf = NarrationFragmentContextManager.get_instance(text_or_func, None, *args, 239 | **kwargs) 240 | if tags is not None: 241 | ifsf.set_tags(frozenset(tags)) 242 | return ifsf 243 | 244 | 245 | # Traceback sanitizers 246 | # errator leaves a bunch of cruft in the stack trace when an exception occurs; this cruft 247 | # appears when you use the various functions in the standard traceback module. The 248 | # following functions provide analogs to a number of the functions in traceback, but they 249 | # filter out the internal function calls to errator functions. 250 | 251 | def extract_tb(tb, limit: int = None) -> list: 252 | """ 253 | behaves like traceback.extract_tb, but removes errator functions from the trace 254 | :param tb: traceback to process 255 | :param limit: optional; int. The number of stack frame entries to return; the actual 256 | number returned may be lower once errator calls are removed 257 | :return: a list of 4-tuples containing (filename, line number, function name, text) 258 | """ 259 | return [f for f in traceback.extract_tb(tb, limit) if _magic_name not in f[2]] 260 | 261 | 262 | def extract_stack(f=None, limit: int = None) -> list: 263 | """ 264 | behaves like traceback.extract_stack, but removes errator functions from the trace 265 | :param f: optional; specifies an alternate stack frame to start at 266 | :param limit: optional; int. The number of stack frame entries to return; the actual 267 | number returned may be lower once errator calls are removed 268 | :return: a list of 4-tuples containing (filename, line number, function name, text) 269 | """ 270 | return [f for f in traceback.extract_stack(f, limit) if _magic_name not in f[2]] 271 | 272 | 273 | def format_tb(tb, limit: int = None) -> list: 274 | """ 275 | behaves like traceback.format_tb, but removes errator functions from the trace 276 | :param tb: The traceback you wish to format 277 | :param limit: optional; int. The number of stack frame entries to return; the actual 278 | number returned may be lower once errator calls are removed 279 | :return: a list of formatted strings for the trace 280 | """ 281 | return traceback.format_list(extract_tb(tb, limit)) 282 | 283 | 284 | def format_stack(f=None, limit: int = None) -> list: 285 | """ 286 | behaves like traceback.format_stack, but removes errator functions from the trace 287 | :param f: optional; specifies an alternate stack frame to start at 288 | :param limit: optional; int. The number of stack frame entries to return; the actual 289 | number returned may be lower once errator calls are removed 290 | :return: a list of formatted strings for the trace 291 | """ 292 | return traceback.format_list(extract_stack(f, limit)) 293 | 294 | 295 | format_exception_only = traceback.format_exception_only 296 | 297 | 298 | def format_exception(etype: type, evalue: Exception, tb, limit: int = None) -> list: 299 | """ 300 | behaves like traceback.format_exception, but removes errator functions from the trace 301 | :param etype: exeption type 302 | :param evalue: exception value 303 | :param tb: traceback to print; these are the values returne by sys.exc_info() 304 | :param limit: optional; int. The number of stack frame entries to return; the actual 305 | number returned may be lower once errator calls are removed 306 | :return: a list of formatted strings for the trace 307 | """ 308 | tb = format_tb(tb, limit) 309 | exc = format_exception_only(etype, evalue) 310 | return tb + exc 311 | 312 | 313 | def print_tb(tb, limit: int = None, file=sys.stderr) -> None: 314 | """ 315 | behaves like traceback.print_tb, but removes errator functions from the trace 316 | :param tb: traceback to print; these are the values returne by sys.exc_info() 317 | :param limit: optional; int. The number of stack frame entries to return; the actual 318 | number returned may be lower once errator calls are removed 319 | :param file: optional; open file-like object to write() to; if not specified defaults 320 | to sys.stderr 321 | """ 322 | for l in format_tb(tb, limit): 323 | file.write(l.decode() if hasattr(l, "decode") else l) 324 | file.flush() 325 | 326 | 327 | def print_exception(etype: type, evalue: Exception, tb, limit: int = None, 328 | file=sys.stderr) -> None: 329 | """ 330 | behaves like traceback.print_exception, but removes errator functions from the trace 331 | :param etype: exeption type 332 | :param evalue: exception value 333 | :param tb: traceback to print; these are the values returne by sys.exc_info() 334 | :param limit: optional; int. The number of stack frame entries to return; the actual 335 | number returned may be lower once errator calls are removed 336 | :param file: optional; open file-like object to write() to; if not specified defaults 337 | to sys.stderr 338 | """ 339 | for l in format_exception(etype, evalue, tb, limit): 340 | file.write(l.decode() if hasattr(l, "decode") else l) 341 | file.flush() 342 | 343 | 344 | def print_exc(limit: int = None, file=sys.stderr) -> None: 345 | """ 346 | behaves like traceback.print_exc, but removes errator functions from the trace 347 | :param limit: optional; int. The number of stack frame entries to return; the actual 348 | number returned may be lower once errator calls are removed 349 | :param file: optional; open file-like object to write() to; if not specified defaults 350 | to sys.stderr 351 | """ 352 | etype, evalue, tb = sys.exc_info() 353 | print_exception(etype, evalue, tb, limit, file) 354 | 355 | 356 | def format_exc(limit: int = None) -> str: 357 | """ 358 | behaves like traceback.format_exc, but removes errator functions from the trace 359 | :param limit: optional; int. The number of stack frame entries to return; the actual 360 | number returned may be lower once errator calls are removed 361 | :return: a string containing the formatted exception and traceback 362 | """ 363 | f = StringIO() 364 | print_exc(limit, f) 365 | return f.getvalue() 366 | 367 | 368 | def print_last(limit: int = None, file=sys.stderr) -> None: 369 | """ 370 | behaves like traceback.print_last, but removes errator functions from the trace. As 371 | noted in the man page for traceback.print_last, this will only work when an exception 372 | has reached the interactive prompt 373 | 374 | :param limit: optional; int. The number of stack frame entries to return; the actual 375 | number returned may be lower once errator calls are removed 376 | :param file: optional; open file-like object to write() to; if not specified defaults 377 | to sys.stderr 378 | """ 379 | print_exception(getattr(sys, "last_type", None), getattr(sys, "last_value", None), 380 | getattr(sys, "last_traceback", None), limit, file) 381 | 382 | 383 | def print_stack(f=None, limit: int = None, file=sys.stderr) -> None: 384 | """ 385 | behaves like traceback.print_stack, but removes errator functions from the trace. 386 | :param f: optional; specifies an alternate stack frame to start at 387 | :param limit: optional; int. The number of stack frame entries to return; the actual 388 | number returned may be lower once errator calls are removed 389 | :param file: optional; open file-like object to write() to; if not specified defaults 390 | to sys.stderr 391 | """ 392 | for l in format_stack(f, limit): 393 | file.write(l.decode() if hasattr(l, "decode") else l) 394 | file.flush() 395 | 396 | 397 | __all__ = ("narrate", "narrate_cm", "copy_narration", "NarrationFragment", 398 | "NarrationFragmentContextManager", "reset_all_narrations", "reset_narration", 399 | "get_narration", "set_narration_options", "ErratorException", 400 | "set_default_options", "extract_tb", "extract_stack", "format_tb", 401 | "format_stack", "format_exception_only", "format_exception", "print_tb", 402 | "print_exception", "print_exc", "format_exc", "print_last", "print_stack") 403 | -------------------------------------------------------------------------------- /manylinux_build.bsh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # This builds for Python 3.6-3.9, excluding 3.8, which is done in another script 4 | 5 | mkdir -p /io/wheels 6 | 7 | rm -f /io/wheels/*.whl 8 | 9 | # compile 10 | for VER in cp36-cp36m cp37-cp37m cp39-cp39; do 11 | PYBIN=/opt/python/${VER}/bin 12 | echo ========================='>' Processing "${PYBIN}" 13 | ${PYBIN}/python -m pip install --upgrade pip 14 | "${PYBIN}/pip" install -r /io/requirements.txt 15 | "${PYBIN}/pip" wheel /io/ --no-deps -w /io/wheels 16 | echo 17 | done 18 | 19 | # repair; make manylinux wheels 20 | for WHEEL in /io/wheels/*.whl; do 21 | if ! auditwheel show "${WHEEL}"; then 22 | echo "Skipping non-platform wheel ${WHEEL}" 23 | else 24 | auditwheel repair "${WHEEL}" --plat manylinux2010_x86_64 -w /io/wheels 25 | fi 26 | done 27 | 28 | # install and test 29 | for VER in cp36-cp36m cp37-cp37m cp39-cp39; do 30 | PYBIN=/opt/python/${VER}/bin 31 | "${PYBIN}"/pip install errator --no-index -f /io/wheels 32 | "${PYBIN}"/pytest /io/tests.py 33 | done 34 | 35 | cd /io/wheels 36 | rm `ls *.whl|grep -v manylinux` 37 | -------------------------------------------------------------------------------- /manylinux_build_py38.bsh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | mkdir -p /io/wheels 4 | 5 | # NOTE: this is a cleanup build for Python 3.8 as the manylinux2010 build can't 6 | # seem to build an extension properly. This one is expected to run after the main 7 | # build and from within manylinux1 8 | 9 | # compile 10 | for VER in cp38-cp38; do 11 | PYBIN=/opt/python/${VER}/bin 12 | echo Processing "${PYBIN}" 13 | ${PYBIN}/python -m pip install --upgrade pip 14 | "${PYBIN}"/pip install -r /io/requirements.txt 15 | "${PYBIN}"/pip wheel /io/ --no-deps -w /io/wheels 16 | done 17 | 18 | # repair; make manylinux wheels 19 | for WHEEL in /io/wheels/*cp38*.whl; do 20 | if ! auditwheel show "${WHEEL}"; then 21 | echo "Skipping non-platform wheel ${WHEEL}" 22 | else 23 | auditwheel repair "${WHEEL}" --plat manylinux2014_x86_64 -w /io/wheels 24 | fi 25 | done 26 | 27 | # install and test 28 | for VER in cp38-cp38; do 29 | PYBIN=/opt/python/${VER}/bin 30 | "${PYBIN}"/pip install errator --no-index -f /io/wheels 31 | "${PYBIN}"/pytest /io/tests.py 32 | done 33 | 34 | cd /io/wheels 35 | rm `ls *.whl|grep -v manylinux` 36 | -------------------------------------------------------------------------------- /requirements.txt: -------------------------------------------------------------------------------- 1 | Cython==0.29.21 2 | pytest==6.2.2 3 | -------------------------------------------------------------------------------- /setup.cfg: -------------------------------------------------------------------------------- 1 | [metadata] 2 | description-file = README.rst 3 | -------------------------------------------------------------------------------- /setup.py: -------------------------------------------------------------------------------- 1 | """ 2 | Use this setup for Python3 builds of errator 3 | """ 4 | 5 | from distutils.core import setup 6 | from Cython.Build import cythonize 7 | 8 | ext_modules = [] 9 | ext_modules.extend(cythonize("_errator.pyx", 10 | compiler_directives={'language_level': '3', 11 | 'embedsignature': True})) 12 | 13 | version = "0.4" 14 | 15 | 16 | def get_readme(): 17 | with open("README.rst", "r") as f: 18 | readme = f.read() 19 | return readme 20 | 21 | 22 | setup( 23 | name="errator", 24 | ext_modules=ext_modules, 25 | py_modules=["errator"], 26 | version=version, 27 | description="Errator allows you to create human-readable exception narrations", 28 | long_description=get_readme(), 29 | author="Tom Carroll", 30 | author_email="tcarroll@incisivetech.co.uk", 31 | url="https://github.com/haxsaw/errator", 32 | download_url="https://github.com/haxsaw/errator/archive/%s.tar.gz" % version, 33 | keywords=["exception", "logging", "traceback", "stacktrace"], 34 | classifiers=[], 35 | license="MIT" 36 | ) 37 | -------------------------------------------------------------------------------- /tests.py: -------------------------------------------------------------------------------- 1 | import traceback 2 | import sys 3 | import threading 4 | from errator import * 5 | from _errator import (_thread_fragments,) 6 | from io import StringIO 7 | 8 | 9 | def test01(): 10 | """ 11 | test01: Check string function of StoryFragment 12 | :return: 13 | """ 14 | the_text = "some text" 15 | sf = NarrationFragment(the_text, test01) 16 | assert the_text in sf.tell(), "The text we supplied wasn't in the fragment's output" 17 | 18 | 19 | def test02(): 20 | """ 21 | test02: Check that a callable gets invoked 22 | :return: 23 | """ 24 | text = "callable text" 25 | 26 | def f(a1, kw1=""): 27 | return "f:{} {}".format(a1, kw1) 28 | 29 | sf = NarrationFragment(f, test02, "callable", kw1="text") 30 | assert text in sf.tell(), "The callable didn't manage to return the expected string" 31 | 32 | 33 | def test03(): 34 | """ 35 | test03: Check that we push a fragment into our thread's list in a function 36 | :return: 37 | """ 38 | reset_all_narrations() 39 | tname = threading.current_thread().name 40 | 41 | @narrate("Visiting inner") 42 | def inner(x, y): 43 | di = _thread_fragments[tname] 44 | assert len(di) == 1, "Expected 1 fragment, got {}".format(len(di)) 45 | return True 46 | 47 | inner(1, 2) 48 | d = _thread_fragments[tname] 49 | assert len(d) == 0, "Expected 0 fragments, got {}".format(len(d)) 50 | 51 | 52 | def test04(): 53 | """ 54 | test04: Check that we push a fragment into our deque when we use a formatting function 55 | :return: 56 | """ 57 | reset_all_narrations() 58 | tname = threading.current_thread().name 59 | 60 | @narrate(lambda x, y: "{}-{}".format(x, y)) 61 | def f(x, y): 62 | di = _thread_fragments[tname] 63 | assert len(di) == 1, "expected 1 fragment, got {}".format(len(di)) 64 | return True 65 | 66 | f(4, 5) 67 | d = _thread_fragments[tname] 68 | assert len(d) == 0, "Expected 0, fragments, got {}".format(len(d)) 69 | 70 | 71 | def test05(): 72 | """ 73 | test05: Check that we accumulate fragments in multiple errated function calls 74 | :return: 75 | """ 76 | reset_all_narrations() 77 | tname = threading.current_thread().name 78 | 79 | @narrate("Calling nf1") 80 | def f1(): 81 | di = _thread_fragments[tname] 82 | assert len(di) == 1, "expected 1 fragment, got {}".format(len(di)) 83 | f2() 84 | assert len(di) == 1, "expected 1 fragment, got {}".format(len(di)) 85 | 86 | @narrate("Calling nf2") 87 | def f2(): 88 | di = _thread_fragments[tname] 89 | assert len(di) == 2, "expected 2 fragments, got {}".format(len(di)) 90 | 91 | f1() 92 | d = _thread_fragments[tname] 93 | assert len(d) == 0, "expected 0 fragments, got {}".format(len(d)) 94 | 95 | 96 | def test06(): 97 | """ 98 | test06: Check that if we raise an exception, we keep the fragments 99 | :return: 100 | """ 101 | reset_all_narrations() 102 | tname = threading.current_thread().name 103 | 104 | class T6Exception(Exception): 105 | pass 106 | 107 | extext = "We got's messed up!" 108 | 109 | @narrate("Buggered!") 110 | def broken(x, y): 111 | raise T6Exception(extext) 112 | 113 | try: 114 | broken(1, 2) 115 | assert False, "We should have encountered the exception our function raised" 116 | except T6Exception as e: 117 | assert extext in str(e) 118 | d = _thread_fragments[tname] 119 | assert len(d) == 1, "Expected 1 fragment, got {}".format(len(d)) 120 | sf = d.pop() 121 | assert isinstance(sf, NarrationFragment) 122 | assert "Buggered!" in sf.tell() 123 | 124 | 125 | def test07(): 126 | """ 127 | test07: Check that we work with methods too 128 | :return: 129 | """ 130 | 131 | reset_all_narrations() 132 | tname = threading.current_thread().name 133 | 134 | class T07(object): 135 | @narrate("T7 narration") 136 | def f(self, x, y): 137 | di = _thread_fragments[tname] 138 | assert x == 4, "x is {}, not 4".format(x) 139 | assert y == 5, "y is {}".format(y) 140 | assert len(di) == 1, "Expected 1 fragment, got {}".format(len(di)) 141 | 142 | t07 = T07() 143 | t07.f(4, 5) 144 | d = _thread_fragments[tname] 145 | assert len(d) == 0, "Expected 0 fragments, got {}".format(len(d)) 146 | 147 | 148 | def test08(): 149 | """ 150 | test08: Check that we promote any non-standard attributes on a callable to the wrapper 151 | :return: 152 | """ 153 | reset_all_narrations() 154 | tname = threading.current_thread().name 155 | 156 | def deco(m): 157 | m.wibble = "Surprise!" 158 | return m 159 | 160 | @narrate("Trying to wibble") 161 | @deco 162 | def f(): 163 | di = _thread_fragments[tname] 164 | assert len(di) == 1, "Expected 1 fragment, got {}".format(len(di)) 165 | return 166 | 167 | @deco 168 | @narrate("Trying to wobble") 169 | def f1(): 170 | di = _thread_fragments[tname] 171 | assert len(di) == 1, "Expected 1 fragment, to {}".format(len(di)) 172 | return 173 | 174 | f() 175 | f1() 176 | d = _thread_fragments[tname] 177 | assert len(d) == 0, "Expected 0 fragments, got {}".format(len(d)) 178 | assert hasattr(f, "wibble") and f.wibble == "Surprise!" 179 | assert hasattr(f1, "wibble") and f.wibble == "Surprise!" 180 | 181 | 182 | def test09(): 183 | """ 184 | test09: Check that the context manager works properly 185 | :return: 186 | """ 187 | reset_all_narrations() 188 | tname = threading.current_thread().name 189 | 190 | for i in range(5): 191 | with narrate_cm(lambda x: "iteration {}".format(x), i): 192 | di = _thread_fragments[tname] 193 | assert len(di) == 1, "Expected 1 fragment, got {}, i={}".format(len(di), i) 194 | d = _thread_fragments[tname] 195 | assert len(d) == 0, "Expected 0 fragments, got {}".format(len(d)) 196 | 197 | 198 | def test10(): 199 | """ 200 | test10: ensure we survive clearing narrations in the middle of one 201 | :return: 202 | """ 203 | reset_all_narrations() 204 | tname = threading.current_thread().name 205 | 206 | @narrate("will it survive?") 207 | def f(): 208 | di = _thread_fragments[tname] 209 | assert len(di) == 1, "Expected 1 fragment, got {}".format(len(di)) 210 | reset_all_narrations() 211 | 212 | f() 213 | d = _thread_fragments[tname] 214 | assert len(d) == 0, "Expected 0 fragments, got {}".format(len(d)) 215 | 216 | 217 | def test11(): 218 | """ 219 | test11: ensure that copied fragments are retained 220 | """ 221 | reset_all_narrations() 222 | 223 | @narrate("nf1") 224 | def f1(): 225 | return f2() 226 | 227 | @narrate("nf2") 228 | def f2(): 229 | return f3() 230 | 231 | @narrate("nf3") 232 | def f3(): 233 | frags = copy_narration() 234 | reset_narration() 235 | return frags 236 | 237 | the_story = f1() 238 | for name in ("nf1", "nf2", "nf3"): 239 | assert any([name in nf.tell() for nf in the_story]), "didn't find function {}".format(name) 240 | 241 | 242 | def test12(): 243 | """ 244 | test12: ensure we work with generators 245 | """ 246 | reset_all_narrations() 247 | tname = threading.current_thread().name 248 | 249 | def g(x): 250 | with narrate_cm("only once"): 251 | di = _thread_fragments[tname] 252 | for i in range(x): 253 | assert len(di) == 1, "Expected 1 fragment, got {}".format(len(di)) 254 | yield i 255 | 256 | for _ in g(5): 257 | pass 258 | 259 | d = _thread_fragments[tname] 260 | assert len(d) == 0, "Expected 0 fragments, got {}".format(len(d)) 261 | 262 | 263 | def test13(): 264 | """ 265 | test13: Check we do the right thing if we raise inside a context 266 | """ 267 | reset_all_narrations() 268 | tname = threading.current_thread().name 269 | 270 | class Test13Exc(Exception): 271 | pass 272 | 273 | try: 274 | with narrate_cm("Blammo"): 275 | raise Test13Exc("oh dear") 276 | assert False, "we should have raised from our context block" 277 | except Test13Exc: 278 | d = _thread_fragments[tname] 279 | assert len(d) == 1, "Expecting 1 fragment, got {}".format(len(d)) 280 | 281 | 282 | def test14(): 283 | """ 284 | test14: check that only the frame where the exception occurred has exc details 285 | """ 286 | reset_all_narrations() 287 | 288 | exc_text = "Here we go!" 289 | 290 | class T14Exc(Exception): 291 | pass 292 | 293 | @narrate("nf1") 294 | def f1(): 295 | f2() 296 | 297 | @narrate("nf2") 298 | def f2(): 299 | f3() 300 | 301 | @narrate("nf3") 302 | def f3(): 303 | raise T14Exc(exc_text) 304 | 305 | try: 306 | f1() 307 | assert False, "We should have had an exception bubble up!" 308 | except T14Exc as e: 309 | assert str(e) == exc_text 310 | lines = get_narration() 311 | assert len(lines) == 3, "Expecting 3 lines, got {}".format(len(lines)) 312 | assert exc_text not in lines[0] 313 | assert exc_text not in lines[1] 314 | assert exc_text in lines[2] 315 | 316 | 317 | def test15(): 318 | """ 319 | test15: check that we can get only some of the narration text 320 | """ 321 | reset_all_narrations() 322 | set_narration_options(auto_prune=False) 323 | 324 | @narrate("nf1") 325 | def f1(): 326 | f2() 327 | 328 | @narrate("nf2") 329 | def f2(): 330 | try: 331 | f3() 332 | except KeyError: 333 | lines = get_narration(from_here=True) 334 | assert len(lines) == 3 335 | reset_narration(from_here=True) 336 | 337 | @narrate("nf3") 338 | def f3(): 339 | f4() 340 | 341 | @narrate("nf4") 342 | def f4(): 343 | raise KeyError("wibble") 344 | 345 | f1() 346 | l2 = get_narration() 347 | assert len(l2) == 1, "Expected there to be one left, got {}".format(len(l2)) 348 | set_narration_options(auto_prune=True) 349 | reset_all_narrations() 350 | 351 | 352 | def test16(): 353 | """ 354 | test16: check that we behave right when we clear from the first fragment, auto_prune=True 355 | """ 356 | reset_all_narrations() 357 | set_narration_options(auto_prune=True) 358 | 359 | @narrate("nf1") 360 | def f1(): 361 | try: 362 | f2() 363 | except KeyError: 364 | reset_narration(from_here=True) 365 | lines = get_narration(from_here=True) 366 | assert len(lines) == 1, "Expected 1 line, got {}".format(len(lines)) 367 | 368 | @narrate("nf2") 369 | def f2(): 370 | f3() 371 | 372 | @narrate("nf3") 373 | def f3(): 374 | raise KeyError("oopsie!") 375 | 376 | f1() 377 | l2 = get_narration() 378 | assert len(l2) == 0, "Expected 0 lines, got {}".format(len(l2)) 379 | 380 | 381 | def test17(): 382 | """ 383 | test17: check that we behave right when we clear from teh first fragment, auto_prune=False 384 | """ 385 | reset_all_narrations() 386 | set_narration_options(auto_prune=False) 387 | 388 | @narrate("nf1") 389 | def f1(): 390 | try: 391 | f2() 392 | except KeyError: 393 | reset_narration(from_here=True) 394 | lines = get_narration(from_here=True) 395 | assert len(lines) == 0, "Expected 0 lines, got {}".format(len(lines)) 396 | 397 | @narrate("nf2") 398 | def f2(): 399 | f3() 400 | 401 | @narrate("nf3") 402 | def f3(): 403 | raise KeyError("youch!") 404 | 405 | lines = get_narration() 406 | assert len(lines) == 0, "Expected 0 lines, got {}".format(len(lines)) 407 | set_narration_options(auto_prune=True) 408 | 409 | 410 | def test18(): 411 | """ 412 | test18: check behavior clear from first fragment frame with contexts, auto_prune=True 413 | :return: 414 | """ 415 | reset_all_narrations() 416 | set_narration_options(auto_prune=True) 417 | 418 | def f1(): 419 | with narrate_cm("nf1 context"): 420 | try: 421 | f2() 422 | except KeyError: 423 | reset_narration(from_here=True) 424 | lines = get_narration(from_here=True) 425 | assert len(lines) == 1, "Expecting 1 lines, got {}".format(len(lines)) 426 | 427 | @narrate("nf2") 428 | def f2(): 429 | lines = get_narration() 430 | assert len(lines) == 2, "Expecting 2 lines, got {}".format(len(lines)) 431 | with narrate_cm("nf3 context"): 432 | lines = get_narration() 433 | assert len(lines) == 3, "Expecting 3 lines, got {}".format(len(lines)) 434 | raise KeyError 435 | 436 | f1() 437 | l2 = get_narration() 438 | assert len(l2) == 0, "Expecting no lines, got {}".format(len(l2)) 439 | 440 | 441 | def test18a(): 442 | """ 443 | test18a: check that clearing all narrations from the midst of a chain of calls doesn't break more global processing 444 | """ 445 | reset_all_narrations() 446 | set_narration_options(auto_prune=True) 447 | 448 | @narrate("in f1") 449 | def f1(): 450 | try: 451 | f2() 452 | except KeyError: 453 | lines = get_narration() 454 | assert 1 == len(lines), "got {}".format(lines) 455 | 456 | @narrate("in f2") 457 | def f2(): 458 | try: 459 | f3() 460 | except KeyError: 461 | lines = get_narration() 462 | assert len(lines) == 3, "only {} lines".format(len(lines)) 463 | reset_narration() 464 | 465 | @narrate("in f3") 466 | def f3(): 467 | raise KeyError() 468 | 469 | f1() 470 | lines = get_narration() 471 | assert 0 == len(lines), "had {} lines".format(len(lines)) 472 | 473 | 474 | def test19(): 475 | """ 476 | test19: making sure the example in the quickstart works as it says! 477 | """ 478 | reset_all_narrations() 479 | 480 | @narrate("I was just showing how it works") 481 | def f1(): 482 | raise KeyError("when I blorked") 483 | 484 | try: 485 | f1() 486 | except KeyError: 487 | lines = get_narration() 488 | assert len(lines) == 1, "Expecting 1 line, got {}".format(len(lines)) 489 | 490 | 491 | def test20(): 492 | """ 493 | test20: 494 | :return: if auto_prune is off, check that 'clear to hear' removes the first tracked func 495 | """ 496 | reset_all_narrations() 497 | set_narration_options(auto_prune=False) 498 | 499 | @narrate("nf1") 500 | def f1(): 501 | f2() 502 | lines = get_narration() 503 | assert "nf1" not in lines 504 | 505 | def f2(): 506 | try: 507 | f3() 508 | except KeyError: 509 | lines = get_narration(from_here=True) 510 | assert "nf1" in lines 511 | reset_narration(from_here=True) 512 | 513 | @narrate("nf3") 514 | def f3(): 515 | raise KeyError("ouch") 516 | 517 | f1() 518 | set_narration_options(auto_prune=True) 519 | 520 | 521 | def test21(): 522 | """ 523 | test21: Check that auto_prune==False still gets all clear when all funcs return 524 | """ 525 | reset_all_narrations() 526 | set_narration_options(auto_prune=False) 527 | 528 | @narrate("nf1") 529 | def f1(): 530 | f2() 531 | 532 | @narrate("nf2") 533 | def f2(): 534 | raise KeyError("another mistake? I'm fired") 535 | 536 | try: 537 | f1() 538 | except KeyError: 539 | lines = get_narration() 540 | assert len(lines) == 2, "Expected 2 lines, got {}".format(len(lines)) 541 | reset_narration(from_here=True) 542 | lines = get_narration() 543 | assert len(lines) == 0, "Expected no lines, got {}".format(len(lines)) 544 | 545 | 546 | def test22(): 547 | """ 548 | test22: ensure proper behavior when decorating a method on a class 549 | """ 550 | reset_all_narrations() 551 | set_narration_options(auto_prune=True) 552 | 553 | class T22(object): 554 | @narrate(lambda _, x: "x is {}".format(x)) 555 | def m(self, x): 556 | raise KeyError("die m die") 557 | 558 | o = T22() 559 | try: 560 | o.m(5) 561 | except KeyError: 562 | assert len(get_narration()) == 1, "We should have have a single narration line" 563 | 564 | 565 | def test23(): 566 | """ 567 | test23: check that the 'check' option works for functions 568 | """ 569 | reset_all_narrations() 570 | set_narration_options(auto_prune=False, check=True) 571 | 572 | @narrate(lambda v: "Entering nf1 with {}".format(v)) 573 | def f1(x): 574 | return f2(x * 2) 575 | 576 | @narrate(lambda v: "Entering nf2 with {}".format(v)) 577 | def f2(y): 578 | y += 2 579 | return y 580 | 581 | a = f1(5) 582 | assert a == 12, "Unexpected value for a: {}".format(a) 583 | frags = copy_narration() 584 | assert len(frags) == 2, "Expected 2 fragments, got {}".format(len(frags)) 585 | tof0 = frags[0].text_or_func 586 | assert "nf1" in tof0, "the name 'nf1' wasn't in the fragment: {}".format(tof0) 587 | assert "5" in tof0, "the value 5 isn't in the fragment: {}".format(tof0) 588 | tof1 = frags[1].text_or_func 589 | assert "nf2" in tof1, "the name 'nf2' wasn't in the fragment: {}".format(tof1) 590 | assert "10" in tof1, "the value 10 wasn't int he fragment: {}".format(tof1) 591 | reset_all_narrations() 592 | set_narration_options(auto_prune=True, check=False) 593 | 594 | 595 | def test24(): 596 | """ 597 | test24: check that the 'check' option works for context managers 598 | """ 599 | reset_all_narrations() 600 | set_narration_options(auto_prune=False, check=True) 601 | 602 | initial = 5 603 | with narrate_cm(lambda v: "Exiting c1 with {}".format(v), initial): 604 | initial *= 2 605 | with narrate_cm(lambda w: "Exiting c2 with {}".format(w), initial): 606 | initial += 2 607 | 608 | assert initial == 12, "Unexpected value for initial: {}".format(initial) 609 | frags = copy_narration() 610 | assert len(frags) == 2, "Expected 2 fragments, got {}".format(len(frags)) 611 | tof0 = frags[0].text_or_func 612 | assert "c1" in tof0, "the name c1 wasn't in the fragment: {}".format(tof0) 613 | assert "5" in tof0, "the value 5 wasn't in the fragment: {}".format(tof0) 614 | tof1 = frags[1].text_or_func 615 | assert "c2" in tof1, "the name c2 wasn't int he fragment: {}".format(tof1) 616 | assert "10" in tof1, "the value 10 wasn't in the fragment: {}".format(tof1) 617 | reset_all_narrations() 618 | set_narration_options(auto_prune=True, check=False) 619 | 620 | 621 | def test25(): 622 | """ 623 | test25: check that we raise a fragment formatting exception for a bad callable 624 | """ 625 | 626 | set_narration_options(check=True) 627 | reset_all_narrations() 628 | 629 | try: 630 | with narrate_cm(lambda arg: "oops"): 631 | pass 632 | except ErratorException: 633 | assert any(">>>>" in l for l in get_narration()), "no format failure messages" 634 | except Exception as e: 635 | assert False, f'{e} was raised' 636 | else: 637 | assert False, "We should have raised an exception" 638 | 639 | set_narration_options(check=False) 640 | 641 | 642 | def test26(): 643 | """ 644 | test26: check that we get an ErratorException when we try to run a broken 645 | callable during an exception 646 | """ 647 | set_narration_options(check=False) 648 | reset_all_narrations() 649 | 650 | try: 651 | with narrate_cm(lambda arg: "oops"): 652 | raise KeyError("not this one") 653 | except ErratorException: 654 | assert any(">>>>" in l for l in get_narration()) 655 | except KeyError: 656 | assert False, "KeyError should not have come through" 657 | else: 658 | assert False, "We should have gotten an exception" 659 | 660 | 661 | def test27(): 662 | """ 663 | test27: check that we get an ErratorException from the decorator for a broken callable with check 664 | """ 665 | set_narration_options(check=True) 666 | reset_all_narrations() 667 | 668 | @narrate(lambda x, y: "again") 669 | def f(): 670 | pass 671 | 672 | try: 673 | f() 674 | except ErratorException: 675 | assert any(">>>>" in l for l in get_narration()) 676 | else: 677 | assert False, "we should have got an exception" 678 | set_narration_options(check=False) 679 | 680 | 681 | def test28(): 682 | """ 683 | test28: check that we get an ErratorException from the decorator for a broken callable without check 684 | """ 685 | set_narration_options(check=False) 686 | reset_all_narrations() 687 | 688 | @narrate(lambda missing: "oops") 689 | def f(): 690 | raise KeyError("heads up") 691 | 692 | try: 693 | f() 694 | except ErratorException: 695 | assert any(">>>>" in l for l in get_narration()) 696 | except KeyError: 697 | assert False, "We shouldn't have gotten a KeyError" 698 | else: 699 | assert False, "we should have gotten an exception" 700 | 701 | 702 | def test29(): 703 | """ 704 | test29: check basic verbose=True narration fetching 705 | """ 706 | set_narration_options(check=False, verbose=True) 707 | reset_all_narrations() 708 | 709 | @narrate("calling f") 710 | def f(): 711 | raise Exception("oops") 712 | 713 | try: 714 | f() 715 | assert False, "there should have been an exception" 716 | except Exception as _: 717 | stuff = get_narration() 718 | assert stuff, "there should have been a narration" 719 | 720 | 721 | # this next batch of functions is in support of testing 722 | # verbose narration 723 | 724 | @narrate(lambda x: "nf1 called with %s" % x) 725 | def f1(arg): 726 | if arg == "nf1": 727 | raise Exception("in nf1") 728 | f2(arg) 729 | 730 | 731 | @narrate(lambda x: "nf2 called with %s" % x) 732 | def f2(arg): 733 | if arg == "nf2": 734 | raise Exception("in nf2") 735 | f3(arg) 736 | 737 | 738 | def f3(arg): 739 | if arg == "nf3": 740 | raise Exception("in nf3") 741 | f4(arg) 742 | 743 | 744 | @narrate(lambda x: "nf4 called with %s" % x) 745 | def f4(arg): 746 | if arg == "nf4": 747 | raise Exception("in nf4") 748 | with narrate_cm(lambda x: "cm1 in nf4 with %s" % x, arg): 749 | if arg == "nf4@cm1": 750 | raise Exception("in nf4@cm1") 751 | f5(arg) 752 | 753 | 754 | def f5(arg): 755 | if arg == "nf5": 756 | raise Exception("in nf5") 757 | with narrate_cm(lambda x: "cm2 in nf5 with %s" % x, arg) as cm2: 758 | if arg == "nf5@cm2": 759 | raise Exception("in nf5@cm2") 760 | f6(arg) 761 | 762 | 763 | @narrate(lambda x: "nf6 called with %s" % x) 764 | def f6(arg): 765 | if arg == "nf6": 766 | raise Exception("in nf6") 767 | 768 | 769 | def count_nones(lines): 770 | return sum(1 for x in lines if "None" in x) 771 | 772 | 773 | def test30(): 774 | """ 775 | test30: checking that basic processing doesn't crater on it's own 776 | """ 777 | set_narration_options(check=False, verbose=True) 778 | reset_all_narrations() 779 | try: 780 | f1("nf6") 781 | assert False, "this should have raised" 782 | except Exception as e: 783 | assert "nf6" in str(e), "got an unexpected exception: %s" % str(e) 784 | lines = get_narration() 785 | assert len(lines) == 6, "unexpected number of strings: %s" % str(lines) 786 | 787 | 788 | def test31(): 789 | """ 790 | test31: checking verbose output from raise in nf1 791 | """ 792 | set_narration_options(check=False, verbose=True) 793 | reset_all_narrations() 794 | try: 795 | f1("nf1") 796 | assert False, "this should have raised" 797 | except Exception: 798 | lines = get_narration() 799 | assert len(lines) == 1, "got the following lines: %s" % str(lines) 800 | assert "nf1 called with nf1" in lines[0], "returned line contains: %s" % lines[0] 801 | assert "\n" in lines[0], "no newline in: %s" % lines[0] 802 | assert not count_nones(lines) 803 | 804 | 805 | def test32(): 806 | """ 807 | test32: checking verbose output from raise in nf2 808 | """ 809 | set_narration_options(check=False, verbose=True) 810 | reset_all_narrations() 811 | try: 812 | f1("nf2") 813 | assert False, "this should have raised" 814 | except Exception as e: 815 | assert "in nf2" in str(e) 816 | lines = get_narration() 817 | assert len(lines) == 2, "got the following lines: %s" % str(lines) 818 | assert "nf2 called with nf2" in lines[-1], "last line contains: %s" % lines[-1] 819 | assert "\n" in lines[0] and "\n" in lines[1], "newline missing: %s" % str(lines) 820 | assert not count_nones(lines) 821 | 822 | 823 | def test33(): 824 | """ 825 | test33: checking verbose output from raise in nf3 826 | """ 827 | set_narration_options(check=False, verbose=True) 828 | reset_all_narrations() 829 | try: 830 | f1("nf3") 831 | assert False, "this should have raised" 832 | except Exception as e: 833 | assert "in nf3" in str(e) 834 | lines = get_narration() 835 | assert len(lines) == 2, "got the following lines: %s" % str(lines) 836 | assert "nf2 called with nf3" in lines[-1], "last line contains: %s" % lines[-1] 837 | assert "\n" in lines[0] and "\n" in lines[1], "newline missing: %s" % str(lines) 838 | assert not count_nones(lines) 839 | 840 | 841 | def test34(): 842 | """ 843 | test34: checking verbose output from raise in nf4 844 | """ 845 | set_narration_options(check=False, verbose=True) 846 | reset_all_narrations() 847 | try: 848 | f1("nf4") 849 | assert False, "this should have raised" 850 | except Exception as e: 851 | assert "in nf4" in str(e), "wrong exception message: %s" % str(e) 852 | lines = get_narration() 853 | assert len(lines) == 3, "got the following lines: %s" % str(lines) 854 | assert "nf4 called with nf4" in lines[-1], "last line contains: %s" % lines[-1] 855 | assert 3 == sum(1 for x in lines if "\n" in x), "newline missing: %s" % str(lines) 856 | assert not count_nones(lines) 857 | 858 | 859 | def test34cm1(): 860 | """ 861 | test34cm1: checking verbose output from a context manager in nf4 862 | """ 863 | set_narration_options(check=False, verbose=True) 864 | reset_all_narrations() 865 | try: 866 | f1("nf4@cm1") 867 | assert False, "this should have raised" 868 | except Exception as e: 869 | assert "nf4@cm1" in str(e), "Unexpected text in exception: %s" % str(e) 870 | lines = get_narration() 871 | assert len(lines) == 4, "got the following lines: %s" % str(lines) 872 | assert "cm1 in nf4 with nf4@cm1" in lines[-1], "last line contains: %s" % lines[-1] 873 | assert 4 == sum(1 for x in lines if "\n" in x), "wrong number of newlines: %s" % str(lines) 874 | assert not count_nones(lines) 875 | 876 | 877 | def test35(): 878 | """ 879 | test35: checking verbose output from nf5 880 | """ 881 | set_narration_options(check=False, verbose=True) 882 | reset_all_narrations() 883 | try: 884 | f1("nf5") 885 | assert False, "this should have raised" 886 | except Exception as e: 887 | assert "in nf5" in str(e), "wrong exception value: %s" % str(e) 888 | lines = get_narration() 889 | assert len(lines) == 4, "got the following lines: %s" % str(lines) 890 | assert "cm1 in nf4 with nf5" in lines[-1], "last line contains: %s" % lines[-1] 891 | assert 4 == sum(1 for x in lines if "\n" in x), "wrong number of newlines: %s" % str(lines) 892 | assert not count_nones(lines) 893 | 894 | 895 | def test35cm2(): 896 | """ 897 | test35cm2: checking verbose output from nf5 at cm2 898 | """ 899 | set_narration_options(check=False, verbose=True) 900 | reset_all_narrations() 901 | try: 902 | f1("nf5@cm2") 903 | assert False, "this should have raised" 904 | except Exception as e: 905 | assert "in nf5@cm2" in str(e), "wrong exception value: %s" % str(e) 906 | lines = get_narration() 907 | assert len(lines) == 5, "got the following lines: %s" % str(lines) 908 | assert "cm2 in nf5 with nf5@cm2" in lines[-1], "last line contains: %s" % lines[-1] 909 | assert 5 == sum(1 for x in lines if "\n" in x), "wrong number of newlines: %s" % str(lines) 910 | assert not count_nones(lines) 911 | 912 | 913 | def test36(): 914 | """ 915 | test36: checking verbose output from nf6 916 | """ 917 | set_narration_options(check=False, verbose=True) 918 | reset_all_narrations() 919 | try: 920 | f1("nf6") 921 | assert False, "this should have raised" 922 | except Exception as e: 923 | assert "in nf6" in str(e), "wrong value in exception: %s" % str(e) 924 | lines = get_narration() 925 | assert len(lines) == 6, "wrong number of lines: %s" % len(lines) 926 | assert "nf6 called with nf6" in lines[-1], "last line is: %s" % lines[-1] 927 | assert 6 == sum(1 for x in lines if "\n" in x), "wrong number of newlines: %s" % str(lines) 928 | assert not count_nones(lines) 929 | 930 | 931 | def test37(): 932 | """ 933 | test37: check that calling get_narration() more than once returns the same exception data 934 | each time 935 | """ 936 | set_narration_options(check=False) 937 | reset_all_narrations() 938 | 939 | @narrate("Calling f") 940 | def f(i): 941 | raise Exception("test37") 942 | 943 | try: 944 | f(1) 945 | assert False, "should have raised" 946 | except Exception: 947 | l1 = get_narration() 948 | l2 = get_narration() 949 | assert len(l1) == 1, "got $%s lines" % len(l1) 950 | assert l1[0] == l2[0], "get_narration() returns different data in sequential calls" 951 | assert "test37" in l1[0], "narration doesn't appear to contain detail on exception" 952 | 953 | 954 | to_find = "narrate_it" 955 | 956 | 957 | def test38(): 958 | """ 959 | test38; check that extract_tb works properly 960 | """ 961 | set_narration_options(check=False) 962 | reset_all_narrations() 963 | 964 | try: 965 | f1("nf6") 966 | assert False, "should have raised" 967 | except: 968 | et, ev, tb = sys.exc_info() 969 | tb_lines = extract_tb(tb) 970 | assert not any(True for x in tb_lines if to_find in x[2]), "found a narrate_it" 971 | 972 | 973 | def test39(): 974 | """ 975 | test39: check that extract_stack works properly 976 | """ 977 | set_narration_options(check=False) 978 | reset_all_narrations() 979 | 980 | @narrate("in f") 981 | def f(): 982 | return extract_stack() 983 | 984 | stack_lines = f() 985 | assert not any(True for x in stack_lines if to_find in x[2]), "found a narrate_it" 986 | 987 | 988 | def test40(): 989 | """ 990 | test40: check that format_tb works 991 | """ 992 | set_narration_options(check=False) 993 | reset_all_narrations() 994 | 995 | try: 996 | f1("nf6") 997 | assert False, "should have raised" 998 | except: 999 | _, _, tb = sys.exc_info() 1000 | tb_lines = format_tb(tb) 1001 | assert not any(True for x in tb_lines if to_find in x), "found a narrate_it" 1002 | 1003 | 1004 | def test41(): 1005 | """ 1006 | test41: check that format_stack works 1007 | """ 1008 | set_narration_options(check=False) 1009 | reset_all_narrations() 1010 | 1011 | @narrate("in f") 1012 | def f(): 1013 | return format_stack() 1014 | 1015 | stack_lines = f() 1016 | assert not any(True for x in stack_lines if to_find in x), "found a %s" % to_find 1017 | 1018 | 1019 | def test42(): 1020 | """ 1021 | test42: check that format_exception works 1022 | """ 1023 | set_narration_options(check=False) 1024 | reset_all_narrations() 1025 | 1026 | try: 1027 | f1("nf6") 1028 | except: 1029 | lines = format_exception(*sys.exc_info()) 1030 | assert not any(True for x in lines if to_find in x), "found a %s" % to_find 1031 | 1032 | 1033 | def test43(): 1034 | """ 1035 | test43: check that print_tb works 1036 | """ 1037 | set_narration_options(check=False) 1038 | reset_all_narrations() 1039 | 1040 | try: 1041 | f1("nf6") 1042 | except: 1043 | _, _, tb = sys.exc_info() 1044 | f = StringIO() 1045 | print_tb(tb, file=f) 1046 | result = f.getvalue() 1047 | assert to_find not in result 1048 | 1049 | 1050 | def test44(): 1051 | """ 1052 | test44: check that print_exception works 1053 | """ 1054 | set_narration_options(check=False) 1055 | reset_all_narrations() 1056 | 1057 | try: 1058 | f1("nf6") 1059 | except: 1060 | et, ev, tb = sys.exc_info() 1061 | f = StringIO() 1062 | print_exception(et, ev, tb, file=f) 1063 | result = f.getvalue() 1064 | assert to_find not in result 1065 | 1066 | 1067 | def test45(): 1068 | """ 1069 | test45: check that print_exc works 1070 | """ 1071 | set_narration_options(check=False) 1072 | reset_all_narrations() 1073 | 1074 | try: 1075 | f1("nf6") 1076 | except: 1077 | f = StringIO() 1078 | print_exc(file=f) 1079 | result = f.getvalue() 1080 | assert to_find not in result 1081 | 1082 | 1083 | def test46(): 1084 | """ 1085 | test46: check that format_exc works 1086 | """ 1087 | set_narration_options(check=False) 1088 | reset_all_narrations() 1089 | 1090 | try: 1091 | f1("nf6") 1092 | except: 1093 | stuff = format_exc() 1094 | assert to_find not in stuff 1095 | 1096 | 1097 | def test47(): 1098 | """ 1099 | test47: check that print_last works 1100 | 1101 | NOTE: this may not have meaningful results due to limitations in traceback.print_last 1102 | """ 1103 | set_narration_options(check=False) 1104 | reset_all_narrations() 1105 | 1106 | try: 1107 | f1("nf6") 1108 | except: 1109 | pass 1110 | f = StringIO() 1111 | print_last(file=f) 1112 | result = f.getvalue() 1113 | assert to_find not in result 1114 | 1115 | 1116 | def test48(): 1117 | """ 1118 | test48: check that print_stack works 1119 | """ 1120 | set_narration_options(check=False) 1121 | reset_all_narrations() 1122 | 1123 | @narrate("test48") 1124 | def f(): 1125 | f = StringIO() 1126 | print_stack(file=f) 1127 | return f.getvalue() 1128 | 1129 | result = f() 1130 | assert to_find not in result 1131 | 1132 | 1133 | def test49(): 1134 | """ 1135 | test49: check that get_narration with different tags don't get the fragments 1136 | """ 1137 | set_narration_options(check=False) 1138 | reset_all_narrations() 1139 | 1140 | @narrate("test49", tags=["wibble"]) 1141 | def f(): 1142 | raise KeyError('ugh') 1143 | 1144 | try: 1145 | f() 1146 | except KeyError: 1147 | assert len(get_narration(with_tags=["wobble"])) == 0 1148 | except Exception as e: 1149 | assert False, f'got an {e}' 1150 | 1151 | 1152 | def test50(): 1153 | """ 1154 | test50: check that get_narration with same tag gets the fragment 1155 | """ 1156 | set_narration_options(check=False) 1157 | reset_all_narrations() 1158 | 1159 | @narrate("test50", tags=["wibble"]) 1160 | def f(): 1161 | raise KeyError('ugh') 1162 | 1163 | try: 1164 | f() 1165 | except KeyError: 1166 | assert any(['test50' in l for l in get_narration(with_tags=['wibble'])]), \ 1167 | f'got: {get_narration(with_tags=["wibble"])}' 1168 | except Exception as e: 1169 | assert False, f'got an {e}' 1170 | 1171 | 1172 | def test51(): 1173 | """ 1174 | test51: check that using get_narration() with no tags gets all fragments 1175 | """ 1176 | set_narration_options(check=False) 1177 | reset_all_narrations() 1178 | 1179 | @narrate("test51", tags=["wibble"]) 1180 | def f(): 1181 | raise KeyError('ugh') 1182 | 1183 | try: 1184 | f() 1185 | except KeyError: 1186 | assert any(['test51' in l for l in get_narration()]), \ 1187 | f'got: {get_narration()}' 1188 | except Exception as e: 1189 | assert False, f'got an {e}' 1190 | 1191 | 1192 | def test52(): 1193 | """ 1194 | test52: check that a fragment with no tag is picked up when looking for a tag 1195 | """ 1196 | set_narration_options(check=False) 1197 | reset_all_narrations() 1198 | 1199 | @narrate("test52") 1200 | def f(): 1201 | raise KeyError('ugh') 1202 | 1203 | try: 1204 | f() 1205 | except KeyError: 1206 | assert any(['test52' in l for l in get_narration(with_tags=["wibble"])]), \ 1207 | f'got: {get_narration(with_tags=["wibble"])}' 1208 | except Exception as e: 1209 | assert False, f'got an {e}' 1210 | 1211 | 1212 | def test53(): 1213 | """ 1214 | test53: Check that we do pick up context manager fragments with the right tag 1215 | """ 1216 | set_narration_options(check=False) 1217 | reset_all_narrations() 1218 | 1219 | def f(): 1220 | with narrate_cm("test53cm", tags=["wibble"]): 1221 | raise KeyError('oops') 1222 | 1223 | try: 1224 | f() 1225 | except KeyError: 1226 | assert len(get_narration(with_tags=['wibble'])) == 1 1227 | except Exception as e: 1228 | assert False, f'got an {e}' 1229 | 1230 | 1231 | def test54(): 1232 | """ 1233 | test54: check that we don't pick up cm fragments with the wrong tag 1234 | """ 1235 | set_narration_options(check=False) 1236 | reset_all_narrations() 1237 | 1238 | def f(): 1239 | with narrate_cm("test54cm", tags=["wibble"]): 1240 | raise KeyError('oops') 1241 | 1242 | try: 1243 | f() 1244 | except KeyError: 1245 | assert len(get_narration(with_tags=['wobble'])) == 0 1246 | except Exception as e: 1247 | assert False, f'got an {e}' 1248 | 1249 | 1250 | def test55(): 1251 | """ 1252 | test55: check we pick up cm fragements that don't have a tag 1253 | """ 1254 | set_narration_options(check=False) 1255 | reset_all_narrations() 1256 | 1257 | def f(): 1258 | with narrate_cm("test55cm"): 1259 | raise KeyError('oops') 1260 | 1261 | try: 1262 | f() 1263 | except KeyError: 1264 | assert len(get_narration(with_tags=['wobble'])) == 1 1265 | except Exception as e: 1266 | assert False, f'got an {e}' 1267 | 1268 | 1269 | def test56(): 1270 | """ 1271 | test56: check we pick up a tagged cm when we don't specify any 1272 | """ 1273 | set_narration_options(check=False) 1274 | reset_all_narrations() 1275 | 1276 | def f(): 1277 | with narrate_cm("test56cm", tags=['ibble']): 1278 | raise KeyError('oops') 1279 | 1280 | try: 1281 | f() 1282 | except KeyError: 1283 | assert len(get_narration()) == 1 1284 | except Exception as e: 1285 | assert False, f'got an {e}' 1286 | 1287 | 1288 | def test57(): 1289 | """ 1290 | test57: test a stack of calls, only pick up 1/2 of the fragments with tags 1291 | """ 1292 | set_narration_options(check=False) 1293 | reset_all_narrations() 1294 | 1295 | @narrate('f1', tags=["hit"]) 1296 | def f1(): 1297 | f2() 1298 | 1299 | @narrate('f2', tags=['miss', 'common']) 1300 | def f2(): 1301 | f3() 1302 | 1303 | def f3(): 1304 | with narrate_cm('f3cm', tags=['hit', 'common']): 1305 | f4() 1306 | 1307 | @narrate('f4', tags=['miss']) 1308 | def f4(): 1309 | raise KeyError('you sunk my battleship') 1310 | 1311 | try: 1312 | f1() 1313 | except KeyError: 1314 | pass 1315 | except Exception as e: 1316 | assert False, f'got an {e}' 1317 | else: 1318 | assert len(get_narration(with_tags=["hit"])) == 2 1319 | assert len(get_narration(with_tags=["miss"])) == 2 1320 | assert len(get_narration(with_tags=["common"])) == 2 1321 | 1322 | 1323 | def do_all(): 1324 | for k, v in sorted(globals().items()): 1325 | if callable(v) and k.startswith("test"): 1326 | print("Running test {}".format(k)) 1327 | try: 1328 | v() 1329 | except Exception as e: 1330 | print("Test {} failed with:\n".format(k)) 1331 | traceback.print_exception(*sys.exc_info()) 1332 | 1333 | 1334 | if __name__ == "__main__": 1335 | do_all() 1336 | -------------------------------------------------------------------------------- /timing.py: -------------------------------------------------------------------------------- 1 | from errator import narrate, get_narration, set_narration_options 2 | import timeit 3 | import platform 4 | 5 | 6 | def f1(borkfunc, catchfunc): 7 | if borkfunc == 1: 8 | raise Exception("bork1") 9 | else: 10 | try: 11 | f2(borkfunc, catchfunc) 12 | except Exception: 13 | if catchfunc == 1: 14 | _ = get_narration() 15 | else: 16 | raise 17 | 18 | 19 | @narrate("in 1") 20 | def nf1(borkfunc, catchfunc): 21 | if borkfunc == 1: 22 | raise Exception("bork1") 23 | else: 24 | try: 25 | nf2(borkfunc, catchfunc) 26 | except Exception: 27 | if catchfunc == 1: 28 | _ = get_narration() 29 | else: 30 | raise 31 | 32 | 33 | def f2(borkfunc, catchfunc): 34 | if borkfunc == 2: 35 | raise Exception("bork2") 36 | else: 37 | try: 38 | f3(borkfunc, catchfunc) 39 | except Exception: 40 | if catchfunc == 2: 41 | _ = get_narration() 42 | else: 43 | raise 44 | 45 | 46 | @narrate("in 2") 47 | def nf2(borkfunc, catchfunc): 48 | if borkfunc == 2: 49 | raise Exception("bork2") 50 | else: 51 | try: 52 | nf3(borkfunc, catchfunc) 53 | except Exception: 54 | if catchfunc == 2: 55 | _ = get_narration() 56 | else: 57 | raise 58 | 59 | 60 | def f3(borkfunc, catchfunc): 61 | if borkfunc == 3: 62 | raise Exception("bork3") 63 | else: 64 | try: 65 | f4(borkfunc, catchfunc) 66 | except Exception: 67 | if catchfunc == 3: 68 | _ = get_narration() 69 | else: 70 | raise 71 | 72 | 73 | @narrate("in 3") 74 | def nf3(borkfunc, catchfunc): 75 | if borkfunc == 3: 76 | raise Exception("bork3") 77 | else: 78 | try: 79 | nf4(borkfunc, catchfunc) 80 | except Exception: 81 | if catchfunc == 3: 82 | _ = get_narration() 83 | else: 84 | raise 85 | 86 | 87 | def f4(borkfunc, catchfunc): 88 | if borkfunc == 4: 89 | raise Exception("bork4") 90 | else: 91 | try: 92 | f5(borkfunc, catchfunc) 93 | except Exception: 94 | if catchfunc == 4: 95 | _ = get_narration() 96 | else: 97 | raise 98 | 99 | 100 | @narrate("in 4") 101 | def nf4(borkfunc, catchfunc): 102 | if borkfunc == 4: 103 | raise Exception("bork4") 104 | else: 105 | try: 106 | nf5(borkfunc, catchfunc) 107 | except Exception: 108 | if catchfunc == 4: 109 | _ = get_narration() 110 | else: 111 | raise 112 | 113 | 114 | def f5(borkfunc, catchfunc): 115 | if borkfunc == 5: 116 | raise Exception("bork5") 117 | else: 118 | try: 119 | f6(borkfunc, catchfunc) 120 | except Exception: 121 | if catchfunc == 5: 122 | _ = get_narration() 123 | else: 124 | raise 125 | 126 | 127 | @narrate("in 5") 128 | def nf5(borkfunc, catchfunc): 129 | if borkfunc == 5: 130 | raise Exception("bork5") 131 | else: 132 | try: 133 | nf6(borkfunc, catchfunc) 134 | except Exception: 135 | if catchfunc == 5: 136 | _ = get_narration() 137 | else: 138 | raise 139 | 140 | 141 | def f6(borkfunc, catchfunc): 142 | if borkfunc == 6: 143 | raise Exception("bork6") 144 | else: 145 | try: 146 | f7(borkfunc, catchfunc) 147 | except Exception: 148 | if catchfunc == 6: 149 | _ = get_narration() 150 | else: 151 | raise 152 | 153 | 154 | @narrate("in 6") 155 | def nf6(borkfunc, catchfunc): 156 | if borkfunc == 6: 157 | raise Exception("bork6") 158 | else: 159 | try: 160 | nf7(borkfunc, catchfunc) 161 | except Exception: 162 | if catchfunc == 6: 163 | _ = get_narration() 164 | else: 165 | raise 166 | 167 | 168 | def f7(borkfunc, catchfunc): 169 | if borkfunc == 7: 170 | raise Exception("bork7") 171 | else: 172 | try: 173 | f8(borkfunc, catchfunc) 174 | except Exception: 175 | if catchfunc == 7: 176 | _ = get_narration() 177 | else: 178 | raise 179 | 180 | 181 | @narrate("in 7") 182 | def nf7(borkfunc, catchfunc): 183 | if borkfunc == 7: 184 | raise Exception("bork7") 185 | else: 186 | try: 187 | nf8(borkfunc, catchfunc) 188 | except Exception: 189 | if catchfunc == 7: 190 | _ = get_narration() 191 | else: 192 | raise 193 | 194 | 195 | def f8(borkfunc, catchfunc): 196 | if borkfunc == 8: 197 | raise Exception("bork8") 198 | else: 199 | try: 200 | f9(borkfunc, catchfunc) 201 | except Exception: 202 | if catchfunc == 8: 203 | _ = get_narration() 204 | else: 205 | raise 206 | 207 | 208 | @narrate("in 8") 209 | def nf8(borkfunc, catchfunc): 210 | if borkfunc == 8: 211 | raise Exception("bork8") 212 | else: 213 | try: 214 | nf9(borkfunc, catchfunc) 215 | except Exception: 216 | if catchfunc == 8: 217 | _ = get_narration() 218 | else: 219 | raise 220 | 221 | 222 | def f9(borkfunc, catchfunc): 223 | if borkfunc == 9: 224 | raise Exception("bork9") 225 | else: 226 | try: 227 | f10(borkfunc, catchfunc) 228 | except Exception: 229 | if catchfunc == 9: 230 | _ = get_narration() 231 | else: 232 | raise 233 | 234 | 235 | @narrate("in 9") 236 | def nf9(borkfunc, catchfunc): 237 | if borkfunc == 9: 238 | raise Exception("bork9") 239 | else: 240 | try: 241 | nf10(borkfunc, catchfunc) 242 | except Exception: 243 | if catchfunc == 9: 244 | _ = get_narration() 245 | else: 246 | raise 247 | 248 | 249 | def f10(borkfunc, catchfunc): 250 | if borkfunc == 10: 251 | raise Exception("bottom") 252 | return 253 | 254 | 255 | @narrate("in 10") 256 | def nf10(borkfunc, catchfunc): 257 | if borkfunc == 10: 258 | raise Exception("bottom") 259 | return 260 | 261 | 262 | def plain(bf, cf): 263 | return bf + cf 264 | 265 | 266 | @narrate("simple") 267 | def simple(bf, cf): 268 | return bf + cf 269 | 270 | 271 | def do_it(errated=True): 272 | if errated: 273 | startfunc = nf1 274 | else: 275 | startfunc = f1 276 | 277 | for bf in range(1, 11): 278 | for cf in range(1, bf): 279 | try: 280 | startfunc(bf, cf) 281 | except Exception: 282 | if errated: 283 | _ = get_narration() 284 | 285 | 286 | def nested_call_timing(errated=True): 287 | if errated: 288 | startfunc = nf1 289 | else: 290 | startfunc = f1 291 | 292 | startfunc(100, 100) # never raise, never catch 293 | if errated: 294 | _ = get_narration() 295 | 296 | 297 | if __name__ == "__main__": 298 | print("Python version: {}".format(platform.python_version())) 299 | set_narration_options(verbose=False) 300 | loops = 1000 301 | timeit.do_it = do_it 302 | timeit.simple = simple 303 | timeit.plain = plain 304 | timeit.nested_call_timing = nested_call_timing 305 | # prime things so there's no first run penalty 306 | do_it(errated=True) 307 | 308 | # now do calls that show handling of narrations when there is a exception 309 | print("==Narrated nested call stack with exceptions==") 310 | narrated_elapsed = timeit.timeit(stmt="do_it(errated=True)", number=loops) 311 | print("{} loops took {}".format(loops, narrated_elapsed)) 312 | print("==Plain nested call stack with exceptions==") 313 | plain_elapsed = timeit.timeit(stmt="do_it(errated=False)", number=loops) 314 | print("{} loops took {}".format(loops, plain_elapsed)) 315 | print("Plain is {} times faster".format(narrated_elapsed / plain_elapsed)) 316 | 317 | # next, do the same nested calls, but never raise an exception 318 | loops = 100000 319 | narrated_elapsed = timeit.timeit(stmt="nested_call_timing(errated=True)", number=loops) 320 | print("\n==Nested call times with narrations but no exceptions==") 321 | print("{} loops took {}".format(loops, narrated_elapsed)) 322 | plain_elapsed = timeit.timeit(stmt="nested_call_timing(errated=False)", number=loops) 323 | print("==Nested call times with no narrations, no exceptions==") 324 | print("{} loops tool {}".format(loops, plain_elapsed)) 325 | print("Plain is {} times faster".format(narrated_elapsed / plain_elapsed)) 326 | 327 | # now do plain functions 328 | simple(1, 1) # prime any first-time overheads out 329 | loops = 1000000 330 | narrated_elapsed = timeit.timeit(stmt="simple(1, 1)", number=loops) 331 | print("\n==Single string narrated call, no exceptions, {} calls: {}".format(loops, narrated_elapsed)) 332 | plain_elapsed = timeit.timeit(stmt="plain(1, 1)", number=loops) 333 | print("==No errator decoration, no exceptions, {} calls: {}".format(loops, plain_elapsed)) 334 | 335 | print("Plain is {} times faster".format(narrated_elapsed / plain_elapsed)) 336 | -------------------------------------------------------------------------------- /win_build_all.bat: -------------------------------------------------------------------------------- 1 | rem make the output wheels directory and clean of previous builds 2 | mkdir wheels 3 | del wheels\*.whl 4 | rem this is the base directory where all the python installs can be found 5 | set PREFIX=c:\py 6 | rem this is the base directory where the build/test virtualenvs will be made 7 | set VPY_ROOT=c:\vpy 8 | rem the build virtualenv 9 | set BUILD=%VPY_ROOT%\err_build 10 | rem the test virtualenv 11 | set TEST=%VPY_ROOT%\err_test 12 | rem all of the available Python releases 13 | set PYPATH=%PREFIX%\py36 %PREFIX%\py37 %PREFIX%\py38 %PREFIX%\py39 14 | 15 | echo %PREFIX%, %VPY_ROOT%, %BUILD%, %TEST% 16 | for %%P in (%PYPATH%) do ( 17 | echo %%P 18 | rem remove previous files 19 | rd /s /q %BUILD% 20 | rd /s /q %TEST% 21 | rem upgrade pip in the version install & install the build tools 22 | %%P\Scripts\python.exe -m pip install --upgrade pip 23 | %%P\Scripts\pip install virtualenv 24 | rem build out and activate the virtualenv for the py version 25 | %%P\Scripts\virtualenv -p %%P\python.exe %BUILD% 26 | %BUILD%\Scripts\activate.bat 27 | rem install the build tools in the build virtualenv & build 28 | %BUILD%\Scripts\pip install wheel 29 | %BUILD%\Scripts\pip install -r requirements.txt 30 | %BUILD%\Scripts\pip wheel . --no-deps -w wheels 31 | deactivate 32 | rem now make the test environment 33 | %%P\Scripts\virtualenv -p %%P\python.exe %TEST% 34 | rem activate and install the new package into test 35 | %TEST%\Scripts\activate.bat 36 | %TEST%\Scripts\pip install errator -f wheels 37 | %TEST%\Scripts\pip install pytest 38 | rem test the install 39 | %TEST%\Scripts\pytest tests.py 40 | deactivate 41 | ) 42 | rem final cleanup 43 | rd /s /q %BUILD% 44 | rd /s /q %TEST% 45 | 46 | 47 | --------------------------------------------------------------------------------