├── .gitignore ├── CHANGES.mkd ├── LICENSE ├── README.mkd ├── dev-requirements.txt ├── setup.cfg ├── setup.py ├── spec ├── __init__.py ├── _version.py ├── cli.py ├── plugin.py ├── trap.py └── utils.py ├── tasks.py ├── test.py ├── tests ├── _spec_test_cases │ ├── containers.py │ ├── docstring_spec_names.py │ ├── doctests.py │ ├── foobar.py │ ├── foobaz.py │ ├── generators.py │ └── generators_with_descriptions.py └── spec_test.py ├── tox.ini └── web ├── after.png ├── before.png └── readme.py /.gitignore: -------------------------------------------------------------------------------- 1 | *.pyc 2 | *.egg-info 3 | .tox 4 | build 5 | dist 6 | -------------------------------------------------------------------------------- /CHANGES.mkd: -------------------------------------------------------------------------------- 1 | ## 1.4.1 2 | ### (2017.09.02) 3 | 4 | * Fix detection of failing `setUp()` methods so reporting is correct, instead 5 | of crashing. Thanks to ``@bendikro``! 6 | * Fix issue with calling ``strip()`` on test descriptions when the test is 7 | generator-driven. Credit: ``@AFriemann``. 8 | 9 | ## 1.4.0 10 | ### (2017.05.02) 11 | 12 | * Python 2 oriented Unicode fixes. 13 | 14 | ## 1.3.1 15 | ### (2015.09.19) 16 | 17 | * A couple of minor packaging-related updates, including marking wheel archives 18 | as `universal`. 19 | 20 | ## 1.3.0 21 | ### (2015.09.19) 22 | 23 | * Added `assert_contains` and `assert_not_contains` test assertions. 24 | 25 | ## 1.2.2 26 | ### (2015.04.23) 27 | 28 | * Fixed a bug re: inner nested classes accessing attributes defined in outer 29 | classes/scopes (symptom: an `AttributeError` about `setup`). 30 | 31 | ## 1.2.1 32 | ### (2015.04.21) 33 | 34 | * Skip (instead of blowing up on) modules lacking a `__file__` attribute (such 35 | as Python 3's builtin `_io` module) during test discovery. 36 | * Handle a map expression more appropriately under Python 3 so `--tests=xxx` 37 | actually works instead of silently selecting no tests. 38 | 39 | ## 1.2.0 40 | ### (2015.04.14) 41 | 42 | * Add `--timing-threshold` option for configuring the threshold of 43 | `--with-timing`. 44 | 45 | ## 1.1.0 46 | ### (2015.04.11) 47 | 48 | * Calculate runtime of tests and add display of same (non-colorized, to stand 49 | out better from the colorized test names themselves). This uses the 'status' 50 | field of test display methods, which didn't seem to be used much previously. 51 | 52 | Use the new `--with-timing` flag to enable this feature. 53 | 54 | ## 1.0.0 55 | ### (2015.03.12) 56 | 57 | * Bump to 1.0 because it's been out for some time & I can't afford major 58 | backwards incompat changes because of that - so being pre-1.0 is silly. 59 | * Don't accidentally discard return values of functions wrapped with `@trap`. 60 | * Offer an improved version of `nose.tools.eq_` which prints both `str()` and 61 | `repr()` style output of the compared strings. 62 | 63 | ## 0.11.3 64 | ### (2015.01.02) 65 | 66 | * Use `__mro__` instead of `mro()` in hopes of appeasing PyPy 2.4. 67 | 68 | ## 0.11.2 69 | ### (2014.12.10) 70 | 71 | * Update license file (which had been initialized as BSD) to be consistent with 72 | the README and packaging metadata (which said MIT). Bluh. Thanks to Eduardo 73 | Téllez for the catch. 74 | 75 | ## 0.11.1 76 | ### (2013.04.14) 77 | 78 | * Fix a bug re: using `@trap` under Python 3 79 | 80 | ## 0.11.0 81 | ### (2013.04.14) 82 | 83 | * Update Nose dependency links in `setup.py` 84 | * Fix #30: allow access to outer/parent classes within nested classes (when 85 | using classes as a visual nesting mechanism). 86 | * Also document the class nesting mechanism (despite it being around for quite 87 | some time...) 88 | 89 | ## 0.10.0 90 | ### (2013.03.15) 91 | 92 | * Python 3 support! 93 | 94 | ## 0.9.7 95 | ### (2012.07.12) 96 | 97 | * Add `@trap` decorator to allow trapping stdout/stderr for certain tests. 98 | * Strip whitespace from docstrings used as test names, to avoid clunky looking 99 | output. 100 | 101 | 102 | ## 0.9.6 103 | ### (2012.06.07) 104 | 105 | * Add number of skipped tests to successful summary output. Previously, number 106 | of skipped tests was only printed when other failures occurred. 107 | * Add `@hide` decorator to explicitly mark functions as not being tests, thanks 108 | to Maciej Konieczny for the initial work. 109 | * Auto-hide `setup` and `teardown` methods/functions so they aren't run as / 110 | shown as tests themselves. Thanks again to Maciej for the initial pull 111 | request. 112 | * Refactor `spec`'s relaxed test discovery so it can be invoked as its own Nose 113 | plugin/option (though `spec` the program still has its own use and still 114 | exists.) Thanks to Marc Abramowitz (and thanks to him also for some minor 115 | internal refactorings & project support tweaks.) 116 | 117 | ## 0.9.5 118 | ### (2012.03.07) 119 | 120 | * Use `nose.core.main` instead of `nose.core.run` so `spec` exits with nonzero 121 | return values upon test failure. 122 | 123 | ## 0.9.4 124 | ### (2012.03.07) 125 | 126 | * Make sure `spec` does not blow up when no `tests/` directory exists. Fixes 127 | #14. Thanks to Janne Härkönen for the patch. 128 | * Filter out `.pyc` files when selecting tests. Fixes #15; thanks again to 129 | Janne. 130 | * Actually skip leading-underscore files; implementation did not match docs. 131 | * Add support for nested inner classes as recursive contexts. 132 | * Add command-line option to disable our automatic toggle of the built-in 133 | `--detailed-errors` flag. 134 | * Tweak color scheme so purples became reds. 135 | * Remove old Nose-style SKIPPED/FAILED/etc suffixes when showing spec 136 | format. 137 | * Re-integrate support for individual test module targeting (`--tests=`). 138 | 139 | ## 0.9.3 140 | #### (2011.11.06) 141 | 142 | * Print skipped test count in summary. (#5) 143 | * Only print `=` error counts in summary for nonzero counts (i.e. 144 | never print e.g. `(failures=0, ...)`.) 145 | * Don't use custom test selection in `spec` tool if user passes in common Nose 146 | selection options such as `--where=` or `--tests=`. 147 | * Refactor of internal colorization functions. (#7) 148 | * Filter out trailing underscores from class names when printing spec output 149 | (so e.g. `class MyClass_` shows up as `MyClass` and not `MyClass_`.) (#8) 150 | 151 | ## 0.9.2 152 | #### (2011.11.02) 153 | 154 | * PEP8 overhaul (I never audited the original source for PEP8 compliancy :() 155 | courtesy of Douglas Soares de Andrade. (#3) 156 | * Docstrings-as-test-identifiers now `strip()`'d, also thanks to Douglas. 157 | * Implement Rudolf-inspired traceback and summary colorization. (#1) 158 | 159 | ## 0.9.1 160 | #### (2011.10.30) 161 | 162 | * Switch from single module to a package. 163 | * Add 'spec' CLI runner tool. 164 | 165 | ## 0.9.0 166 | #### (2011.10.30) 167 | 168 | * Import original Pinocchio/Spec codebase. 169 | * Trim down to just 'spec' plugin. 170 | * Nose 1.x fixes. 171 | * Python 2.7 fixes. 172 | * Packaging, docs, internal tests all overhauled. 173 | * Set colorized output as the default behavior. 174 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | The MIT License (MIT) 2 | 3 | Copyright (c) 2019 Jeffrey Forcier. 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in 13 | all copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN 21 | THE SOFTWARE. 22 | -------------------------------------------------------------------------------- /README.mkd: -------------------------------------------------------------------------------- 1 | # spec 2 | 3 | ## What is it? 4 | 5 | `spec` is a Python (2.6+ and 3.3+) testing tool that turns this: 6 | 7 | ![Boring old nosetests 8 | output](https://github.com/bitprophet/spec/raw/master/web/before.png) 9 | 10 | into this: 11 | 12 | ![Awesome new spec output](https://github.com/bitprophet/spec/raw/master/web/after.png) 13 | 14 | Specifically, `spec` provides: 15 | 16 | * Colorized, specification style output 17 | * Colorized tracebacks and summary 18 | * Optional timing display for slow tests 19 | * Test-running CLI tool which enables useful non-default options and implements 20 | relaxed test discovery for less `test_annoying.py:TestBoilerplate.test_code` 21 | and more `readable.py:Classes.and_methods`. 22 | 23 | ## Spec-style output 24 | 25 | `spec` is a [BDD](http://behavior-driven.org)-esque 26 | [nose](http://nose.readthedocs.io) plugin designed to provide "specification" 27 | style test output (similar to Java's 28 | [TestDox](http://agiledox.sourceforge.net/) or Ruby's 29 | [RSpec](https://www.relishapp.com/rspec)). Spec-style output provides a more 30 | structured view of what your tests assert, compared to `nose`/`unittest`'s 31 | default "flat" mode of operation. 32 | 33 | For example, this `nose`-style test module: 34 | 35 | ```python 36 | class TestShape(object): 37 | def test_has_sides(self): 38 | pass 39 | 40 | def test_can_calculate_its_perimeter(self): 41 | pass 42 | 43 | class TestSquare(object): 44 | def test_is_a_shape(self): 45 | pass 46 | 47 | def test_has_four_sides(self): 48 | pass 49 | 50 | def test_has_sides_of_equal_length(self): 51 | pass 52 | ``` 53 | 54 | normally tests like so, in a single flat list: 55 | 56 | TestShape.test_has_sides ... ok 57 | TestShape.test_can_calculate_its_perimeter ... ok 58 | TestSquare.test_has_four_sides ... ok 59 | TestSquare.test_has_sides_of_equal_length ... ok 60 | TestSquare.test_is_a_shape ... ok 61 | 62 | With `spec` enabled (`--with-spec`), the tests are visually grouped by class, 63 | and the member names are tweaked to read more like regular English: 64 | 65 | Shape 66 | - has sides 67 | - can calculate its perimeter 68 | 69 | Square 70 | - has four sides 71 | - has sides of equal length 72 | - is a shape 73 | 74 | In other words: 75 | 76 | * Class-based tests are arranged with the class name as the subject, and the 77 | methods as the specifications; 78 | * Any module-level tests are arranged with the module name as the subject; 79 | * All objects' docstrings are used as their descriptions, if found. Otherwise: 80 | * `CamelCaseNames` (typically classes) have any leading/trailing `Test` 81 | stripped, as well as any trailing underscore; 82 | * `CamelCaseNames` also get turned into sentences if necessary, so e.g. 83 | `CamelCaseNames` becomes `Camel case names`; 84 | * `underscored_names` have any leading/trailing `test` (with its attached 85 | underscore) stripped; 86 | * `underscored_names` have underscores turned into spaces; 87 | 88 | 89 | ## Test runner 90 | 91 | `spec` ships with a same-name command-line tool which may be used as a more 92 | liberal `nosetests`. In addition to toggling a number of useful default options 93 | (such as `nose`'s builtin `--detailed-errors`) `spec`-the-program will honor 94 | any and all public objects defined within your project's `tests` directory, 95 | meaning any file, function or class whose name does not begin with an 96 | underscore (`'_'`) and which is defined locally. 97 | 98 | For example, given the following code inside `tests/feature_name.py`: 99 | 100 | ```python 101 | from external_module import a_function, AClass 102 | 103 | def _helper_function(args): 104 | return a_function(args) 105 | 106 | class _Parent(object): 107 | def this_will_not_get_tested(): 108 | pass 109 | 110 | class Feature(_Parent): 111 | def should_have_some_attribute(self): 112 | _helper_function(AClass) 113 | 114 | def does_something_awesome(self): 115 | self._helper_method() 116 | 117 | def _helper_method(self): 118 | pass 119 | 120 | def something_tested_by_itself_outside_a_class(): 121 | pass 122 | ``` 123 | 124 | only the following items will be picked up as test cases: 125 | 126 | * `Feature.should_have_some_attribute` 127 | * `Feature.does_something_awesome` 128 | * `something_tested_by_itself_outside_a_class` 129 | 130 | The imported function and class, the underscored functions/methods, and the 131 | methods inherited from a parent class, are all ignored. 132 | 133 | ### Enhanced output via the Spec class 134 | 135 | As with some other spec-style tools, `spec` provides a means for nesting your 136 | test "contexts" so they display nicely during test runs. Just use the `Spec` 137 | class as your primary superclass and inner classes will get parsed 138 | automatically. 139 | 140 | For example: 141 | 142 | ```python 143 | from spec import Spec 144 | 145 | class ClassUnderTest(Spec): 146 | def it_behaves_like_this(self): 147 | # ... 148 | 149 | class init: 150 | "__init__" 151 | def takes_arg1(self): 152 | # ... 153 | 154 | def takes_arg2(self): 155 | # ... 156 | ``` 157 | 158 | The above results in output like so: 159 | 160 | Class under test 161 | - it behaves like this 162 | 163 | __init__ 164 | - takes arg1 165 | - takes arg2 166 | 167 | This indentation makes output even easier to follow & helps keep things 168 | organized. 169 | 170 | #### Accessing outer classes from inner ones 171 | 172 | Frequently, you may have a useful `setup` method in your outer class, and wish 173 | to access objects attached to `self` from inner classes. As of Spec 0.11.0 this 174 | is now possible and is quite transparent; failed attribute lookups will check 175 | an instantiated + setup'd copy of the outer class: 176 | 177 | ```python 178 | class MainClass(Spec): 179 | def setup(self): 180 | self.x = 'y' 181 | 182 | def outer_test(self): 183 | assert self.x == 'y' 184 | 185 | class some_inner_class: 186 | def inner_test(self): 187 | # Here, because some_inner_class has no real 'x' attribute, we end 188 | # up seeing the outer class' value. 189 | assert self.x == 'y' 190 | ``` 191 | 192 | Right now this support is pretty basic and assumes your setup methods are 193 | literally named `setup`, not `setUp` or whatnot. This will likely improve in 194 | the future. 195 | 196 | ### Usage tips 197 | 198 | Following from `spec`-the-tool's discovery algorithm, and `spec`-the-plugin's 199 | name transformation, we suggest the following for both readable code and 200 | readable test output: 201 | 202 | * Store tests in `tests/`, with whatever file-by-file organization you like 203 | best; 204 | * Within files, import the classes under test normally, e.g. `from mymodule 205 | import MyClass`; 206 | * Name the test classes identically, but with a trailing underscore to avoid 207 | name collisions, and inheriting from the `Spec` class, e.g. `class MyClass_(Spec): [...]` 208 | * Name their methods like English sentences, e.g. `def has_attribute_X(self): 209 | [...]`. 210 | * Tests not yet filled out should call the `skip` function, which raises a 211 | "skip this test" exception Nose handles correctly. 212 | 213 | For example: 214 | 215 | ```python 216 | from spec import Spec, skip 217 | from mypackage import MyClass, MyOtherClass 218 | 219 | class MyClass_(Spec): 220 | def has_attribute_A(self): 221 | skip() 222 | 223 | class MyOtherClass_(Spec): 224 | def also_has_attribute_A(self): 225 | skip() 226 | 227 | def has_attribute_B(Spec): 228 | skip() 229 | ``` 230 | 231 | tests as: 232 | 233 | ``` 234 | MyClass 235 | - has attribute A 236 | 237 | MyOtherClass 238 | - also has attribute A 239 | - has attribute B 240 | ``` 241 | 242 | 243 | ## Activation / command-line use 244 | 245 | After installation via `setup.py`, `pip` or what have you, `nosetests` will 246 | expose these new additional options/flags: 247 | 248 | * `--with-spec`: enables the plugin and prints out your tests in specification 249 | format. Also automatically sets `--verbose` (i.e. the spec output is a 250 | verbose format.) 251 | * `--no-spec-color`: disables color output. Normally, successes are green, 252 | failures/errors are red, and 253 | [skipped](http://nose.readthedocs.io/en/latest/plugins/skip.html) tests are 254 | yellow. 255 | * `--spec-doctests`: enables (experimental) support for doctests. 256 | * `--with-timing`: enables timing display for tests that take >= 257 | `TIMING_THRESHOLD` (see below) to run. 258 | * `--timing-threshold`: configuration of timing threshold (in seconds) for 259 | `--with-timing`. Defaults to 0.1. 260 | 261 | 262 | ## Why would I want to use it? 263 | 264 | Specification-style output can make large test suites easier to read, and like 265 | any other BDD tool, it's more about framing the way we think about (and view) 266 | our tests, and less about introducing new technical methods for writing them. 267 | 268 | 269 | ## Where did it come from? 270 | 271 | `spec` is heavily based on the `spec` plugin for Titus Brown's 272 | [pinocchio](http://darcs.idyll.org/~t/projects/pinocchio/doc/#spec-generate-test-description-from-test-class-method-names) 273 | set of Nose extensions. Said plugin was originally written by Michal 274 | Kwiatkowski. Both `pinocchio` and its `spec` plugin are copyright © 2007 275 | in the above two gentlemen's names, respectively. 276 | 277 | This version of the plugin was created and distributed by Jeff Forcier, © 278 | 2011. It tweaks the original source to be Python 2.7 compatible, based on 279 | [similar](https://github.com/unpluggd/pinocchio/commit/de30d5f7868280a2b9e3545c48e68dd0d9a343a0) 280 | [changes](https://github.com/bitprophet/rudolf/commit/7c872e7deeff622de62a439b8e4dd807047c095e). 281 | It also fixes a handful of bugs such as broken 282 | [SkipTest](http://nose.readthedocs.io/en/latest/plugins/skip.html) 283 | compatibility under Nose 1.x, and then adds some additional functionality on 284 | top (most notably the `spec` command-line tool.) 285 | 286 | 287 | ## What's the license? 288 | 289 | Because this is heavily derivative of `pinocchio`, `spec` is licensed the same 290 | way -- under the [MIT 291 | license](http://www.opensource.org/licenses/mit-license.php). 292 | -------------------------------------------------------------------------------- /dev-requirements.txt: -------------------------------------------------------------------------------- 1 | invoke<2.0 2 | invocations<2.0 3 | semantic_version<3.0 4 | wheel==0.24.0 5 | twine==1.9.1 6 | -------------------------------------------------------------------------------- /setup.cfg: -------------------------------------------------------------------------------- 1 | [wheel] 2 | universal = 1 3 | -------------------------------------------------------------------------------- /setup.py: -------------------------------------------------------------------------------- 1 | from setuptools import setup, find_packages 2 | 3 | # Version info -- read without importing 4 | _locals = {} 5 | with open('spec/_version.py') as fp: 6 | exec(fp.read(), None, _locals) 7 | version = _locals['__version__'] 8 | 9 | setup( 10 | name='spec', 11 | version=version, 12 | description='Specification-style output for nose', 13 | author='Jeff Forcier', 14 | author_email='jeff@bitprophet.org', 15 | url='https://github.com/bitprophet/spec', 16 | license='MIT', 17 | packages=find_packages(), 18 | install_requires=['nose>=1.3,<2.0', 'six<2.0'], 19 | dependency_links=[ 20 | 'https://github.com/nose-devs/nose/tarball/c0f777e488337dc7dde933453799986c46b37deb#egg=nose-1.3.0', 21 | ], 22 | entry_points={ 23 | 'nose.plugins.0.10': [ 24 | 'spec = spec:SpecPlugin', 25 | 'specselector = spec.cli:CustomSelector', 26 | ], 27 | 'console_scripts': [ 28 | 'spec = spec:main' 29 | ], 30 | }, 31 | classifiers=[ 32 | 'Development Status :: 4 - Beta', 33 | 'Environment :: Plugins', 34 | 'Intended Audience :: Developers', 35 | 'License :: OSI Approved :: MIT License', 36 | 'Programming Language :: Python', 37 | 'Programming Language :: Python :: 2.6', 38 | 'Programming Language :: Python :: 2.7', 39 | 'Programming Language :: Python :: 3.3', 40 | 'Topic :: Software Development :: Testing' 41 | ] 42 | ) 43 | -------------------------------------------------------------------------------- /spec/__init__.py: -------------------------------------------------------------------------------- 1 | from __future__ import unicode_literals 2 | 3 | import re 4 | from functools import partial 5 | 6 | import six 7 | 8 | from nose import SkipTest 9 | # Gets us eq_, ok_, etc 10 | # TODO: see what I or others are using here and make explicit, ugh 11 | from nose.tools import * 12 | from nose.tools import ok_ as upstream_ok_ 13 | 14 | from spec.plugin import SpecPlugin 15 | from spec.cli import main 16 | from spec.utils import InnerClassParser, hide 17 | from spec.trap import trap 18 | 19 | 20 | class Spec(six.with_metaclass(InnerClassParser, object)): 21 | """ 22 | Parent class for spec classes wishing to use inner class contexts. 23 | """ 24 | 25 | 26 | # Simple helper 27 | def skip(): 28 | raise SkipTest 29 | 30 | 31 | # Multiline string comparison helper ripped from Fabric 1.x 32 | def eq_(result, expected, msg=None): 33 | """ 34 | Shadow of the Nose builtin which presents easier to read multiline output. 35 | """ 36 | params = {'expected': expected, 'result': result} 37 | aka = """ 38 | 39 | --------------------------------- aka ----------------------------------------- 40 | 41 | Expected: 42 | %(expected)r 43 | 44 | Got: 45 | %(result)r 46 | """ % params 47 | default_msg = """ 48 | Expected: 49 | %(expected)s 50 | 51 | Got: 52 | %(result)s 53 | """ % params 54 | if ( 55 | (repr(result) != six.text_type(result)) or 56 | (repr(expected) != six.text_type(expected)) 57 | ): 58 | default_msg += aka 59 | assertion_msg = msg or default_msg 60 | # This assert will bubble up to Nose's failure handling, which at some 61 | # point calls explicit str() - which will UnicodeDecodeError on any non 62 | # ASCII text. 63 | # To work around this, we make sure Unicode strings become bytestrings 64 | # beforehand, with explicit encode. 65 | if isinstance(assertion_msg, six.text_type): 66 | assertion_msg = assertion_msg.encode('utf-8') 67 | assert result == expected, assertion_msg 68 | 69 | 70 | # Unicode-friendlier ok_ 71 | def ok_(assertion, msg=None): 72 | if msg is not None: 73 | # Same as in eq_ above re: need to correctly encode before nose calls 74 | # str()... 75 | if isinstance(msg, six.text_type): 76 | msg = msg.encode('utf-8') 77 | upstream_ok_(assertion, msg) 78 | 79 | 80 | def _assert_contains(haystack, needle, invert, escape=False): 81 | """ 82 | Test for existence of ``needle`` regex within ``haystack``. 83 | 84 | Say ``escape`` to escape the ``needle`` if you aren't really using the 85 | regex feature & have special characters in it. 86 | """ 87 | myneedle = re.escape(needle) if escape else needle 88 | matched = re.search(myneedle, haystack, re.M) 89 | if (invert and matched) or (not invert and not matched): 90 | raise AssertionError("'%s' %sfound in '%s'" % ( 91 | needle, 92 | "" if invert else "not ", 93 | haystack 94 | )) 95 | 96 | assert_contains = partial(_assert_contains, invert=False) 97 | assert_not_contains = partial(_assert_contains, invert=True) 98 | -------------------------------------------------------------------------------- /spec/_version.py: -------------------------------------------------------------------------------- 1 | __version_info__ = (1, 4, 1) 2 | __version__ = '.'.join(map(str, __version_info__)) 3 | -------------------------------------------------------------------------------- /spec/cli.py: -------------------------------------------------------------------------------- 1 | import inspect 2 | import sys 3 | import os 4 | 5 | import nose 6 | import six 7 | 8 | from spec.utils import class_members 9 | 10 | 11 | # 12 | # Custom selection logic 13 | # 14 | 15 | 16 | def private(obj): 17 | return obj.__name__.startswith('_') or \ 18 | getattr(obj, '_spec__is_private', False) 19 | 20 | 21 | class SpecSelector(nose.selector.Selector): 22 | def __init__(self, *args, **kwargs): 23 | super(SpecSelector, self).__init__(*args, **kwargs) 24 | self._valid_modules = [] 25 | # Handle --tests= 26 | self._valid_named_modules = list(map(os.path.abspath, self.config.testNames)) 27 | self._valid_classes = [] 28 | 29 | def wantDirectory(self, dirname): 30 | # Given a sane root such as tests/, we want everything. 31 | # Some other mechanism already allows for hidden directories using a _ 32 | # prefix, e.g. _support. 33 | return True 34 | 35 | def wantFile(self, filename): 36 | # Same as with directories -- anything unhidden goes. 37 | # Also skip .pyc files 38 | is_pyc = os.path.splitext(filename)[1] == '.pyc' 39 | is_hidden = os.path.basename(filename).startswith('_') 40 | return not (is_pyc or is_hidden) 41 | 42 | def wantModule(self, module): 43 | # You guessed it -- if it's being picked up as a module, we want it. 44 | # However, also store it so we can tell apart "native" class/func 45 | # objects from ones imported *into* test modules. 46 | self._valid_modules.append(module) 47 | return True 48 | 49 | def wantFunction(self, function): 50 | # Only use locally-defined functions 51 | local = inspect.getmodule(function) in self._valid_modules 52 | # And not ones which are conventionally private 53 | good = local and not private(function) 54 | return good 55 | 56 | def registerGoodClass(self, class_): 57 | """ 58 | Internal bookkeeping to handle nested classes 59 | """ 60 | # Class itself added to "good" list 61 | self._valid_classes.append(class_) 62 | # Recurse into any inner classes 63 | for name, cls in class_members(class_): 64 | if self.isValidClass(cls): 65 | self.registerGoodClass(cls) 66 | 67 | def isValidClass(self, class_): 68 | """ 69 | Needs to be its own method so it can be called from both wantClass and 70 | registerGoodClass. 71 | """ 72 | module = inspect.getmodule(class_) 73 | valid = ( 74 | module in self._valid_modules 75 | or ( 76 | hasattr(module, '__file__') 77 | and module.__file__ in self._valid_named_modules 78 | ) 79 | ) 80 | return valid and not private(class_) 81 | 82 | def wantClass(self, class_): 83 | # As with modules, track the valid ones for use in method testing. 84 | # Valid meaning defined locally in a valid module, and not private. 85 | good = self.isValidClass(class_) 86 | if good: 87 | self.registerGoodClass(class_) 88 | return good 89 | 90 | def wantMethod(self, method): 91 | if six.PY3: 92 | cls = method.__self__.__class__ 93 | else: 94 | # Short-circuit on odd results 95 | if not hasattr(method, 'im_class'): 96 | return False 97 | cls = method.im_class 98 | 99 | # As with functions, we want only items defined on also-valid 100 | # containers (classes), and only ones not conventionally private. 101 | valid_class = cls in self._valid_classes 102 | # And ones only defined local to the class in question, not inherited 103 | # from its parents. Also handle oddball 'type' cases. 104 | if cls is type: 105 | return False 106 | # Handle 'contributed' methods not defined on class itself 107 | if not hasattr(cls, method.__name__): 108 | return False 109 | # Only test for mro on new-style classes. (inner old-style classes lack 110 | # it.) 111 | if hasattr(cls, '__mro__'): 112 | candidates = list(reversed(cls.__mro__))[:-1] 113 | for candidate in candidates: 114 | if hasattr(candidate, method.__name__): 115 | return False 116 | ok = valid_class and not private(method) 117 | return ok 118 | 119 | 120 | # Plugin for loading selector & implementing some custom hooks too 121 | # (such as appending more test cases from gathered classes) 122 | class CustomSelector(nose.plugins.Plugin): 123 | name = "specselector" 124 | 125 | def configure(self, options, conf): 126 | nose.plugins.Plugin.configure(self, options, conf) 127 | 128 | def prepareTestLoader(self, loader): 129 | loader.selector = SpecSelector(loader.config) 130 | self.loader = loader 131 | 132 | def loadTestsFromTestClass(self, cls): 133 | """ 134 | Manually examine test class for inner classes. 135 | """ 136 | results = [] 137 | for name, subclass in class_members(cls): 138 | results.extend(self.loader.loadTestsFromTestClass(subclass)) 139 | return results 140 | 141 | 142 | def args_contains(options): 143 | for opt in options: 144 | for arg in sys.argv[1:]: 145 | if arg.startswith(opt): 146 | return True 147 | return False 148 | 149 | 150 | # Nose invocation 151 | def main(): 152 | defaults = [ 153 | # Don't capture stdout 154 | '--nocapture', 155 | # Use the spec plugin 156 | '--with-specplugin', '--with-specselector', 157 | # Enable useful asserts 158 | '--detailed-errors', 159 | ] 160 | # Set up default test location ('tests/') and custom selector, 161 | # only if user isn't giving us specific options of their own. 162 | # FIXME: see if there's a way to do it post-optparse, this is brittle. 163 | good = not args_contains("--match -m -i --include -e --exclude".split()) 164 | plugins = [] 165 | if good and os.path.isdir('tests'): 166 | plugins = [CustomSelector()] 167 | if not args_contains(['--tests', '-w', '--where']): 168 | defaults.append("--where=tests") 169 | nose.core.main( 170 | argv=['nosetests'] + defaults + sys.argv[1:], 171 | addplugins=plugins 172 | ) 173 | -------------------------------------------------------------------------------- /spec/plugin.py: -------------------------------------------------------------------------------- 1 | import doctest 2 | import os 3 | import re 4 | import types 5 | import time 6 | import unittest 7 | from functools import partial 8 | # Python 2.7: _WritelnDecorator moved. 9 | try: 10 | from unittest import _WritelnDecorator 11 | except ImportError: 12 | from unittest.runner import _WritelnDecorator 13 | 14 | import six 15 | from six import StringIO as IO 16 | import nose 17 | from nose.plugins import Plugin 18 | # Python 2.7: nose uses unittest's builtin SkipTest class 19 | try: 20 | SkipTest = unittest.case.SkipTest 21 | except AttributeError: 22 | SkipTest = nose.SkipTest 23 | 24 | # Use custom-as-of-nose-1.3 format_exception which bridges some annoying 25 | # python2 vs python3 issues. 26 | from nose.plugins.xunit import format_exception 27 | 28 | ################################################################################ 29 | ## Functions for constructing specifications based on nose testing objects. 30 | ################################################################################ 31 | 32 | def dispatch_on_type(dispatch_table, instance): 33 | for type, func in dispatch_table: 34 | if type is True or isinstance(instance, type): 35 | return func(instance) 36 | 37 | 38 | def remove_leading(needle, haystack): 39 | """Remove leading needle string (if exists). 40 | 41 | >>> remove_leading('Test', 'TestThisAndThat') 42 | 'ThisAndThat' 43 | >>> remove_leading('Test', 'ArbitraryName') 44 | 'ArbitraryName' 45 | """ 46 | if haystack[:len(needle)] == needle: 47 | return haystack[len(needle):] 48 | return haystack 49 | 50 | 51 | def remove_trailing(needle, haystack): 52 | """Remove trailing needle string (if exists). 53 | 54 | >>> remove_trailing('Test', 'ThisAndThatTest') 55 | 'ThisAndThat' 56 | >>> remove_trailing('Test', 'ArbitraryName') 57 | 'ArbitraryName' 58 | """ 59 | if haystack[-len(needle):] == needle: 60 | return haystack[:-len(needle)] 61 | return haystack 62 | 63 | 64 | def remove_leading_and_trailing(needle, haystack): 65 | return remove_leading(needle, remove_trailing(needle, haystack)) 66 | 67 | 68 | def camel2word(string): 69 | """Covert name from CamelCase to "Normal case". 70 | 71 | >>> camel2word('CamelCase') 72 | 'Camel case' 73 | >>> camel2word('CaseWithSpec') 74 | 'Case with spec' 75 | """ 76 | def wordize(match): 77 | return ' ' + match.group(1).lower() 78 | 79 | return string[0] + re.sub(r'([A-Z])', wordize, string[1:]) 80 | 81 | 82 | def complete_english(string): 83 | """ 84 | >>> complete_english('dont do this') 85 | "don't do this" 86 | >>> complete_english('doesnt is matched as well') 87 | "doesn't is matched as well" 88 | """ 89 | for x, y in [("dont", "don't"), 90 | ("doesnt", "doesn't"), 91 | ("wont", "won't"), 92 | ("wasnt", "wasn't")]: 93 | string = string.replace(x, y) 94 | return string 95 | 96 | 97 | def underscore2word(string): 98 | return string.replace('_', ' ') 99 | 100 | 101 | def argumentsof(test): 102 | if test.arg: 103 | if len(test.arg) == 1: 104 | return " for %s" % test.arg[0] 105 | else: 106 | return " for %s" % (test.arg,) 107 | return "" 108 | 109 | 110 | def underscored2spec(name): 111 | return complete_english(underscore2word(remove_trailing('_test', remove_leading('test_', name)))) 112 | 113 | 114 | def camelcase2spec(name): 115 | return camel2word( 116 | remove_trailing('_', 117 | remove_leading_and_trailing('Test', name))) 118 | 119 | 120 | def camelcaseDescription(object): 121 | description = object.__doc__ or camelcase2spec(object.__name__) 122 | return description.strip() 123 | 124 | 125 | def underscoredDescription(object): 126 | return object.__doc__ or underscored2spec(object.__name__).capitalize() 127 | 128 | 129 | def doctestContextDescription(doctest): 130 | return doctest._dt_test.name 131 | 132 | 133 | def noseMethodDescription(test): 134 | return test.method.__doc__ or underscored2spec(test.method.__name__) 135 | 136 | 137 | def unittestMethodDescription(test): 138 | if test._testMethodDoc is None: 139 | return underscored2spec(test._testMethodName) 140 | else: 141 | description = test._testMethodDoc.split("\n") 142 | return "".join([text.strip() for text in description]) 143 | 144 | 145 | def noseFunctionDescription(test): 146 | # Special case for test generators. 147 | if test.descriptor is not None: 148 | if hasattr(test.test, 'description'): 149 | return test.test.description 150 | return "holds for %s" % ', '.join(map(six.text_type, test.arg)) 151 | return test.test.__doc__ or underscored2spec(test.test.__name__) 152 | 153 | 154 | # Different than other similar functions, this one returns a generator 155 | # of specifications. 156 | def doctestExamplesDescription(test): 157 | for ex in test._dt_test.examples: 158 | source = ex.source.replace("\n", " ") 159 | want = None 160 | if '#' in source: 161 | source, want = source.rsplit('#', 1) 162 | elif ex.exc_msg: 163 | want = "throws \"%s\"" % ex.exc_msg.rstrip() 164 | elif ex.want: 165 | want = "returns %s" % ex.want.replace("\n", " ") 166 | 167 | if want: 168 | yield "%s %s" % (source.strip(), want.strip()) 169 | 170 | 171 | def testDescription(test): 172 | supported_test_types = [ 173 | (nose.case.MethodTestCase, noseMethodDescription), 174 | (nose.case.FunctionTestCase, noseFunctionDescription), 175 | (doctest.DocTestCase, doctestExamplesDescription), 176 | (unittest.TestCase, unittestMethodDescription), 177 | ] 178 | return dispatch_on_type(supported_test_types, test.test) 179 | 180 | 181 | def contextDescription(context): 182 | supported_context_types = [ 183 | (types.ModuleType, underscoredDescription), 184 | (types.FunctionType, underscoredDescription), 185 | (doctest.DocTestCase, doctestContextDescription), 186 | (type, camelcaseDescription), 187 | ] 188 | 189 | if not six.PY3: 190 | supported_context_types += [ 191 | # Handle both old and new style classes. 192 | (types.ClassType, camelcaseDescription), 193 | ] 194 | 195 | return dispatch_on_type(supported_context_types, context) 196 | 197 | 198 | def testContext(test): 199 | # Test generators set their own contexts. 200 | if isinstance(test.test, nose.case.FunctionTestCase) \ 201 | and test.test.descriptor is not None: 202 | return test.test.descriptor 203 | # So do doctests. 204 | elif isinstance(test.test, doctest.DocTestCase): 205 | return test.test 206 | else: 207 | return test.context 208 | 209 | 210 | ################################################################################ 211 | ## Output stream that can be easily enabled and disabled. 212 | ################################################################################ 213 | 214 | class OutputStream(_WritelnDecorator): 215 | def __init__(self, on_stream, off_stream): 216 | self.capture_stream = IO() 217 | self.on_stream = on_stream 218 | self.off_stream = off_stream 219 | self.stream = on_stream 220 | 221 | def on(self): 222 | self.stream = self.on_stream 223 | 224 | def off(self): 225 | self.stream = self.off_stream 226 | 227 | def capture(self): 228 | self.capture_stream.truncate() 229 | self.stream = self.capture_stream 230 | 231 | def get_captured(self): 232 | self.capture_stream.seek(0) 233 | return self.capture_stream.read() 234 | 235 | 236 | def depth(context): 237 | level = 0 238 | while hasattr(context, '_parent'): 239 | level += 1 240 | context = context._parent 241 | return level 242 | 243 | 244 | class SpecOutputStream(OutputStream): 245 | def print_text(self, text): 246 | self.on() 247 | self.write(text) 248 | self.off() 249 | 250 | def print_line(self, line=''): 251 | self.print_text(line + "\n") 252 | 253 | @property 254 | def _indent(self): 255 | return " " * self._depth 256 | 257 | def print_context(self, context): 258 | # Ensure parents get printed too (e.g. an outer class with nothing but 259 | # inner classes will otherwise never get printed.) 260 | if ( 261 | hasattr(context, '_parent') 262 | and not getattr(context._parent, '_printed', False) 263 | ): 264 | self.print_context(context._parent) 265 | # Adjust indentation depth 266 | self._depth = depth(context) 267 | self.print_line("\n%s%s" % (self._indent, contextDescription(context))) 268 | context._printed = True 269 | 270 | def print_spec(self, color_func, test, status=None): 271 | spec = testDescription(test) 272 | if not isinstance(spec, types.GeneratorType): 273 | spec = [spec.strip()] 274 | for s in spec: 275 | name = "- %s" % s 276 | paren = (" (%s)" % status) if status else "" 277 | indent = getattr(self, '_indent', "") 278 | self.print_line(indent + color_func(name) + paren) 279 | 280 | 281 | 282 | ################################################################################ 283 | ## Color helpers. 284 | ################################################################################ 285 | 286 | color_end = "\x1b[1;0m" 287 | colors = dict( 288 | green="32", 289 | red="31", 290 | yellow="33", 291 | purple="35", 292 | cyan="36", 293 | blue="34" 294 | ) 295 | 296 | def colorize(color, text, bold=False): 297 | bold = 1 if bold else 0 298 | return "\x1b[%s;%sm%s%s" % (bold, colors[color], text, color_end) 299 | 300 | ################################################################################ 301 | ## Plugin itself. 302 | ################################################################################ 303 | 304 | class SpecPlugin(Plugin): 305 | """Generate specification from test class/method names. 306 | """ 307 | score = 1100 # must be higher than Deprecated and Skip plugins scores 308 | 309 | def __init__(self, *args, **kwargs): 310 | super(SpecPlugin, self).__init__(*args, **kwargs) 311 | self._failures = [] 312 | self._errors = [] 313 | self.color = {} 314 | 315 | def options(self, parser, env=os.environ): 316 | Plugin.options(self, parser, env) 317 | parser.add_option('--no-spec-color', action='store_true', 318 | dest='no_spec_color', 319 | default=env.get('NOSE_NO_SPEC_COLOR'), 320 | help="Don't show colors with --with-spec" 321 | "[NOSE_NO_SPEC_COLOR]") 322 | parser.add_option('--spec-doctests', action='store_true', 323 | dest='spec_doctests', 324 | default=env.get('NOSE_SPEC_DOCTESTS'), 325 | help="Include doctests in specifications " 326 | "[NOSE_SPEC_DOCTESTS]") 327 | parser.add_option('--no-detailed-errors', action='store_false', 328 | dest='detailedErrors', 329 | help="Force detailed errors off") 330 | parser.add_option('--with-timing', action='store_true', 331 | help="Display timing info for slow tests") 332 | parser.add_option('--timing-threshold', 333 | metavar="SECONDS", 334 | type=float, 335 | default=0.1, 336 | help="Number (float) of seconds above which to display test runtime. Default: 0.1") 337 | 338 | def configure(self, options, config): 339 | # Configure 340 | Plugin.configure(self, options, config) 341 | # Set options 342 | if options.enable_plugin_specplugin: 343 | options.verbosity = max(options.verbosity, 2) 344 | self.spec_doctests = options.spec_doctests 345 | self.show_timing = options.with_timing 346 | self.timing_threshold = options.timing_threshold 347 | # Color setup 348 | for label, color in list({ 349 | 'error': 'red', 350 | 'ok': 'green', 351 | 'deprecated': 'yellow', 352 | 'skipped': 'yellow', 353 | 'failure': 'red', 354 | 'identifier': 'cyan', 355 | 'file': 'blue', 356 | }.items()): 357 | # No color: just print() really 358 | func = lambda text, bold=False: text 359 | if not options.no_spec_color: 360 | # Color: colorizes! 361 | func = partial(colorize, color) 362 | # Store in dict (slightly quicker/nicer than getattr) 363 | self.color[label] = func 364 | # Add attribute for easier hardcoded access 365 | setattr(self, label, func) 366 | 367 | def begin(self): 368 | self.current_context = None 369 | self.start_time = time.time() 370 | 371 | def setOutputStream(self, stream): 372 | self.stream = SpecOutputStream(stream, open(os.devnull, 'w')) 373 | return self.stream 374 | 375 | def beforeTest(self, test): 376 | context = testContext(test) 377 | if context != self.current_context: 378 | self._print_context(context) 379 | self.current_context = context 380 | 381 | self.stream.off() 382 | test._starttime = time.time() 383 | 384 | def addSuccess(self, test): 385 | runtime = round(time.time() - test._starttime, 2) 386 | if runtime >= self.timing_threshold and self.show_timing: 387 | status = "{0}s".format(runtime) 388 | else: 389 | status = None 390 | self._print_spec('ok', test, status) 391 | 392 | def addFailure(self, test, err): 393 | self._print_spec('failure', test, '') 394 | self._failures.append((test, err)) 395 | 396 | def addError(self, test, err): 397 | def blurt(color, label): 398 | self._print_spec(color, test, label) 399 | 400 | klass = err[0] 401 | if issubclass(klass, nose.DeprecatedTest): 402 | blurt('deprecated', '') 403 | elif issubclass(klass, SkipTest): 404 | blurt('skipped', '') 405 | else: 406 | self._errors.append((test, err)) 407 | blurt('error', '') 408 | 409 | def afterTest(self, test): 410 | self.stream.capture() 411 | 412 | def print_tracebacks(self, label, items): 413 | problem_color = { 414 | "ERROR": "error", 415 | "FAIL": "failure" 416 | }[label] 417 | for item in items: 418 | test, trace = item 419 | desc = test.shortDescription() or six.text_type(test) 420 | self.stream.writeln("=" * 70) 421 | self.stream.writeln("%s: %s" % ( 422 | self.color[problem_color](label), 423 | self.identifier(desc, bold=True), 424 | )) 425 | self.stream.writeln("-" * 70) 426 | # format_exception() is...very odd re: how it breaks into lines. 427 | trace = "".join(format_exception(trace)).split("\n") 428 | self.print_colorized_traceback(trace) 429 | 430 | def print_colorized_traceback(self, formatted_traceback, indent_level=0): 431 | indentation = " " * indent_level 432 | for line in formatted_traceback: 433 | if line.startswith(" File"): 434 | m = re.match(r' File "(.*)", line (\d*)(?:, in (.*))?$', line) 435 | if m: 436 | filename, lineno, test = m.groups() 437 | tb_lines = [ 438 | ' File "', 439 | self.file(filename), 440 | '", line ', 441 | self.error(lineno), 442 | ] 443 | if test: 444 | # this is missing for the first traceback in doctest 445 | # failure report 446 | tb_lines.extend([ 447 | ", in ", 448 | self.identifier(test, bold=True) 449 | ]) 450 | tb_lines.extend(["\n"]) 451 | self.stream.write(indentation) 452 | self.stream.writelines(tb_lines) 453 | else: 454 | six.print_(indentation + line, file=self.stream) 455 | elif line.startswith(" "): 456 | six.print_(self.identifier(indentation + line), file=self.stream) 457 | elif line.startswith("Traceback (most recent call last)"): 458 | six.print_(indentation + line, file=self.stream) 459 | else: 460 | six.print_(self.error(indentation + line), file=self.stream) 461 | 462 | def finalize(self, result): 463 | self.stream.on() 464 | six.print_("", file=self.stream) 465 | self.print_tracebacks("ERROR", self._errors) 466 | self.print_tracebacks("FAIL", self._failures) 467 | self.print_summary(result) 468 | 469 | def print_summary(self, result): 470 | # Setup 471 | num_tests = result.testsRun 472 | success = result.wasSuccessful() 473 | # How many in how long 474 | six.print_("Ran %s test%s in %s" % ( 475 | (self.ok if success else self.error)(num_tests), 476 | "s" if num_tests > 1 else "", 477 | self.format_seconds(time.time() - self.start_time) 478 | ), file=self.stream) 479 | # Did we fail, and if so, how badly? 480 | if success: 481 | skipped = len(result.skipped) 482 | skipped_str = "(" + self.skipped("%i skipped" % skipped) + ")" 483 | six.print_(self.ok("OK"), skipped_str if skipped else "", file=self.stream) 484 | else: 485 | types = ( 486 | ('failures', 'failure'), 487 | ('errors', 'error'), 488 | ('skipped', 'skipped'), 489 | ) 490 | pairs = [] 491 | for label, color in types: 492 | num = len(getattr(result, label)) 493 | text = six.text_type(num) 494 | if num: 495 | text = self.color[color](text) 496 | pairs.append("%s=%s" % (label, text)) 497 | six.print_("%s (%s)" % ( 498 | self.failure("FAILED"), 499 | ", ".join(pairs) 500 | ), file=self.stream) 501 | six.print_("", file=self.stream) 502 | 503 | def format_seconds(self, n_seconds): 504 | """Format a time in seconds.""" 505 | func = self.ok 506 | if n_seconds >= 60: 507 | n_minutes, n_seconds = divmod(n_seconds, 60) 508 | return "%s minutes %s seconds" % ( 509 | func("%d" % n_minutes), 510 | func("%.3f" % n_seconds)) 511 | else: 512 | return "%s seconds" % ( 513 | func("%.3f" % n_seconds)) 514 | 515 | def _print_context(self, context): 516 | if isinstance(context, doctest.DocTestCase) and not self.spec_doctests: 517 | return 518 | self.stream.print_context(context) 519 | 520 | def _print_spec(self, color, test, status=None): 521 | # If setUp() fails during nose.suite.run(), there is no test attribute 522 | if not hasattr(test, 'test') or \ 523 | isinstance(test.test, doctest.DocTestCase) and not self.spec_doctests: 524 | return 525 | self.stream.print_spec(self.color[color], test, status) 526 | -------------------------------------------------------------------------------- /spec/trap.py: -------------------------------------------------------------------------------- 1 | """ 2 | Test decorator for capturing stdout/stderr/both. 3 | 4 | Based on original code from Fabric 1.x, specifically: 5 | 6 | * fabric/tests/utils.py 7 | * as of Git SHA 62abc4e17aab0124bf41f9c5f9c4bc86cc7d9412 8 | 9 | Though modifications have been made since. 10 | """ 11 | import sys 12 | from functools import wraps 13 | 14 | import six 15 | from six import BytesIO as IO 16 | 17 | 18 | class CarbonCopy(IO): 19 | """ 20 | An IO wrapper capable of multiplexing its writes to other buffer objects. 21 | """ 22 | # NOTE: because StringIO.StringIO on Python 2 is an old-style class we 23 | # cannot use super() :( 24 | def __init__(self, buffer=b'', cc=None): 25 | """ 26 | If ``cc`` is given and is a file-like object or an iterable of same, 27 | it/they will be written to whenever this instance is written to. 28 | """ 29 | IO.__init__(self, buffer) 30 | if cc is None: 31 | cc = [] 32 | elif hasattr(cc, 'write'): 33 | cc = [cc] 34 | self.cc = cc 35 | 36 | def write(self, s): 37 | # Ensure we always write bytes. This means that wrapped code calling 38 | # print() in Python 3 will still work. Sigh. 39 | if isinstance(s, six.text_type): 40 | s = s.encode('utf-8') 41 | # Write out to our capturing object & any CC's 42 | IO.write(self, s) 43 | for writer in self.cc: 44 | writer.write(s) 45 | 46 | # Dumb hack to deal with py3 expectations; real sys.std(out|err) in Py3 47 | # requires writing to a buffer attribute obj in some situations. 48 | @property 49 | def buffer(self): 50 | return self 51 | 52 | # Make sure we always hand back strings, even on Python 3 53 | def getvalue(self): 54 | ret = IO.getvalue(self) 55 | if isinstance(ret, six.binary_type): 56 | ret = ret.decode('utf-8') 57 | return ret 58 | 59 | 60 | def trap(func): 61 | """ 62 | Replace sys.std(out|err) with a wrapper during execution, restored after. 63 | 64 | In addition, a new combined-streams output (another wrapper) will appear at 65 | ``sys.stdall``. This stream will resemble what a user sees at a terminal, 66 | i.e. both out/err streams intermingled. 67 | """ 68 | @wraps(func) 69 | def wrapper(*args, **kwargs): 70 | # Use another CarbonCopy even though we're not cc'ing; for our "write 71 | # bytes, return strings on py3" behavior. Meh. 72 | sys.stdall = CarbonCopy() 73 | my_stdout, sys.stdout = sys.stdout, CarbonCopy(cc=sys.stdall) 74 | my_stderr, sys.stderr = sys.stderr, CarbonCopy(cc=sys.stdall) 75 | try: 76 | return func(*args, **kwargs) 77 | finally: 78 | sys.stdout = my_stdout 79 | sys.stderr = my_stderr 80 | del sys.stdall 81 | return wrapper 82 | -------------------------------------------------------------------------------- /spec/utils.py: -------------------------------------------------------------------------------- 1 | import six 2 | 3 | from nose.util import isclass 4 | 5 | 6 | def hide(obj): 7 | """ 8 | Mark object as private. 9 | """ 10 | obj._spec__is_private = True 11 | return obj 12 | 13 | def is_public_class(name, value): 14 | return isclass(value) and not name.startswith('_') 15 | 16 | def class_members(obj): 17 | return [x for x in six.iteritems(vars(obj)) if is_public_class(*x)] 18 | 19 | def my_getattr(self, name): 20 | if not self._parent_inst: 21 | parent = self._parent() 22 | if hasattr(parent, 'setup') and callable(getattr(parent, 'setup')): 23 | # TODO: how to call higher-up methods with lower-down selves? can't 24 | # even manually rebind because there's no real inheritance going 25 | # on. (CAN there be any inheritance going on?) 26 | parent.setup() 27 | self._parent_inst = parent 28 | return getattr(self._parent_inst, name) 29 | 30 | def flag_inner_classes(obj): 31 | """ 32 | Mutates any attributes on ``obj`` which are classes, with link to ``obj``. 33 | 34 | Adds a convenience accessor which instantiates ``obj`` and then calls its 35 | ``setup`` method. 36 | 37 | Recurses on those objects as well. 38 | """ 39 | for tup in class_members(obj): 40 | tup[1]._parent = obj 41 | tup[1]._parent_inst = None 42 | tup[1].__getattr__ = my_getattr 43 | flag_inner_classes(tup[1]) 44 | 45 | def autohide(obj): 46 | """ 47 | Automatically hide setup() and teardown() methods, recursively. 48 | """ 49 | # Members on obj 50 | for name, item in six.iteritems(vars(obj)): 51 | if callable(item) and name in ('setup', 'teardown'): 52 | item = hide(item) 53 | # Recurse into class members 54 | for name, subclass in class_members(obj): 55 | autohide(subclass) 56 | 57 | 58 | class InnerClassParser(type): 59 | """ 60 | Metaclass that tags inner classes with a link to the parent class. 61 | 62 | Allows test loading machinery to determine if a given test is part of an 63 | inner class or a top level one. 64 | """ 65 | def __new__(cls, name, bases, attrs): 66 | new_class = type.__new__(cls, name, bases, attrs) 67 | flag_inner_classes(new_class) 68 | autohide(new_class) 69 | return new_class 70 | -------------------------------------------------------------------------------- /tasks.py: -------------------------------------------------------------------------------- 1 | from invoke import Collection 2 | 3 | from invocations.packaging import release 4 | 5 | 6 | ns = Collection(release) 7 | ns.configure({ 8 | 'packaging': { 9 | 'sign': True, 10 | 'wheel': True, 11 | 'check_desc': True, 12 | }, 13 | }) 14 | -------------------------------------------------------------------------------- /test.py: -------------------------------------------------------------------------------- 1 | def test_good(): 2 | assert True 3 | 4 | def test_bad(): 5 | assert False 6 | 7 | def test_boom(): 8 | what 9 | 10 | def test_skip(): 11 | from nose.plugins.skip import SkipTest 12 | raise SkipTest 13 | 14 | 15 | class Foo(object): 16 | def has_no_underscore(self): 17 | assert True 18 | 19 | def has_no_Test(self): 20 | assert True 21 | 22 | class Foo_(object): 23 | def should_print_out_as_Foo(self): 24 | pass 25 | -------------------------------------------------------------------------------- /tests/_spec_test_cases/containers.py: -------------------------------------------------------------------------------- 1 | def test_are_marked_as_deprecated(): 2 | pass 3 | 4 | 5 | def test_doesnt_work_with_sets(): 6 | pass 7 | -------------------------------------------------------------------------------- /tests/_spec_test_cases/docstring_spec_names.py: -------------------------------------------------------------------------------- 1 | "This module" 2 | 3 | 4 | def test_function(): 5 | """uses function to do this and that""" 6 | 7 | 8 | class TestYetAnotherClass: 9 | def test_has_a_lot_of_methods(self): 10 | pass 11 | 12 | def test_foobared(self): 13 | "has a nice descriptions inside test methods" 14 | pass 15 | -------------------------------------------------------------------------------- /tests/_spec_test_cases/doctests.py: -------------------------------------------------------------------------------- 1 | """ 2 | >>> 2 + 3 3 | 5 4 | >>> None 5 | >>> None # is nothing 6 | >>> foobar 7 | Traceback (most recent call last): 8 | ... 9 | NameError: name 'foobar' is not defined 10 | """ 11 | -------------------------------------------------------------------------------- /tests/_spec_test_cases/foobar.py: -------------------------------------------------------------------------------- 1 | import unittest 2 | 3 | 4 | class TestFoobar: 5 | def test_can_be_automatically_documented(self): 6 | pass 7 | 8 | def test_is_a_singleton(self): 9 | pass 10 | 11 | 12 | class TestBazBar(unittest.TestCase): 13 | def test_does_this_and_that(self): 14 | pass 15 | -------------------------------------------------------------------------------- /tests/_spec_test_cases/foobaz.py: -------------------------------------------------------------------------------- 1 | import nose 2 | 3 | 4 | class TestFoobaz(object): 5 | def test_behaves_such_and_such(self): 6 | assert True 7 | 8 | def test_causes_an_error(self): 9 | raise NameError 10 | 11 | def test_fails_to_satisfy_this_specification(self): 12 | assert False 13 | 14 | def test_throws_deprecated_exception(self): 15 | raise nose.DeprecatedTest 16 | 17 | def test_throws_skip_test_exception(self): 18 | raise nose.SkipTest 19 | -------------------------------------------------------------------------------- /tests/_spec_test_cases/generators.py: -------------------------------------------------------------------------------- 1 | def is_even(n): 2 | return n % 2 == 0 3 | 4 | 5 | def test_product_of_even_numbers_is_even(): 6 | evens = [(18, 8), (14, 12), (0, 4), (6, 2), (16, 10)] 7 | for e1, e2 in evens: 8 | yield check_even, e1, e2 9 | 10 | 11 | def check_even(e1, e2): 12 | assert is_even(e1 * e2) 13 | -------------------------------------------------------------------------------- /tests/_spec_test_cases/generators_with_descriptions.py: -------------------------------------------------------------------------------- 1 | def is_even(n): 2 | return n % 2 == 0 3 | 4 | 5 | def test_product_of_even_numbers_is_even(): 6 | "Natural numbers truths" 7 | evens = [(18, 8), (14, 12), (0, 4), (6, 2), (16, 10)] 8 | for e1, e2 in evens: 9 | check_even.description = "for even numbers %d and %d their product is even as well" % (e1, e2) 10 | yield check_even, e1, e2 11 | 12 | 13 | def check_even(e1, e2): 14 | assert is_even(e1 * e2) 15 | -------------------------------------------------------------------------------- /tests/spec_test.py: -------------------------------------------------------------------------------- 1 | """Unit tests for Spec plugin. 2 | """ 3 | 4 | import unittest 5 | import nose 6 | import six 7 | from nose.plugins import PluginTester 8 | 9 | from spec import Spec 10 | 11 | 12 | def _prepend_in_each_line(string, prefix=' '): 13 | return ''.join([prefix + s for s in string.splitlines(True)]) 14 | 15 | 16 | class _SpecPluginTestCase(PluginTester, unittest.TestCase): 17 | activate = '--with-spec' 18 | args = ['--no-spec-color'] 19 | plugins = [Spec()] 20 | 21 | def _get_suitepath(self): 22 | return '_spec_test_cases/%s.py' % self.suitename 23 | suitepath = property(_get_suitepath) 24 | 25 | def assertContains(self, needle, haystack): 26 | assert needle in haystack,\ 27 | "Failed to find:\n\n%s\ninside\n%s\n" % \ 28 | (_prepend_in_each_line(needle), _prepend_in_each_line(haystack)) 29 | 30 | def assertContainsInOutput(self, string): 31 | self.assertContains(string, six.text_type(self.output)) 32 | 33 | def failIfContains(self, needle, haystack): 34 | assert needle not in haystack,\ 35 | "Found:\n\n%s\ninside\n%s\n" % \ 36 | (_prepend_in_each_line(needle), _prepend_in_each_line(haystack)) 37 | 38 | def failIfContainsInOutput(self, string): 39 | self.failIfContains(string, six.text_type(self.output)) 40 | 41 | 42 | class TestPluginSpecWithFoobar(_SpecPluginTestCase): 43 | suitename = 'foobar' 44 | expected_test_foobar_output = """Foobar 45 | - can be automatically documented 46 | - is a singleton 47 | """ 48 | expected_test_bazbar_output = """Baz bar 49 | - does this and that 50 | """ 51 | 52 | def test_builds_specifications_for_test_classes(self): 53 | self.assertContainsInOutput(self.expected_test_foobar_output) 54 | 55 | def test_builds_specifications_for_unittest_test_cases(self): 56 | self.assertContainsInOutput(self.expected_test_bazbar_output) 57 | 58 | 59 | class TestPluginSpecWithFoobaz(_SpecPluginTestCase): 60 | suitename = 'foobaz' 61 | expected_test_foobaz_output = """Foobaz 62 | - behaves such and such 63 | - causes an error (ERROR) 64 | - fails to satisfy this specification (FAILED) 65 | - throws deprecated exception (DEPRECATED) 66 | - throws skip test exception (SKIPPED) 67 | """ 68 | 69 | def test_marks_failed_specifications_properly(self): 70 | self.assertContainsInOutput(self.expected_test_foobaz_output) 71 | 72 | 73 | # Make sure DEPRECATED and SKIPPED are still present in the output when set 74 | # of standard nose plugins is enabled. 75 | class TestPluginSpecWithFoobazAndStandardPluginsEnabled(TestPluginSpecWithFoobaz): 76 | plugins = [Spec(), nose.plugins.skip.Skip(), nose.plugins.deprecated.Deprecated()] 77 | 78 | 79 | class TestPluginSpecWithContainers(_SpecPluginTestCase): 80 | suitename = 'containers' 81 | expected_test_containers_output = """Containers 82 | - are marked as deprecated 83 | - doesn't work with sets 84 | """ 85 | 86 | def test_builds_specifications_for_test_modules(self): 87 | self.assertContainsInOutput(self.expected_test_containers_output) 88 | 89 | 90 | class TestPluginSpecWithDocstringSpecNames(_SpecPluginTestCase): 91 | suitename = 'docstring_spec_names' 92 | expected_test_docstring_spec_modules_names_output = """This module 93 | - uses function to do this and that 94 | """ 95 | expected_test_docstring_spec_class_names_output = """Yet another class 96 | - has a nice descriptions inside test methods 97 | - has a lot of methods 98 | """ 99 | 100 | def test_names_specifications_after_docstrings_if_present(self): 101 | self.assertContainsInOutput(self.expected_test_docstring_spec_modules_names_output) 102 | self.assertContainsInOutput(self.expected_test_docstring_spec_class_names_output) 103 | 104 | 105 | class TestPluginSpecWithTestGenerators(_SpecPluginTestCase): 106 | suitename = 'generators' 107 | expected_test_test_generators_output = """Product of even numbers is even 108 | - holds for 18, 8 109 | - holds for 14, 12 110 | - holds for 0, 4 111 | - holds for 6, 2 112 | - holds for 16, 10 113 | """ 114 | 115 | def test_builds_specifications_for_test_generators(self): 116 | self.assertContainsInOutput(self.expected_test_test_generators_output) 117 | 118 | 119 | class TestPluginSpecWithTestGeneratorsWithDescriptions(_SpecPluginTestCase): 120 | suitename = 'generators_with_descriptions' 121 | expected_test_test_generators_with_descriptions_output = """Natural numbers truths 122 | - for even numbers 18 and 8 their product is even as well 123 | - for even numbers 14 and 12 their product is even as well 124 | - for even numbers 0 and 4 their product is even as well 125 | - for even numbers 6 and 2 their product is even as well 126 | - for even numbers 16 and 10 their product is even as well 127 | """ 128 | 129 | def test_builds_specifications_for_test_generators_using_description_attribute_if_present(self): 130 | self.assertContainsInOutput(self.expected_test_test_generators_with_descriptions_output) 131 | 132 | 133 | class TestPluginSpecWithDoctests(_SpecPluginTestCase): 134 | activate = '--with-spec' 135 | args = ['--with-doctest', '--doctest-tests', '--spec-doctests', '--no-spec-color'] 136 | plugins = [Spec(), nose.plugins.doctests.Doctest()] 137 | 138 | suitename = 'doctests' 139 | expected_test_doctests_output = """doctests 140 | - 2 + 3 returns 5 141 | - None is nothing 142 | - foobar throws "NameError: name 'foobar' is not defined" 143 | """ 144 | 145 | def test_builds_specifications_for_doctests(self): 146 | self.assertContainsInOutput(self.expected_test_doctests_output) 147 | 148 | 149 | class TestPluginSpecWithDoctestsButDisabled(_SpecPluginTestCase): 150 | activate = '--with-spec' 151 | 152 | # no --spec-doctests option 153 | args = ['--with-doctest', '--doctest-tests', '--no-spec-color'] 154 | plugins = [Spec(), nose.plugins.doctests.Doctest()] 155 | suitename = 'doctests' 156 | 157 | def test_doesnt_build_specifications_for_doctests_when_spec_doctests_option_wasnt_set(self): 158 | self.failIfContainsInOutput("test_doctests") 159 | self.failIfContainsInOutput("2 + 3 returns 5") 160 | -------------------------------------------------------------------------------- /tox.ini: -------------------------------------------------------------------------------- 1 | # Tox (http://tox.testrun.org/) is a tool for running tests 2 | # in multiple virtualenvs. This configuration file will run the 3 | # test suite on all supported python versions. To use it, "pip install tox" 4 | # and then run "tox" from this directory. 5 | 6 | [tox] 7 | envlist = py25, py26, py27, py30, py31, py32, py33, pypy 8 | 9 | [testenv] 10 | commands = spec --help 11 | -------------------------------------------------------------------------------- /web/after.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bitprophet/spec/d9646c5daf8e479937f970d21ebe185ad936a35a/web/after.png -------------------------------------------------------------------------------- /web/before.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/bitprophet/spec/d9646c5daf8e479937f970d21ebe185ad936a35a/web/before.png -------------------------------------------------------------------------------- /web/readme.py: -------------------------------------------------------------------------------- 1 | from nose.plugins.skip import SkipTest 2 | 3 | 4 | class TestMyClass(object): 5 | def test_has_an_attribute(self): 6 | assert True 7 | 8 | def test_should_perform_some_action(self): 9 | assert False 10 | 11 | def test_can_stand_on_its_head(self): 12 | raise SkipTest 13 | --------------------------------------------------------------------------------