├── .github └── workflows │ └── tests.yml ├── .gitignore ├── LICENSE ├── README.md ├── markdown_pytest.py ├── poetry.lock ├── pyproject.toml └── tests.md /.github/workflows/tests.yml: -------------------------------------------------------------------------------- 1 | name: tests 2 | 3 | 4 | on: 5 | push: 6 | branches: [ master ] 7 | pull_request: 8 | branches: [ master ] 9 | 10 | 11 | jobs: 12 | tests: 13 | runs-on: ubuntu-latest 14 | strategy: 15 | fail-fast: false 16 | 17 | matrix: 18 | python: 19 | - '3.7' 20 | - '3.8' 21 | - '3.9' 22 | - '3.10' 23 | - '3.11' 24 | steps: 25 | - uses: actions/checkout@v2 26 | - name: Setup python${{ matrix.python }} 27 | uses: actions/setup-python@v2 28 | with: 29 | python-version: "${{ matrix.python }}" 30 | - run: python -m pip install poetry 31 | - run: poetry install 32 | - name: pytest 33 | run: poetry run pytest -vv README.md tests.md 34 | env: 35 | FORCE_COLOR: 1 36 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | # Created by .ignore support plugin (hsz.mobi) 2 | ### VirtualEnv template 3 | # Virtualenv 4 | # http://iamzed.com/2009/05/07/a-primer-on-virtualenv/ 5 | .Python 6 | [Bb]in 7 | [Ii]nclude 8 | [Ll]ib 9 | [Ll]ib64 10 | [Ll]ocal 11 | [Ss]cripts 12 | pyvenv.cfg 13 | .venv 14 | pip-selfcheck.json 15 | ### IPythonNotebook template 16 | # Temporary data 17 | .ipynb_checkpoints/ 18 | ### Python template 19 | # Byte-compiled / optimized / DLL files 20 | __pycache__/ 21 | *.py[cod] 22 | *$py.class 23 | 24 | # C extensions 25 | *.so 26 | 27 | # Distribution / packaging 28 | .Python 29 | env/ 30 | build/ 31 | develop-eggs/ 32 | dist/ 33 | downloads/ 34 | eggs/ 35 | .eggs/ 36 | lib/ 37 | lib64/ 38 | parts/ 39 | sdist/ 40 | var/ 41 | *.egg-info/ 42 | .installed.cfg 43 | *.egg 44 | 45 | # PyInstaller 46 | # Usually these files are written by a python script from a template 47 | # before PyInstaller builds the exe, so as to inject date/other infos into it. 48 | *.manifest 49 | *.spec 50 | 51 | # Installer logs 52 | pip-log.txt 53 | pip-delete-this-directory.txt 54 | 55 | # Unit test / coverage reports 56 | htmlcov/ 57 | .tox/ 58 | .coverage 59 | .coverage.* 60 | .cache 61 | nosetests.xml 62 | coverage.xml 63 | *,cover 64 | .hypothesis/ 65 | 66 | # Translations 67 | *.mo 68 | *.pot 69 | 70 | # Django stuff: 71 | *.log 72 | local_settings.py 73 | 74 | # Flask stuff: 75 | instance/ 76 | .webassets-cache 77 | 78 | # Scrapy stuff: 79 | .scrapy 80 | 81 | # Sphinx documentation 82 | docs/_build/ 83 | docs/source/apidoc 84 | 85 | # PyBuilder 86 | target/ 87 | 88 | # IPython Notebook 89 | .ipynb_checkpoints 90 | 91 | # pyenv 92 | .python-version 93 | 94 | # pytest 95 | .pytest_cache 96 | 97 | # celery beat schedule file 98 | celerybeat-schedule 99 | 100 | # dotenv 101 | .env 102 | 103 | # virtualenv 104 | venv/ 105 | ENV/ 106 | 107 | # Spyder project settings 108 | .spyderproject 109 | 110 | # Rope project settings 111 | .ropeproject 112 | ### JetBrains template 113 | # Covers JetBrains IDEs: IntelliJ, RubyMine, PhpStorm, AppCode, PyCharm, CLion, Android Studio and Webstorm 114 | # Reference: https://intellij-support.jetbrains.com/hc/en-us/articles/206544839 115 | 116 | # User-specific stuff: 117 | .idea/ 118 | 119 | ## File-based project format: 120 | *.iws 121 | 122 | ## Plugin-specific files: 123 | 124 | # IntelliJ 125 | /out/ 126 | 127 | # mpeltonen/sbt-idea plugin 128 | .idea_modules/ 129 | 130 | # JIRA plugin 131 | atlassian-ide-plugin.xml 132 | 133 | # Crashlytics plugin (for Android Studio and IntelliJ) 134 | com_crashlytics_export_strings.xml 135 | crashlytics.properties 136 | crashlytics-build.properties 137 | fabric.properties 138 | 139 | /htmlcov 140 | /temp 141 | .DS_Store 142 | 143 | .*cache 144 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | Apache License 2 | Version 2.0, January 2004 3 | http://www.apache.org/licenses/ 4 | 5 | TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION 6 | 7 | 1. Definitions. 8 | 9 | "License" shall mean the terms and conditions for use, reproduction, 10 | and distribution as defined by Sections 1 through 9 of this document. 11 | 12 | "Licensor" shall mean the copyright owner or entity authorized by 13 | the copyright owner that is granting the License. 14 | 15 | "Legal Entity" shall mean the union of the acting entity and all 16 | other entities that control, are controlled by, or are under common 17 | control with that entity. For the purposes of this definition, 18 | "control" means (i) the power, direct or indirect, to cause the 19 | direction or management of such entity, whether by contract or 20 | otherwise, or (ii) ownership of fifty percent (50%) or more of the 21 | outstanding shares, or (iii) beneficial ownership of such entity. 22 | 23 | "You" (or "Your") shall mean an individual or Legal Entity 24 | exercising permissions granted by this License. 25 | 26 | "Source" form shall mean the preferred form for making modifications, 27 | including but not limited to software source code, documentation 28 | source, and configuration files. 29 | 30 | "Object" form shall mean any form resulting from mechanical 31 | transformation or translation of a Source form, including but 32 | not limited to compiled object code, generated documentation, 33 | and conversions to other media types. 34 | 35 | "Work" shall mean the work of authorship, whether in Source or 36 | Object form, made available under the License, as indicated by a 37 | copyright notice that is included in or attached to the work 38 | (an example is provided in the Appendix below). 39 | 40 | "Derivative Works" shall mean any work, whether in Source or Object 41 | form, that is based on (or derived from) the Work and for which the 42 | editorial revisions, annotations, elaborations, or other modifications 43 | represent, as a whole, an original work of authorship. For the purposes 44 | of this License, Derivative Works shall not include works that remain 45 | separable from, or merely link (or bind by name) to the interfaces of, 46 | the Work and Derivative Works thereof. 47 | 48 | "Contribution" shall mean any work of authorship, including 49 | the original version of the Work and any modifications or additions 50 | to that Work or Derivative Works thereof, that is intentionally 51 | submitted to Licensor for inclusion in the Work by the copyright owner 52 | or by an individual or Legal Entity authorized to submit on behalf of 53 | the copyright owner. For the purposes of this definition, "submitted" 54 | means any form of electronic, verbal, or written communication sent 55 | to the Licensor or its representatives, including but not limited to 56 | communication on electronic mailing lists, source code control systems, 57 | and issue tracking systems that are managed by, or on behalf of, the 58 | Licensor for the purpose of discussing and improving the Work, but 59 | excluding communication that is conspicuously marked or otherwise 60 | designated in writing by the copyright owner as "Not a Contribution." 61 | 62 | "Contributor" shall mean Licensor and any individual or Legal Entity 63 | on behalf of whom a Contribution has been received by Licensor and 64 | subsequently incorporated within the Work. 65 | 66 | 2. Grant of Copyright License. Subject to the terms and conditions of 67 | this License, each Contributor hereby grants to You a perpetual, 68 | worldwide, non-exclusive, no-charge, royalty-free, irrevocable 69 | copyright license to reproduce, prepare Derivative Works of, 70 | publicly display, publicly perform, sublicense, and distribute the 71 | Work and such Derivative Works in Source or Object form. 72 | 73 | 3. Grant of Patent License. Subject to the terms and conditions of 74 | this License, each Contributor hereby grants to You a perpetual, 75 | worldwide, non-exclusive, no-charge, royalty-free, irrevocable 76 | (except as stated in this section) patent license to make, have made, 77 | use, offer to sell, sell, import, and otherwise transfer the Work, 78 | where such license applies only to those patent claims licensable 79 | by such Contributor that are necessarily infringed by their 80 | Contribution(s) alone or by combination of their Contribution(s) 81 | with the Work to which such Contribution(s) was submitted. If You 82 | institute patent litigation against any entity (including a 83 | cross-claim or counterclaim in a lawsuit) alleging that the Work 84 | or a Contribution incorporated within the Work constitutes direct 85 | or contributory patent infringement, then any patent licenses 86 | granted to You under this License for that Work shall terminate 87 | as of the date such litigation is filed. 88 | 89 | 4. Redistribution. You may reproduce and distribute copies of the 90 | Work or Derivative Works thereof in any medium, with or without 91 | modifications, and in Source or Object form, provided that You 92 | meet the following conditions: 93 | 94 | (a) You must give any other recipients of the Work or 95 | Derivative Works a copy of this License; and 96 | 97 | (b) You must cause any modified files to carry prominent notices 98 | stating that You changed the files; and 99 | 100 | (c) You must retain, in the Source form of any Derivative Works 101 | that You distribute, all copyright, patent, trademark, and 102 | attribution notices from the Source form of the Work, 103 | excluding those notices that do not pertain to any part of 104 | the Derivative Works; and 105 | 106 | (d) If the Work includes a "NOTICE" text file as part of its 107 | distribution, then any Derivative Works that You distribute must 108 | include a readable copy of the attribution notices contained 109 | within such NOTICE file, excluding those notices that do not 110 | pertain to any part of the Derivative Works, in at least one 111 | of the following places: within a NOTICE text file distributed 112 | as part of the Derivative Works; within the Source form or 113 | documentation, if provided along with the Derivative Works; or, 114 | within a display generated by the Derivative Works, if and 115 | wherever such third-party notices normally appear. The contents 116 | of the NOTICE file are for informational purposes only and 117 | do not modify the License. You may add Your own attribution 118 | notices within Derivative Works that You distribute, alongside 119 | or as an addendum to the NOTICE text from the Work, provided 120 | that such additional attribution notices cannot be construed 121 | as modifying the License. 122 | 123 | You may add Your own copyright statement to Your modifications and 124 | may provide additional or different license terms and conditions 125 | for use, reproduction, or distribution of Your modifications, or 126 | for any such Derivative Works as a whole, provided Your use, 127 | reproduction, and distribution of the Work otherwise complies with 128 | the conditions stated in this License. 129 | 130 | 5. Submission of Contributions. Unless You explicitly state otherwise, 131 | any Contribution intentionally submitted for inclusion in the Work 132 | by You to the Licensor shall be under the terms and conditions of 133 | this License, without any additional terms or conditions. 134 | Notwithstanding the above, nothing herein shall supersede or modify 135 | the terms of any separate license agreement you may have executed 136 | with Licensor regarding such Contributions. 137 | 138 | 6. Trademarks. This License does not grant permission to use the trade 139 | names, trademarks, service marks, or product names of the Licensor, 140 | except as required for reasonable and customary use in describing the 141 | origin of the Work and reproducing the content of the NOTICE file. 142 | 143 | 7. Disclaimer of Warranty. Unless required by applicable law or 144 | agreed to in writing, Licensor provides the Work (and each 145 | Contributor provides its Contributions) on an "AS IS" BASIS, 146 | WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or 147 | implied, including, without limitation, any warranties or conditions 148 | of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A 149 | PARTICULAR PURPOSE. You are solely responsible for determining the 150 | appropriateness of using or redistributing the Work and assume any 151 | risks associated with Your exercise of permissions under this License. 152 | 153 | 8. Limitation of Liability. In no event and under no legal theory, 154 | whether in tort (including negligence), contract, or otherwise, 155 | unless required by applicable law (such as deliberate and grossly 156 | negligent acts) or agreed to in writing, shall any Contributor be 157 | liable to You for damages, including any direct, indirect, special, 158 | incidental, or consequential damages of any character arising as a 159 | result of this License or out of the use or inability to use the 160 | Work (including but not limited to damages for loss of goodwill, 161 | work stoppage, computer failure or malfunction, or any and all 162 | other commercial damages or losses), even if such Contributor 163 | has been advised of the possibility of such damages. 164 | 165 | 9. Accepting Warranty or Additional Liability. While redistributing 166 | the Work or Derivative Works thereof, You may choose to offer, 167 | and charge a fee for, acceptance of support, warranty, indemnity, 168 | or other liability obligations and/or rights consistent with this 169 | License. However, in accepting such obligations, You may act only 170 | on Your own behalf and on Your sole responsibility, not on behalf 171 | of any other Contributor, and only if You agree to indemnify, 172 | defend, and hold each Contributor harmless for any liability 173 | incurred by, or claims asserted against, such Contributor by reason 174 | of your accepting any such warranty or additional liability. 175 | 176 | END OF TERMS AND CONDITIONS 177 | 178 | APPENDIX: How to apply the Apache License to your work. 179 | 180 | To apply the Apache License to your work, attach the following 181 | boilerplate notice, with the fields enclosed by brackets "[]" 182 | replaced with your own identifying information. (Don't include 183 | the brackets!) The text should be enclosed in the appropriate 184 | comment syntax for the file format. We also recommend that a 185 | file or class name and description of purpose be included on the 186 | same "printed page" as the copyright notice for easier 187 | identification within third-party archives. 188 | 189 | Copyright [yyyy] [name of copyright owner] 190 | 191 | Licensed under the Apache License, Version 2.0 (the "License"); 192 | you may not use this file except in compliance with the License. 193 | You may obtain a copy of the License at 194 | 195 | http://www.apache.org/licenses/LICENSE-2.0 196 | 197 | Unless required by applicable law or agreed to in writing, software 198 | distributed under the License is distributed on an "AS IS" BASIS, 199 | WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 200 | See the License for the specific language governing permissions and 201 | limitations under the License. 202 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | markdown-pytest 2 | =============== 3 | 4 | The `markdown-pytest` plugin is a `pytest` plugin that allows you to run tests 5 | directly from Markdown files. 6 | 7 | With this plugin, you can write your tests inside Markdown files, making it 8 | easy to read, understand and maintain your documentation samples. 9 | The tests are executed just like any other Pytest tests. 10 | 11 | Sample of markdown file content: 12 | 13 | ````markdown 14 | 15 | ```python 16 | assert True 17 | ``` 18 | ```` 19 | 20 |
21 | Will be shown as 22 | 23 | 24 | ```python 25 | assert True 26 | ``` 27 | 28 |
29 | 30 | Restrictions 31 | ------------ 32 | 33 | Since there is no way to add attributes to a block of code in markdown, this 34 | module only runs those tests that are marked with a special comment. 35 | 36 | The general format of this comment is as follows: parts separated by semicolons 37 | are a colon separated key-value pairs, the last semicolon is optional, 38 | and parts not containing a colon bill be ignored. 39 | 40 | Example: 41 | 42 | ```markdown 43 | 44 | ``` 45 | 46 | Multiline example: 47 | 48 | ```markdown 49 | 53 | ``` 54 | 55 | This comment should be placed right before the block of code, exactly upper 56 | the backticks, for example: 57 | 58 | ```` 59 | 60 | ```python 61 | ``` 62 | ```` 63 | 64 | The `name` key is required, and blocks that do not contain it will be ignored. 65 | 66 | Some Markdown parsers support two or three dashes around comments, this module 67 | supports both variants. The `case` parameter is optional and might be used for 68 | subtests, see "Code split" section. 69 | 70 | Additionally, a code block can be put inside the comment block to hide some 71 | initialization from the readers. 72 | 73 | ````markdown 74 | 79 | ```python 80 | assert init_some_variable == 123 81 | ``` 82 | ```` 83 | 84 | Common parsing rules 85 | -------------------- 86 | 87 | This module uses its own, very simple Markdown parser, which only supports code 88 | block parsing. In general, the parsing behavior of a file follows the following 89 | rules: 90 | 91 | * Code without `` comment will not be executed. 92 | * Allowed two or three dashes in the comment symbols 93 | 94 | For example following line will be parsed identically: 95 | 96 | ````markdown 97 | 98 | 99 | 100 | 101 | ```` 102 | 103 | * Code blocks with same names will be merged in one code and executed once 104 | * The optional comment parameter `case` will execute the block as a subtest. 105 | * Indented code blocks will be shifted left. 106 | 107 | For example: 108 | 109 | ````markdown 110 | 111 | ```python 112 | assert True 113 | ``` 114 | ```` 115 | 116 | Is the same of: 117 | 118 | ````markdown 119 | 120 | ```python 121 | assert True 122 | ``` 123 | ```` 124 | 125 | Code split 126 | ---------- 127 | 128 | You can split a test into multiple blocks with the same test name: 129 | 130 | Markdown: 131 | 132 | ````markdown 133 | This block performs import: 134 | 135 | 136 | ```python 137 | from itertools import chain 138 | ``` 139 | 140 | `chain` usage example: 141 | 142 | 143 | ```python 144 | assert list(chain(range(2), range(2))) == [0, 1, 0, 1] 145 | ``` 146 | ```` 147 | 148 |
149 | Will be shown as 150 | 151 | This block performs import: 152 | 153 | 154 | ```python 155 | from itertools import chain 156 | ``` 157 | 158 | `chain` usage example: 159 | 160 | 161 | ```python 162 | assert list(chain(range(2), range(2))) == [0, 1, 0, 1] 163 | ``` 164 | 165 |
166 | 167 | subtests support 168 | ---------------- 169 | 170 | Of course, you can break tests into subtests by simply adding `case: case_name` 171 | to the markdown comment. 172 | 173 | ````markdown 174 | 175 | ```python 176 | from collections import Counter 177 | ``` 178 | 179 | 183 | ```python 184 | counter = Counter() 185 | ``` 186 | 187 | 191 | ```python 192 | counter["foo"] += 1 193 | 194 | assert counter["foo"] == 1 195 | ``` 196 | ```` 197 | 198 |
199 | Will be shown as 200 | 201 | 202 | ```python 203 | from collections import Counter 204 | ``` 205 | 206 | 210 | ```python 211 | counter = Counter() 212 | ``` 213 | 214 | 218 | ```python 219 | counter["foo"] += 1 220 | 221 | assert counter["foo"] == 1 222 | ``` 223 | 224 |
225 | 226 | Fictional Code Examples 227 | ----------------------- 228 | 229 | Code without `` comment will not be executed. 230 | 231 | ````markdown 232 | ```python 233 | from universe import antigravity, WrongPlanet 234 | 235 | try: 236 | antigravity() 237 | except WrongPlanet: 238 | print("You are on the wrong planet.") 239 | exit(1) 240 | ``` 241 | ```` 242 | 243 |
244 | Will be shown as 245 | 246 | ```python 247 | from universe import antigravity, WrongPlanet 248 | 249 | try: 250 | antigravity() 251 | except WrongPlanet: 252 | print("You are on the wrong planet.") 253 | exit(1) 254 | ``` 255 |
256 | 257 | Usage example 258 | ------------- 259 | 260 | This README.md file can be tested like this: 261 | 262 | ```bash 263 | $ pytest -v README.md 264 | ``` 265 | ```bash 266 | ======================= test session starts ======================= 267 | plugins: subtests, markdown-pytest 268 | collected 3 items 269 | 270 | README.md::test_assert_true PASSED [ 33%] 271 | README.md::test_example PASSED [ 66%] 272 | README.md::test_counter SUBPASS [100%] 273 | README.md::test_counter SUBPASS [100%] 274 | README.md::test_counter PASSED [100%] 275 | ``` 276 | -------------------------------------------------------------------------------- /markdown_pytest.py: -------------------------------------------------------------------------------- 1 | from functools import partial 2 | from itertools import groupby 3 | from pathlib import Path 4 | from types import CodeType 5 | from typing import Dict, Iterable, Iterator, NamedTuple, Optional, TextIO, Tuple 6 | 7 | import pytest 8 | 9 | 10 | class CodeBlock(NamedTuple): 11 | start_line: int 12 | lines: Tuple[str, ...] 13 | arguments: Tuple[Tuple[str, str], ...] 14 | path: str 15 | name: str 16 | 17 | @property 18 | def end_line(self) -> int: 19 | return self.start_line + len(self.lines) 20 | 21 | 22 | COMMENT_BRACKETS = ("") 23 | 24 | 25 | LineType = Tuple[int, str] 26 | 27 | 28 | class LinesIterator: 29 | lines: Tuple[LineType, ...] 30 | 31 | def __init__(self, lines: Iterable[str]): 32 | self.lines = tuple( 33 | map(tuple, enumerate(line.rstrip() for line in lines)), 34 | ) 35 | self.index = 0 36 | 37 | @classmethod 38 | def from_fp(cls, fp: TextIO) -> "LinesIterator": 39 | return cls(fp.readlines()) 40 | 41 | @classmethod 42 | def from_file(cls, filename) -> "LinesIterator": 43 | with open(filename, "r") as fp: 44 | return cls.from_fp(fp) 45 | 46 | def get_relative(self, index: int) -> LineType: 47 | return self.lines[self.index + index] 48 | 49 | def is_last_line(self) -> bool: 50 | return self.index >= len(self.lines) 51 | 52 | def next(self) -> LineType: 53 | lineno, line = self.get_relative(0) 54 | self.index += 1 55 | return lineno, line 56 | 57 | def seek_relative(self, index: int) -> None: 58 | self.index += index 59 | 60 | def reverse_iterator(self, start_from: int = 0): 61 | for i in range(start_from, self.index): 62 | yield self.get_relative(-i - 1) 63 | 64 | def __iter__(self): 65 | return self 66 | 67 | def __next__(self): 68 | try: 69 | return self.next() 70 | except IndexError: 71 | raise StopIteration 72 | 73 | 74 | def parse_arguments(line_iterator: LinesIterator) -> Dict[str, str]: 75 | 76 | outside_comment = inside_comment = False 77 | index = line_iterator.index 78 | # Checking if the code block is outside of the comment block 79 | for lineno, line in line_iterator.reverse_iterator(1): 80 | if not line.strip(): 81 | continue 82 | if line.strip().endswith(COMMENT_BRACKETS[1]): 83 | outside_comment = True 84 | break 85 | 86 | # Checking if the code block is inside of the comment block 87 | if not outside_comment: 88 | for lineno, line in line_iterator: 89 | if not line.strip(): 90 | continue 91 | if line.strip().endswith(COMMENT_BRACKETS[0]): 92 | return {} 93 | elif line.strip().endswith(COMMENT_BRACKETS[1]): 94 | inside_comment = True 95 | line_iterator.seek_relative(1) 96 | break 97 | 98 | if not outside_comment and not inside_comment: 99 | return {} 100 | 101 | lines = [] 102 | reverse_iterator = line_iterator.reverse_iterator(1) 103 | for lineno, line in reverse_iterator: 104 | if line.strip().startswith("```"): 105 | for _, line in reverse_iterator: 106 | if line.strip().startswith("```"): 107 | break 108 | continue 109 | lines.append(line) 110 | if line.strip().startswith(COMMENT_BRACKETS[0]): 111 | break 112 | 113 | # Restore the iterator (due to inside comment forward iterations) 114 | line_iterator.index = index 115 | 116 | if not lines: 117 | return {} 118 | 119 | lines = lines[::-1] 120 | result = {} 121 | args = "".join( 122 | "".join(lines).strip()[ 123 | len(COMMENT_BRACKETS[0]):-len(COMMENT_BRACKETS[1]) + 1 124 | ].strip("-").strip().splitlines(), 125 | ).split(";") 126 | 127 | for arg in args: 128 | if ":" not in arg: 129 | continue 130 | 131 | key, value = arg.split(":", 1) 132 | result[key.strip()] = value.strip() 133 | 134 | return result 135 | 136 | 137 | def parse_code_blocks(fspath) -> Iterator[CodeBlock]: 138 | line_iterator = LinesIterator.from_file(fspath) 139 | 140 | for lineno, line in line_iterator: 141 | if ( 142 | line.rstrip().endswith("```") and 143 | line.lstrip().startswith("```") 144 | ): 145 | # skip all blocks without '```python` 146 | end_of_block = "`" * line.count("`") 147 | try: 148 | lineno, line = line_iterator.next() 149 | except IndexError: 150 | return 151 | 152 | for lineno, line in line_iterator: 153 | if line.rstrip() == end_of_block: 154 | break 155 | 156 | if not line.endswith("```python"): 157 | continue 158 | 159 | indent = line.rstrip().count(" ") 160 | end_of_block = (" " * indent) + ("`" * line.count("`")) 161 | 162 | arguments = parse_arguments(line_iterator) 163 | 164 | # the next line after ```python 165 | start_lineno = lineno + 1 166 | code_lines = [] 167 | 168 | for lineno, line in line_iterator: 169 | if line.startswith(end_of_block): 170 | break 171 | code_lines.append(line[indent:]) 172 | 173 | if not arguments or "name" not in arguments: 174 | continue 175 | 176 | case = arguments.get("case") 177 | if case is not None: 178 | start_lineno -= 1 179 | # indent test case lines 180 | code_lines = [f" {code_line}" for code_line in code_lines] 181 | code_lines.insert( 182 | 0, "with __markdown_pytest_subtests_fixture.test(" 183 | f"msg='{case} line={start_lineno}'):", 184 | ) 185 | 186 | block = CodeBlock( 187 | start_line=start_lineno, 188 | lines=tuple(code_lines), 189 | arguments=tuple(arguments.items()), 190 | path=str(fspath), 191 | name=arguments.pop("name"), 192 | ) 193 | 194 | yield block 195 | 196 | 197 | def compile_code_blocks(*blocks: CodeBlock) -> Optional[CodeType]: 198 | blocks = sorted(blocks, key=lambda x: x.start_line) 199 | if not blocks: 200 | return None 201 | lines = [""] * blocks[-1].end_line 202 | path = blocks[0].path 203 | for block in blocks: 204 | lines[block.start_line:block.end_line] = block.lines 205 | return compile(source="\n".join(lines), mode="exec", filename=path) 206 | 207 | 208 | class MDModule(pytest.Module): 209 | 210 | @staticmethod 211 | def caller(code, subtests): 212 | eval(code, dict(__markdown_pytest_subtests_fixture=subtests)) 213 | 214 | def collect(self) -> Iterable[pytest.Function]: 215 | test_prefix = self.config.getoption("--md-prefix") 216 | 217 | for test_name, blocks in groupby( 218 | parse_code_blocks(self.fspath), 219 | key=lambda x: x.name, 220 | ): 221 | if not test_name.startswith(test_prefix): 222 | continue 223 | 224 | blocks = list(blocks) 225 | code = compile_code_blocks(*blocks) 226 | if code is None: 227 | continue 228 | 229 | yield pytest.Function.from_parent( 230 | name=test_name, 231 | parent=self, 232 | callobj=partial(self.caller, code), 233 | ) 234 | 235 | 236 | def pytest_addoption(parser): 237 | parser.addoption( 238 | "--md-prefix", default="test", 239 | help="Markdown test code-block prefix from comment", 240 | ) 241 | 242 | 243 | @pytest.hookimpl(trylast=True) 244 | def pytest_collect_file(path, parent: pytest.Collector) -> Optional[MDModule]: 245 | if path.ext.lower() not in (".md", ".markdown"): 246 | return None 247 | return MDModule.from_parent(parent=parent, path=Path(path)) 248 | -------------------------------------------------------------------------------- /poetry.lock: -------------------------------------------------------------------------------- 1 | # This file is automatically @generated by Poetry 1.8.2 and should not be changed by hand. 2 | 3 | [[package]] 4 | name = "attrs" 5 | version = "23.2.0" 6 | description = "Classes Without Boilerplate" 7 | optional = false 8 | python-versions = ">=3.7" 9 | files = [ 10 | {file = "attrs-23.2.0-py3-none-any.whl", hash = "sha256:99b87a485a5820b23b879f04c2305b44b951b502fd64be915879d77a7e8fc6f1"}, 11 | {file = "attrs-23.2.0.tar.gz", hash = "sha256:935dc3b529c262f6cf76e50877d35a4bd3c1de194fd41f47a2b7ae8f19971f30"}, 12 | ] 13 | 14 | [package.extras] 15 | cov = ["attrs[tests]", "coverage[toml] (>=5.3)"] 16 | dev = ["attrs[tests]", "pre-commit"] 17 | docs = ["furo", "myst-parser", "sphinx", "sphinx-notfound-page", "sphinxcontrib-towncrier", "towncrier", "zope-interface"] 18 | tests = ["attrs[tests-no-zope]", "zope-interface"] 19 | tests-mypy = ["mypy (>=1.6)", "pytest-mypy-plugins"] 20 | tests-no-zope = ["attrs[tests-mypy]", "cloudpickle", "hypothesis", "pympler", "pytest (>=4.3.0)", "pytest-xdist[psutil]"] 21 | 22 | [[package]] 23 | name = "colorama" 24 | version = "0.4.6" 25 | description = "Cross-platform colored terminal text." 26 | optional = false 27 | python-versions = "!=3.0.*,!=3.1.*,!=3.2.*,!=3.3.*,!=3.4.*,!=3.5.*,!=3.6.*,>=2.7" 28 | files = [ 29 | {file = "colorama-0.4.6-py2.py3-none-any.whl", hash = "sha256:4f1d9991f5acc0ca119f9d443620b77f9d6b33703e51011c16baf57afb285fc6"}, 30 | {file = "colorama-0.4.6.tar.gz", hash = "sha256:08695f5cb7ed6e0531a20572697297273c47b8cae5a63ffc6d6ed5c201be6e44"}, 31 | ] 32 | 33 | [[package]] 34 | name = "exceptiongroup" 35 | version = "1.2.0" 36 | description = "Backport of PEP 654 (exception groups)" 37 | optional = false 38 | python-versions = ">=3.7" 39 | files = [ 40 | {file = "exceptiongroup-1.2.0-py3-none-any.whl", hash = "sha256:4bfd3996ac73b41e9b9628b04e079f193850720ea5945fc96a08633c66912f14"}, 41 | {file = "exceptiongroup-1.2.0.tar.gz", hash = "sha256:91f5c769735f051a4290d52edd0858999b57e5876e9f85937691bd4c9fa3ed68"}, 42 | ] 43 | 44 | [package.extras] 45 | test = ["pytest (>=6)"] 46 | 47 | [[package]] 48 | name = "iniconfig" 49 | version = "2.0.0" 50 | description = "brain-dead simple config-ini parsing" 51 | optional = false 52 | python-versions = ">=3.7" 53 | files = [ 54 | {file = "iniconfig-2.0.0-py3-none-any.whl", hash = "sha256:b6a85871a79d2e3b22d2d1b94ac2824226a63c6b741c88f7ae975f18b6778374"}, 55 | {file = "iniconfig-2.0.0.tar.gz", hash = "sha256:2d91e135bf72d31a410b17c16da610a82cb55f6b0477d1a902134b24a455b8b3"}, 56 | ] 57 | 58 | [[package]] 59 | name = "packaging" 60 | version = "24.0" 61 | description = "Core utilities for Python packages" 62 | optional = false 63 | python-versions = ">=3.7" 64 | files = [ 65 | {file = "packaging-24.0-py3-none-any.whl", hash = "sha256:2ddfb553fdf02fb784c234c7ba6ccc288296ceabec964ad2eae3777778130bc5"}, 66 | {file = "packaging-24.0.tar.gz", hash = "sha256:eb82c5e3e56209074766e6885bb04b8c38a0c015d0a30036ebe7ece34c9989e9"}, 67 | ] 68 | 69 | [[package]] 70 | name = "pluggy" 71 | version = "1.4.0" 72 | description = "plugin and hook calling mechanisms for python" 73 | optional = false 74 | python-versions = ">=3.8" 75 | files = [ 76 | {file = "pluggy-1.4.0-py3-none-any.whl", hash = "sha256:7db9f7b503d67d1c5b95f59773ebb58a8c1c288129a88665838012cfb07b8981"}, 77 | {file = "pluggy-1.4.0.tar.gz", hash = "sha256:8c85c2876142a764e5b7548e7d9a0e0ddb46f5185161049a79b7e974454223be"}, 78 | ] 79 | 80 | [package.extras] 81 | dev = ["pre-commit", "tox"] 82 | testing = ["pytest", "pytest-benchmark"] 83 | 84 | [[package]] 85 | name = "pytest" 86 | version = "8.1.1" 87 | description = "pytest: simple powerful testing with Python" 88 | optional = false 89 | python-versions = ">=3.8" 90 | files = [ 91 | {file = "pytest-8.1.1-py3-none-any.whl", hash = "sha256:2a8386cfc11fa9d2c50ee7b2a57e7d898ef90470a7a34c4b949ff59662bb78b7"}, 92 | {file = "pytest-8.1.1.tar.gz", hash = "sha256:ac978141a75948948817d360297b7aae0fcb9d6ff6bc9ec6d514b85d5a65c044"}, 93 | ] 94 | 95 | [package.dependencies] 96 | colorama = {version = "*", markers = "sys_platform == \"win32\""} 97 | exceptiongroup = {version = ">=1.0.0rc8", markers = "python_version < \"3.11\""} 98 | iniconfig = "*" 99 | packaging = "*" 100 | pluggy = ">=1.4,<2.0" 101 | tomli = {version = ">=1", markers = "python_version < \"3.11\""} 102 | 103 | [package.extras] 104 | testing = ["argcomplete", "attrs (>=19.2)", "hypothesis (>=3.56)", "mock", "pygments (>=2.7.2)", "requests", "setuptools", "xmlschema"] 105 | 106 | [[package]] 107 | name = "pytest-subtests" 108 | version = "0.12.1" 109 | description = "unittest subTest() support and subtests fixture" 110 | optional = false 111 | python-versions = ">=3.7" 112 | files = [ 113 | {file = "pytest-subtests-0.12.1.tar.gz", hash = "sha256:d6605dcb88647e0b7c1889d027f8ef1c17d7a2c60927ebfdc09c7b0d8120476d"}, 114 | {file = "pytest_subtests-0.12.1-py3-none-any.whl", hash = "sha256:100d9f7eb966fc98efba7026c802812ae327e8b5b37181fb260a2ea93226495c"}, 115 | ] 116 | 117 | [package.dependencies] 118 | attrs = ">=19.2.0" 119 | pytest = ">=7.0" 120 | 121 | [[package]] 122 | name = "tomli" 123 | version = "2.0.1" 124 | description = "A lil' TOML parser" 125 | optional = false 126 | python-versions = ">=3.7" 127 | files = [ 128 | {file = "tomli-2.0.1-py3-none-any.whl", hash = "sha256:939de3e7a6161af0c887ef91b7d41a53e7c5a1ca976325f429cb46ea9bc30ecc"}, 129 | {file = "tomli-2.0.1.tar.gz", hash = "sha256:de526c12914f0c550d15924c62d72abc48d6fe7364aa87328337a31007fe8a4f"}, 130 | ] 131 | 132 | [metadata] 133 | lock-version = "2.0" 134 | python-versions = "^3.8" 135 | content-hash = "3e10720190c010b917488a2ea55915cb2530d94d77eabc762ed4fe400df210ce" 136 | -------------------------------------------------------------------------------- /pyproject.toml: -------------------------------------------------------------------------------- 1 | [tool.poetry] 2 | name = "markdown-pytest" 3 | version = "0.3.2" 4 | license = "Apache-2.0" 5 | description = "Pytest plugin for runs tests directly from Markdown files" 6 | authors = ["Dmitry Orlov "] 7 | readme = "README.md" 8 | keywords=["pytest", "markdown", "documentation"] 9 | homepage = "https://github.com/mosquito/markdown-pytest" 10 | classifiers = [ 11 | "Development Status :: 4 - Beta", 12 | "Environment :: Plugins", 13 | "Framework :: Pytest", 14 | "Intended Audience :: Developers", 15 | "License :: OSI Approved :: Apache Software License", 16 | "Natural Language :: English", 17 | "Operating System :: MacOS", 18 | "Operating System :: Microsoft", 19 | "Operating System :: POSIX", 20 | "Programming Language :: Python :: 3 :: Only", 21 | "Programming Language :: Python :: 3", 22 | "Programming Language :: Python :: 3.10", 23 | "Programming Language :: Python :: 3.11", 24 | "Programming Language :: Python :: 3.8", 25 | "Programming Language :: Python :: 3.9", 26 | "Programming Language :: Python :: Implementation :: CPython", 27 | "Programming Language :: Python :: Implementation :: PyPy", 28 | "Programming Language :: Python", 29 | "Topic :: Software Development :: Libraries", 30 | "Topic :: Software Development", 31 | ] 32 | packages = [{include = "markdown_pytest.py"}] 33 | 34 | [tool.poetry.dependencies] 35 | python = "^3.8" 36 | pytest-subtests = ">=0.12.0" 37 | 38 | [tool.poetry.group.dev.dependencies] 39 | pytest = ">=8.0" 40 | 41 | [build-system] 42 | requires = ["poetry-core"] 43 | build-backend = "poetry.core.masonry.api" 44 | 45 | [tool.poetry.plugins.pytest11] 46 | markdown-pytest = "markdown_pytest" 47 | 48 | [tool.poetry.urls] 49 | "Source" = "https://github.com/mosquito/markdown-pytest" 50 | "Tracker" = "https://github.com/mosquito/markdown-pytest/issues" 51 | "Documentation" = "https://github.com/mosquito/markdown-pytest/blob/master/README.rst" 52 | -------------------------------------------------------------------------------- /tests.md: -------------------------------------------------------------------------------- 1 | 2 | ```python 3 | # test_assert_true 4 | assert True 5 | ``` 6 | 7 | 10 | ```python 11 | assert True 12 | ``` 13 | 14 | 17 | ```python 18 | assert True 19 | ``` 20 | 21 | 27 | ```python 28 | assert hidden_value == 123 29 | ``` 30 | 31 | 32 | ```python 33 | from collections import Counter 34 | ``` 35 | 36 | 39 | ```python 40 | counter = Counter() 41 | ``` 42 | 43 | 44 | 47 | ```python 48 | counter["foo"] = 1 49 | ``` 50 | 51 | 52 | 53 | ````python 54 | # test_four_backticks 55 | assert True 56 | ```` 57 | 58 | Split test 59 | ---------- 60 | 61 | Part one 62 | 63 | 64 | ```python 65 | from collections import deque 66 | 67 | queue = deque(maxlen=2) 68 | ``` 69 | 70 | Part Two 71 | 72 | 73 | ```python 74 | queue.append(1) 75 | 76 | assert list(queue) == [1] 77 | ``` 78 | 79 | Part Three 80 | 81 | 82 | ```python 83 | queue.append(2) 84 | 85 | assert list(queue) == [1, 2] 86 | ``` 87 | 88 | Part Four 89 | 90 | 91 | `````python 92 | queue.append(3) 93 | assert list(queue) == [2, 3] 94 | ````` 95 | 96 | 97 | ```````````````python 98 | # test nervous backtick 99 | assert True 100 | ``````````````` 101 | 102 | 103 | 104 | ```python 105 | from pytest import xfail 106 | 107 | xfail("it's ok") 108 | ``` 109 | 110 | 111 | 112 | ```python 113 | from pytest import raises 114 | 115 | with raises(AssertionError): 116 | assert False 117 | ``` 118 | 119 | 120 | ```python 121 | assert True 122 | ``` 123 | 124 | 125 | ```python 126 | assert True 127 | ``` 128 | 129 | 130 | ```python 131 | assert True 132 | ``` 133 | 134 | 135 | ```python 136 | assert True 137 | ``` 138 | 139 | 140 | 141 | ```python 142 | assert True 143 | ``` 144 | 145 | ```` 146 | 147 | ```python 148 | assert False 149 | ``` 150 | ```` 151 | 152 | 153 |
154 | Indented code block 155 | 156 | 157 | ```python 158 | assert True 159 | ``` 160 | 161 |
162 | 163 | 164 | ```python 165 | def mul(*args): 166 | result = args[0] 167 | for i in args[1:]: 168 | result *= i 169 | return result 170 | ``` 171 | 172 | 173 | ```python 174 | assert mul(0) == 0 175 | ``` 176 | 177 | 178 | ```python 179 | assert mul(1, 2) == 2 180 | ``` 181 | 182 | 183 | ```python 184 | assert mul(*range(1, 10)) == 362880 185 | ``` 186 | --------------------------------------------------------------------------------