├── metacode ├── py.typed ├── errors.py ├── comment.py ├── __init__.py ├── typing.py ├── building.py └── parsing.py ├── tests ├── __init__.py ├── test_building.py └── test_parsing.py ├── docs └── assets │ ├── logo_2.png │ └── logo_3.svg ├── requirements_dev.txt ├── .gitignore ├── .github ├── ISSUE_TEMPLATE │ ├── question.md │ ├── feature_request.md │ ├── documentation.md │ └── bug_report.md └── workflows │ ├── release.yml │ ├── lint.yml │ └── tests_and_coverage.yml ├── LICENSE ├── pyproject.toml └── README.md /metacode/py.typed: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /tests/__init__.py: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /metacode/errors.py: -------------------------------------------------------------------------------- 1 | class UnknownArgumentTypeError(TypeError): 2 | ... 3 | -------------------------------------------------------------------------------- /docs/assets/logo_2.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/pomponchik/metacode/HEAD/docs/assets/logo_2.png -------------------------------------------------------------------------------- /requirements_dev.txt: -------------------------------------------------------------------------------- 1 | pytest==8.0.2 2 | coverage==7.6.1 3 | build==1.2.2.post1 4 | twine==6.1.0 5 | mypy==1.14.1 6 | pytest-mypy-testing==0.1.3 7 | ruff==0.14.6 8 | mutmut==3.2.3 9 | full_match==0.0.3 10 | -------------------------------------------------------------------------------- /metacode/comment.py: -------------------------------------------------------------------------------- 1 | from dataclasses import dataclass 2 | 3 | from metacode.typing import Arguments 4 | 5 | 6 | @dataclass 7 | class ParsedComment: 8 | key: str 9 | command: str 10 | arguments: Arguments 11 | -------------------------------------------------------------------------------- /metacode/__init__.py: -------------------------------------------------------------------------------- 1 | from metacode.building import build as build 2 | from metacode.building import insert as insert 3 | from metacode.comment import ParsedComment as ParsedComment 4 | from metacode.errors import UnknownArgumentTypeError as UnknownArgumentTypeError 5 | from metacode.parsing import parse as parse 6 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | __pycache__ 2 | .pytest_cache 3 | .DS_Store 4 | test.py 5 | test_2.py 6 | test_3.py 7 | test_4.py 8 | *.egg-info 9 | dist 10 | venv 11 | .venv 12 | build 13 | .ruff_cache 14 | .mypy_cache 15 | .coverage 16 | uv.lock 17 | .history 18 | .vscode/ 19 | .idea/ 20 | .ropeproject 21 | node_modules 22 | mutants 23 | -------------------------------------------------------------------------------- /.github/ISSUE_TEMPLATE/question.md: -------------------------------------------------------------------------------- 1 | --- 2 | name: Question or consultation 3 | about: Ask anything about this project 4 | title: '' 5 | labels: guestion 6 | assignees: pomponchik 7 | 8 | --- 9 | 10 | ## Your question 11 | 12 | Here you can freely describe your question about the project. Please, before doing this, read the documentation provided, and ask the question only if the necessary answer is not there. In addition, please keep in mind that this is a free non-commercial project and user support is optional for its author. The response time is not guaranteed in any way. 13 | -------------------------------------------------------------------------------- /.github/ISSUE_TEMPLATE/feature_request.md: -------------------------------------------------------------------------------- 1 | --- 2 | name: Feature request 3 | about: Suggest an idea for this project 4 | title: '' 5 | labels: enhancement 6 | assignees: pomponchik 7 | 8 | --- 9 | 10 | ## Short description 11 | 12 | What do you propose and why do you consider it important? 13 | 14 | 15 | ## Some details 16 | 17 | If you can, provide code examples that will show how your proposal will work. Also, if you can, indicate which alternatives to this behavior you have considered. And finally, how do you propose to test the correctness of the implementation of your idea, if at all possible? 18 | -------------------------------------------------------------------------------- /metacode/typing.py: -------------------------------------------------------------------------------- 1 | from ast import AST 2 | from typing import List, Optional, Union 3 | 4 | # TODO: delete this catch blocks and "type: ignore" if minimum supported version of Python is > 3.9. 5 | try: 6 | from typing import TypeAlias # type: ignore[attr-defined, unused-ignore] 7 | except ImportError: # pragma: no cover 8 | from typing_extensions import TypeAlias 9 | 10 | try: 11 | from types import EllipsisType # type: ignore[attr-defined, unused-ignore] 12 | except ImportError: # pragma: no cover 13 | EllipsisType = type(...) # type: ignore[misc, unused-ignore] 14 | 15 | 16 | Argument: TypeAlias = Union[str, int, float, complex, bool, EllipsisType, AST] 17 | Arguments: TypeAlias = List[Optional[Argument]] 18 | -------------------------------------------------------------------------------- /.github/ISSUE_TEMPLATE/documentation.md: -------------------------------------------------------------------------------- 1 | --- 2 | name: Documentation fix 3 | about: Add something to the documentation, delete it, or change it 4 | title: '' 5 | labels: documentation 6 | assignees: pomponchik 7 | --- 8 | 9 | ## It's cool that you're here! 10 | 11 | Documentation is an important part of the project, we strive to make it high-quality and keep it up to date. Please adjust this template by outlining your proposal. 12 | 13 | 14 | ## Type of action 15 | 16 | What do you want to do: remove something, add it, or change it? 17 | 18 | 19 | ## Where? 20 | 21 | Specify which part of the documentation you want to make a change to? For example, the name of an existing documentation section or the line number in a file `README.md`. 22 | 23 | 24 | ## The essence 25 | 26 | Please describe the essence of the proposed change 27 | -------------------------------------------------------------------------------- /.github/ISSUE_TEMPLATE/bug_report.md: -------------------------------------------------------------------------------- 1 | --- 2 | name: Bug report 3 | about: Create a report to help us improve 4 | title: '' 5 | labels: bug 6 | assignees: pomponchik 7 | 8 | --- 9 | 10 | ## Short description 11 | 12 | Replace this text with a short description of the error and the behavior that you expected to see instead. 13 | 14 | 15 | ## Describe the bug in detail 16 | 17 | Please add this test in such a way that it reproduces the bug you found and does not pass: 18 | 19 | ```python 20 | def test_your_bug(): 21 | ... 22 | ``` 23 | 24 | Writing the test, please keep compatibility with the [`pytest`](https://docs.pytest.org/) framework. 25 | 26 | If for some reason you cannot describe the error in the test format, describe here the steps to reproduce it. 27 | 28 | 29 | ## Environment 30 | - OS: ... 31 | - Python version (the output of the `python --version` command): ... 32 | - Version of this package: ... 33 | -------------------------------------------------------------------------------- /.github/workflows/release.yml: -------------------------------------------------------------------------------- 1 | name: Release 2 | 3 | on: 4 | push: 5 | branches: 6 | - main 7 | 8 | jobs: 9 | pypi-publish: 10 | name: upload release to PyPI 11 | runs-on: ubuntu-latest 12 | # Specifying a GitHub environment is optional, but strongly encouraged 13 | environment: release 14 | permissions: 15 | # IMPORTANT: this permission is mandatory for trusted publishing 16 | id-token: write 17 | steps: 18 | - uses: actions/checkout@v4 19 | 20 | - name: Set up Python ${{ matrix.python-version }} 21 | uses: actions/setup-python@v5 22 | with: 23 | python-version: ${{ matrix.python-version }} 24 | 25 | - name: Install dependencies 26 | shell: bash 27 | run: pip install -r requirements_dev.txt 28 | 29 | - name: Build the project 30 | shell: bash 31 | run: python -m build . 32 | 33 | - name: Publish package distributions to PyPI 34 | uses: pypa/gh-action-pypi-publish@release/v1 35 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2025 pomponchik 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /.github/workflows/lint.yml: -------------------------------------------------------------------------------- 1 | name: Lint 2 | 3 | on: push 4 | 5 | jobs: 6 | build: 7 | runs-on: ubuntu-latest 8 | strategy: 9 | matrix: 10 | python-version: ["3.8", "3.9", "3.10", "3.11", "3.12", "3.13", "3.14", "3.14t"] 11 | 12 | steps: 13 | - uses: actions/checkout@v4 14 | 15 | - name: Set up Python ${{ matrix.python-version }} 16 | uses: actions/setup-python@v5 17 | with: 18 | python-version: ${{ matrix.python-version }} 19 | 20 | - name: Cache pip dependencies 21 | uses: actions/cache@v4 22 | with: 23 | path: ~/.cache/pip 24 | key: ${{ runner.os }}-pip-${{ github.workflow }}-${{ hashFiles('requirements_dev.txt') }} 25 | restore-keys: | 26 | ${{ runner.os }}-pip-${{ github.workflow }}- 27 | 28 | - name: Install dependencies 29 | shell: bash 30 | run: pip install -r requirements_dev.txt 31 | 32 | - name: Install the library 33 | shell: bash 34 | run: pip install . 35 | 36 | - name: Run ruff 37 | shell: bash 38 | run: ruff check metacode 39 | 40 | - name: Run ruff for tests 41 | shell: bash 42 | run: ruff check tests 43 | 44 | - name: Run mypy 45 | shell: bash 46 | run: mypy --strict metacode 47 | 48 | - name: Run mypy for tests 49 | shell: bash 50 | run: mypy tests 51 | -------------------------------------------------------------------------------- /metacode/building.py: -------------------------------------------------------------------------------- 1 | from ast import AST 2 | 3 | from metacode.comment import ParsedComment 4 | from metacode.typing import EllipsisType # type: ignore[attr-defined] 5 | 6 | 7 | def build(comment: ParsedComment) -> str: 8 | if not comment.key.isidentifier(): 9 | raise ValueError('The key must be valid Python identifier.') 10 | if not comment.command.isidentifier(): 11 | raise ValueError('The command must be valid Python identifier.') 12 | 13 | result = f'# {comment.key}: {comment.command}' 14 | 15 | if comment.arguments: 16 | arguments_representations = [] 17 | 18 | for argument in comment.arguments: 19 | if isinstance(argument, AST): 20 | raise TypeError('AST nodes are read-only and cannot be written to.') 21 | if isinstance(argument, EllipsisType): 22 | arguments_representations.append('...') 23 | elif isinstance(argument, str) and argument.isidentifier(): 24 | arguments_representations.append(argument) 25 | else: 26 | arguments_representations.append(repr(argument)) 27 | 28 | result += f'[{", ".join(arguments_representations)}]' 29 | 30 | return result 31 | 32 | 33 | def insert(comment: ParsedComment, existing_comment: str, at_end: bool = False) -> str: 34 | if not existing_comment: 35 | return build(comment) 36 | 37 | if not existing_comment.lstrip().startswith('#'): 38 | raise ValueError('The existing part of the comment should start with a #.') 39 | 40 | if at_end: 41 | if existing_comment.endswith(' '): 42 | return existing_comment + build(comment) 43 | return f'{existing_comment} {build(comment)}' 44 | 45 | if existing_comment.startswith(' '): 46 | return f'{build(comment)}{existing_comment}' 47 | return f'{build(comment)} {existing_comment}' 48 | -------------------------------------------------------------------------------- /.github/workflows/tests_and_coverage.yml: -------------------------------------------------------------------------------- 1 | name: Tests 2 | 3 | on: push 4 | 5 | jobs: 6 | build: 7 | runs-on: ${{ matrix.os }} 8 | strategy: 9 | matrix: 10 | os: [macos-latest, ubuntu-latest, windows-latest] 11 | python-version: ["3.8", "3.9", "3.10", "3.11", "3.12", "3.13", "3.14", "3.14t"] 12 | 13 | steps: 14 | - uses: actions/checkout@v4 15 | - name: Set up Python ${{ matrix.python-version }} 16 | uses: actions/setup-python@v5 17 | with: 18 | python-version: ${{ matrix.python-version }} 19 | 20 | - name: Install the library 21 | shell: bash 22 | run: pip install . 23 | 24 | - name: Cache pip dependencies 25 | uses: actions/cache@v4 26 | with: 27 | path: ~/.cache/pip 28 | key: ${{ runner.os }}-pip-${{ github.workflow }}-${{ hashFiles('requirements_dev.txt') }} 29 | restore-keys: | 30 | ${{ runner.os }}-pip-${{ github.workflow }}- 31 | 32 | - name: Install dependencies 33 | shell: bash 34 | run: pip install -r requirements_dev.txt 35 | 36 | - name: Print all libs 37 | shell: bash 38 | run: pip list 39 | 40 | - name: Run tests and show coverage on the command line 41 | run: | 42 | coverage run --source=metacode --omit="*tests*" -m pytest --cache-clear --assert=plain && coverage report -m --fail-under=100 43 | coverage xml 44 | 45 | - name: Upload coverage to Coveralls 46 | if: runner.os == 'Linux' 47 | env: 48 | COVERALLS_REPO_TOKEN: ${{secrets.COVERALLS_REPO_TOKEN}} 49 | uses: coverallsapp/github-action@v2 50 | with: 51 | format: cobertura 52 | file: coverage.xml 53 | 54 | - name: Run tests and show the branch coverage on the command line 55 | run: coverage run --branch --source=metacode --omit="*tests*" -m pytest --cache-clear --assert=plain && coverage report -m --fail-under=100 56 | -------------------------------------------------------------------------------- /pyproject.toml: -------------------------------------------------------------------------------- 1 | [build-system] 2 | requires = ["setuptools==68.0.0"] 3 | build-backend = "setuptools.build_meta" 4 | 5 | [project] 6 | name = "metacode" 7 | version = "0.0.4" 8 | authors = [{ name = "Evgeniy Blinov", email = "zheni-b@yandex.ru" }] 9 | description = 'A standard language for machine-readable code comments' 10 | readme = "README.md" 11 | requires-python = ">=3.8" 12 | dependencies = ["libcst>=1.1.0 ; python_version == '3.8'", "libcst>=1.8.6 ; python_version > '3.8'", "typing_extensions ; python_version <= '3.9'"] 13 | classifiers = [ 14 | "Operating System :: OS Independent", 15 | 'Operating System :: MacOS :: MacOS X', 16 | 'Operating System :: Microsoft :: Windows', 17 | 'Operating System :: POSIX', 18 | 'Operating System :: POSIX :: Linux', 19 | 'Programming Language :: Python', 20 | 'Programming Language :: Python :: 3.8', 21 | 'Programming Language :: Python :: 3.9', 22 | 'Programming Language :: Python :: 3.10', 23 | 'Programming Language :: Python :: 3.11', 24 | 'Programming Language :: Python :: 3.12', 25 | 'Programming Language :: Python :: 3.13', 26 | 'Programming Language :: Python :: 3.14', 27 | 'Programming Language :: Python :: Free Threading', 28 | 'Programming Language :: Python :: Free Threading :: 3 - Stable', 29 | 'License :: OSI Approved :: MIT License', 30 | 'Intended Audience :: Developers', 31 | 'Topic :: Software Development :: Libraries', 32 | ] 33 | keywords = ['CST', 'comments', 'type comments', 'lints'] 34 | 35 | [tool.setuptools.package-data] 36 | "metacode" = ["py.typed"] 37 | 38 | [tool.mutmut] 39 | paths_to_mutate = "metacode" 40 | runner = "pytest" 41 | 42 | [tool.pytest.ini_options] 43 | markers = ["mypy_testing"] 44 | 45 | [tool.ruff] 46 | lint.ignore = ['E501', 'E712', 'PTH123', 'PTH118', 'PLR2004', 'PTH107', 'SIM105', 'SIM102', 'RET503', 'PLR0912', 'C901'] 47 | lint.select = ["ERA001", "YTT", "ASYNC", "BLE", "B", "A", "COM", "INP", "PIE", "T20", "PT", "RSE", "RET", "SIM", "SLOT", "TID252", "ARG", "PTH", "I", "C90", "N", "E", "W", "D201", "D202", "D419", "F", "PL", "PLE", "PLR", "PLW", "RUF", "TRY201", "TRY400", "TRY401"] 48 | format.quote-style = "single" 49 | 50 | [project.urls] 51 | 'Source' = 'https://github.com/pomponchik/metacode' 52 | 'Tracker' = 'https://github.com/pomponchik/metacode/issues' 53 | -------------------------------------------------------------------------------- /metacode/parsing.py: -------------------------------------------------------------------------------- 1 | from ast import AST, AnnAssign, BinOp, Constant, Index, Name, Sub, Subscript, Tuple 2 | from ast import parse as ast_parse 3 | from typing import Generator, List, Optional, Union 4 | 5 | from libcst import SimpleStatementLine 6 | from libcst import parse_module as cst_parse 7 | 8 | from metacode.comment import ParsedComment 9 | from metacode.errors import UnknownArgumentTypeError 10 | from metacode.typing import Arguments 11 | 12 | 13 | def get_right_part(comment: str) -> str: 14 | return '#'.join(comment.split('#')[1:]) 15 | 16 | 17 | def get_commment_from_cst(comment: str) -> Optional[str]: 18 | comment = comment.lstrip() 19 | 20 | if not comment: 21 | return None 22 | 23 | module = cst_parse(comment) 24 | 25 | try: 26 | statement = next(s for s in module.body if isinstance(s, SimpleStatementLine)) 27 | except StopIteration: 28 | return get_right_part(comment) 29 | 30 | trailing_whitespace = statement.trailing_whitespace 31 | comment_of_comment = trailing_whitespace.comment.value if trailing_whitespace.comment is not None else None 32 | 33 | if comment_of_comment is None: 34 | return get_right_part(comment) 35 | 36 | return comment_of_comment.lstrip("#").lstrip() 37 | 38 | 39 | def get_candidates(comment: str) -> Generator[ParsedComment, None, None]: 40 | comment = comment.lstrip() 41 | try: 42 | parsed_ast = ast_parse(comment) 43 | if not (len(parsed_ast.body) != 1 or not isinstance(parsed_ast.body[0], AnnAssign) or not isinstance(parsed_ast.body[0].target, Name) or not isinstance(parsed_ast.body[0].annotation, (Name, Subscript))): 44 | 45 | assign = parsed_ast.body[0] 46 | key = assign.target.id # type: ignore[union-attr] 47 | 48 | arguments: Arguments = [] 49 | if isinstance(assign.annotation, Name): 50 | command = assign.annotation.id 51 | 52 | else: 53 | 54 | command = assign.annotation.value.id # type: ignore[attr-defined] 55 | 56 | if isinstance(assign.annotation.slice, Tuple): # type: ignore[attr-defined] 57 | slice_content = assign.annotation.slice.elts # type: ignore[attr-defined] # pragma: no cover 58 | # TODO: delete this branch if minimum supported version of Python is > 3.8 (we have the Index node only in old Pythons). 59 | # TODO: also delete this the pragmas here. 60 | elif isinstance(assign.annotation.slice, Index) and isinstance(assign.annotation.slice.value, Tuple): # type: ignore[attr-defined] 61 | slice_content = assign.annotation.slice.value.elts # type: ignore[attr-defined] # pragma: no cover 62 | else: 63 | slice_content = [assign.annotation.slice] # type: ignore[attr-defined] 64 | 65 | for argument in slice_content: 66 | # TODO: delete this branch if minimum supported version of Python is > 3.8 (we have the Index node only in old Pythons). 67 | if isinstance(argument, Index): # pragma: no cover 68 | argument = argument.value # noqa: PLW2901 69 | if isinstance(argument, Name): 70 | arguments.append(argument.id) 71 | elif isinstance(argument, Constant): 72 | arguments.append(argument.value) 73 | elif isinstance(argument, BinOp) and isinstance(argument.left, Name) and isinstance(argument.right, Name) and isinstance(argument.op, Sub): 74 | arguments.append(f'{argument.left.id}-{argument.right.id}') 75 | else: 76 | arguments.append(argument) 77 | 78 | yield ParsedComment( 79 | key=key, 80 | command=command, 81 | arguments=arguments, 82 | ) 83 | 84 | sub_comment = get_commment_from_cst(comment) 85 | if sub_comment is not None: 86 | yield from get_candidates(sub_comment) 87 | 88 | except SyntaxError: 89 | splitted_comment = comment.split('#') 90 | if len(splitted_comment) > 1: 91 | yield from get_candidates(get_right_part(comment)) 92 | 93 | 94 | def parse(comment: str, key: Union[str, List[str]], allow_ast: bool = False, ignore_case: bool = False) -> List[ParsedComment]: 95 | keys: List[str] = [key] if isinstance(key, str) else key 96 | for one_key in keys: 97 | if not one_key.isidentifier(): 98 | raise ValueError('The key must be valid Python identifier.') 99 | if ignore_case: 100 | keys = [x.lower() for x in keys] 101 | 102 | result: List[ParsedComment] = [] 103 | 104 | comment = comment.lstrip() 105 | 106 | if not comment: 107 | return result 108 | 109 | for candidate in get_candidates(comment): 110 | if candidate.key in keys or (candidate.key.lower() in keys and ignore_case): 111 | for argument in candidate.arguments: 112 | if isinstance(argument, AST) and not allow_ast: 113 | raise UnknownArgumentTypeError(f'An argument of unknown type was found in the comment {comment!r}. If you want to process arbitrary code variants, not just constants, pass allow_ast=True.') 114 | result.append(candidate) 115 | 116 | return result 117 | -------------------------------------------------------------------------------- /tests/test_building.py: -------------------------------------------------------------------------------- 1 | from ast import Name 2 | 3 | import pytest 4 | from full_match import match 5 | 6 | from metacode import ParsedComment, build, insert 7 | 8 | 9 | def test_run_build_with_wrong_key_or_action(): 10 | with pytest.raises(ValueError, match=match('The key must be valid Python identifier.')): 11 | build(ParsedComment( 12 | key='123', 13 | command='action', 14 | arguments=[], 15 | )) 16 | 17 | with pytest.raises(ValueError, match=match('The command must be valid Python identifier.')): 18 | build(ParsedComment( 19 | key='key', 20 | command='123', 21 | arguments=[], 22 | )) 23 | 24 | 25 | def test_build_ast(): 26 | with pytest.raises(TypeError, match=match('AST nodes are read-only and cannot be written to.')): 27 | build(ParsedComment( 28 | key='key', 29 | command='command', 30 | arguments=[Name()], 31 | )) 32 | 33 | 34 | def test_create_simple_comment(): 35 | assert build(ParsedComment( 36 | key='key', 37 | command='command', 38 | arguments=[], 39 | )) == '# key: command' 40 | 41 | 42 | def test_create_difficult_comment(): 43 | assert build(ParsedComment( 44 | key='key', 45 | command='command', 46 | arguments=[1], 47 | )) == '# key: command[1]' 48 | 49 | assert build(ParsedComment( 50 | key='key', 51 | command='command', 52 | arguments=[1, 2, 3], 53 | )) == '# key: command[1, 2, 3]' 54 | 55 | assert build(ParsedComment( 56 | key='key', 57 | command='command', 58 | arguments=['build'], 59 | )) == '# key: command[build]' 60 | 61 | assert build(ParsedComment( 62 | key='key', 63 | command='command', 64 | arguments=['build', 'build'], 65 | )) == '# key: command[build, build]' 66 | 67 | assert build(ParsedComment( 68 | key='key', 69 | command='command', 70 | arguments=['lol-kek'], 71 | )) == "# key: command['lol-kek']" 72 | 73 | assert build(ParsedComment( 74 | key='key', 75 | command='command', 76 | arguments=['lol-kek', 'lol-kek-chedurek'], 77 | )) == "# key: command['lol-kek', 'lol-kek-chedurek']" 78 | 79 | assert build(ParsedComment( 80 | key='key', 81 | command='command', 82 | arguments=[...], 83 | )) == "# key: command[...]" 84 | 85 | assert build(ParsedComment( 86 | key='key', 87 | command='command', 88 | arguments=[..., ...], 89 | )) == "# key: command[..., ...]" 90 | 91 | assert build(ParsedComment( 92 | key='key', 93 | command='command', 94 | arguments=[1.5], 95 | )) == "# key: command[1.5]" 96 | 97 | assert build(ParsedComment( 98 | key='key', 99 | command='command', 100 | arguments=[1.5, 3.0], 101 | )) == "# key: command[1.5, 3.0]" 102 | 103 | assert build(ParsedComment( 104 | key='key', 105 | command='command', 106 | arguments=[5j], 107 | )) == "# key: command[5j]" 108 | 109 | assert build(ParsedComment( 110 | key='key', 111 | command='command', 112 | arguments=[None], 113 | )) == "# key: command[None]" 114 | 115 | assert build(ParsedComment( 116 | key='key', 117 | command='command', 118 | arguments=[True], 119 | )) == "# key: command[True]" 120 | 121 | assert build(ParsedComment( 122 | key='key', 123 | command='command', 124 | arguments=[False], 125 | )) == "# key: command[False]" 126 | 127 | assert build(ParsedComment( 128 | key='key', 129 | command='command', 130 | arguments=[1, 2, 3, 1.5, 3.0, 5j, 1000j, 'build', 'build2', 'lol-kek', 'lol-kek-chedurek', None, True, False, ...], 131 | )) == "# key: command[1, 2, 3, 1.5, 3.0, 5j, 1000j, build, build2, 'lol-kek', 'lol-kek-chedurek', None, True, False, ...]" 132 | 133 | 134 | def test_insert_to_strange_comment(): 135 | with pytest.raises(ValueError, match=match('The existing part of the comment should start with a #.')): 136 | insert(ParsedComment(key='key', command='command', arguments=[]), 'kek', at_end=True) 137 | 138 | with pytest.raises(ValueError, match=match('The existing part of the comment should start with a #.')): 139 | insert(ParsedComment(key=' key', command='command', arguments=[]), 'kek', at_end=True) 140 | 141 | with pytest.raises(ValueError, match=match('The existing part of the comment should start with a #.')): 142 | insert(ParsedComment(key=' key', command='command', arguments=[]), 'kek') 143 | 144 | with pytest.raises(ValueError, match=match('The existing part of the comment should start with a #.')): 145 | insert(ParsedComment(key='key', command='command', arguments=[]), 'kek') 146 | 147 | 148 | def test_insert_at_begin_to_empty(): 149 | comment = ParsedComment( 150 | key='key', 151 | command='command', 152 | arguments=['build'], 153 | ) 154 | 155 | assert insert(comment, '') == build(comment) 156 | 157 | 158 | def test_insert_at_end_to_empty(): 159 | comment = ParsedComment( 160 | key='key', 161 | command='command', 162 | arguments=['build'], 163 | ) 164 | 165 | assert insert(comment, '', at_end=True) == build(comment) 166 | 167 | 168 | def test_insert_at_begin_to_not_empty(): 169 | comment = ParsedComment( 170 | key='key', 171 | command='command', 172 | arguments=['build'], 173 | ) 174 | 175 | assert insert(comment, '# kek') == build(comment) + ' # kek' 176 | assert insert(comment, ' # kek') == build(comment) + ' # kek' 177 | assert insert(comment, build(comment)) == build(comment) + ' ' + build(comment) 178 | 179 | 180 | def test_insert_at_end_to_not_empty(): 181 | comment = ParsedComment( 182 | key='key', 183 | command='command', 184 | arguments=['build'], 185 | ) 186 | 187 | assert insert(comment, '# kek', at_end=True) == '# kek ' + build(comment) 188 | assert insert(comment, '# kek ', at_end=True) == '# kek ' + build(comment) 189 | assert insert(comment, build(comment), at_end=True) == build(comment) + ' ' + build(comment) 190 | -------------------------------------------------------------------------------- /tests/test_parsing.py: -------------------------------------------------------------------------------- 1 | from ast import AST, BinOp, Index, Name, Subscript 2 | 3 | import pytest 4 | from full_match import match 5 | 6 | from metacode import ParsedComment, UnknownArgumentTypeError, parse 7 | 8 | 9 | def test_wrong_key(): 10 | with pytest.raises(ValueError, match=match('The key must be valid Python identifier.')): 11 | parse('abc', '123') 12 | 13 | 14 | def test_empty_string(): 15 | assert parse('', 'kek') == [] 16 | 17 | 18 | def test_only_not_python_code(): 19 | assert parse('run, Forest, run!', 'lol') == [] 20 | assert parse('run, Forest, run! # kek!', 'lol') == [] 21 | 22 | 23 | def test_one_simplest_expression(): 24 | assert parse('lol: kek', 'lol') == [ParsedComment(key='lol', command='kek', arguments=[])] 25 | assert parse('lol: kek', 'kek') == [] 26 | 27 | 28 | def test_expressions_with_not_python_code(): 29 | assert parse('lol: kek # run, Forest, run!', 'lol') == [ParsedComment(key='lol', command='kek', arguments=[])] 30 | assert parse('run, Forest, run! #lol: kek', 'lol') == [ParsedComment(key='lol', command='kek', arguments=[])] 31 | assert parse('run, Forest, run! # lol: kek', 'lol') == [ParsedComment(key='lol', command='kek', arguments=[])] 32 | assert parse('run, Forest, run! # lol: kek # run, Forest, run!', 'lol') == [ParsedComment(key='lol', command='kek', arguments=[])] 33 | assert parse('run, Forest, run! #lol: kek# run, Forest, run!', 'lol') == [ParsedComment(key='lol', command='kek', arguments=[])] 34 | assert parse('run, Forest, run! #lol: kek[1, 2, 3]# run, Forest, run!', 'lol') == [ParsedComment(key='lol', command='kek', arguments=[1, 2, 3])] 35 | assert parse('run, Forest, run! #lol: kek[1, 2, 3]# run, Forest, run!#lol: kek', 'lol') == [ParsedComment(key='lol', command='kek', arguments=[1, 2, 3]), ParsedComment(key='lol', command='kek', arguments=[])] 36 | 37 | 38 | def test_two_simplest_expressions_with_same_keys(): 39 | assert parse('lol: kek # lol: kekokek', 'lol') == [ParsedComment(key='lol', command='kek', arguments=[]), ParsedComment(key='lol', command='kekokek', arguments=[])] 40 | assert parse('lol: kek # lol: kekokek', 'kek') == [] 41 | 42 | 43 | def test_one_difficult_expression(): 44 | assert parse('lol: kek[a]', 'lol') == [ParsedComment(key='lol', command='kek', arguments=['a'])] 45 | assert parse('lol: kek[a, b, c]', 'lol') == [ParsedComment(key='lol', command='kek', arguments=['a', 'b', 'c'])] 46 | assert parse('lol: kek[a, b, "c"]', 'lol') == [ParsedComment(key='lol', command='kek', arguments=['a', 'b', 'c'])] 47 | assert parse('lol: kek["a", "b", "c"]', 'lol') == [ParsedComment(key='lol', command='kek', arguments=['a', 'b', 'c'])] 48 | assert parse('lol: kek["a", False, 111]', 'lol') == [ParsedComment(key='lol', command='kek', arguments=['a', False, 111])] 49 | assert parse('lol: kek[True, None, 111.5, 5j]', 'lol') == [ParsedComment(key='lol', command='kek', arguments=[True, None, 111.5, 5j])] 50 | assert parse('lol: kek[...]', 'lol') == [ParsedComment(key='lol', command='kek', arguments=[Ellipsis])] 51 | 52 | assert parse('lol: kek[a]', 'kek') == [] 53 | 54 | 55 | def test_parse_ast_complex_sum_argument_when_its_allowed(): 56 | parsed_comments = parse('lol: kek[3 + 5j]', 'lol', allow_ast=True) 57 | 58 | assert len(parsed_comments) == 1 59 | 60 | parsed_comment = parsed_comments[0] 61 | 62 | assert parsed_comment.key == 'lol' 63 | assert parsed_comment.command == 'kek' 64 | assert len(parsed_comment.arguments) == 1 65 | 66 | ast_argument = parsed_comment.arguments[0] 67 | 68 | assert isinstance(ast_argument, AST) 69 | assert isinstance(ast_argument, BinOp) 70 | assert ast_argument.left.value == 3 71 | assert ast_argument.right.value == 5j 72 | 73 | 74 | def test_parse_ast_subscription_argument_when_its_allowed(): 75 | parsed_comments = parse('lol: kek[jej[ok]]', 'lol', allow_ast=True) 76 | 77 | assert len(parsed_comments) == 1 78 | 79 | parsed_comment = parsed_comments[0] 80 | 81 | assert parsed_comment.key == 'lol' 82 | assert parsed_comment.command == 'kek' 83 | assert len(parsed_comment.arguments) == 1 84 | 85 | ast_argument = parsed_comment.arguments[0] 86 | 87 | assert isinstance(ast_argument, AST) 88 | # TODO: delete this shit about Index if minimum supported version of Python is > 3.8 (we have the Index node only in old Pythons). 89 | assert isinstance(ast_argument, (Subscript, Index)) 90 | if isinstance(ast_argument, Index): 91 | ast_argument = ast_argument.value 92 | assert ast_argument.value.id == 'jej' 93 | assert isinstance(ast_argument.slice, (Name, Index)) 94 | if isinstance(ast_argument.slice, Index): 95 | assert ast_argument.slice.value.id == 'ok' 96 | else: 97 | assert ast_argument.slice.id == 'ok' 98 | 99 | 100 | def test_parse_ast_complex_sum_argument_when_its_not_allowed(): 101 | with pytest.raises(UnknownArgumentTypeError, match=match('An argument of unknown type was found in the comment \'lol: kek[3 + 5j]\'. If you want to process arbitrary code variants, not just constants, pass allow_ast=True.')): 102 | parse('lol: kek[3 + 5j]', 'lol') 103 | 104 | 105 | def test_multiple_not_simple_expressions(): 106 | assert parse('lol: kek[a] # lol: kek[a, b, c]', 'lol') == [ParsedComment(key='lol', command='kek', arguments=['a']), ParsedComment(key='lol', command='kek', arguments=['a', 'b', 'c'])] 107 | assert parse('lol: kek[a] # lol: kek', 'lol') == [ParsedComment(key='lol', command='kek', arguments=['a']), ParsedComment(key='lol', command='kek', arguments=[])] 108 | assert parse('lol: kek[a, b, c] # lol: kek', 'lol') == [ParsedComment(key='lol', command='kek', arguments=['a', 'b', 'c']), ParsedComment(key='lol', command='kek', arguments=[])] 109 | 110 | 111 | def test_empty_subcomment(): 112 | assert parse('kek! # #c[]: lel', 'lol') == [] 113 | assert parse('kek! ##c[]: lel', 'lol') == [] 114 | assert parse('##c[]: lel', 'lol') == [] 115 | assert parse('#####################', 'lol') == [] 116 | assert parse('# ###### ##### # ## #### ##', 'lol') == [] 117 | assert parse(' # ###### ##### # ## #### ##', 'lol') == [] 118 | assert parse(' # ###### ##### # ## #### ##lol: kek[a, b, c]', 'lol') == [ParsedComment(key='lol', command='kek', arguments=['a', 'b', 'c'])] 119 | 120 | 121 | def test_sub_expressions_in_arguments(): 122 | assert parse('lol: kek[a-b]', 'lol') == [ParsedComment(key='lol', command='kek', arguments=['a-b'])] 123 | 124 | 125 | def test_plus_expressions_in_arguments(): 126 | with pytest.raises(UnknownArgumentTypeError, match=match('An argument of unknown type was found in the comment \'lol: kek[a+b]\'. If you want to process arbitrary code variants, not just constants, pass allow_ast=True.')): 127 | parse('lol: kek[a+b]', 'lol') 128 | 129 | parsed_comments = parse('lol: kek[a+b]', 'lol', allow_ast=True) 130 | comment = parsed_comments[0] 131 | 132 | assert comment.key == 'lol' 133 | assert comment.command == 'kek' 134 | 135 | assert len(comment.arguments) == 1 136 | assert isinstance(comment.arguments[0], AST) 137 | 138 | 139 | def test_triple_subs(): 140 | with pytest.raises(UnknownArgumentTypeError, match=match('An argument of unknown type was found in the comment \'lol: kek[a-b-c]\'. If you want to process arbitrary code variants, not just constants, pass allow_ast=True.')): 141 | parse('lol: kek[a-b-c]', 'lol') 142 | 143 | parsed_comment = parse('lol: kek[a-b-c]', 'lol', allow_ast=True)[0] 144 | 145 | assert len(parsed_comment.arguments) == 1 146 | 147 | argument = parsed_comment.arguments[0] 148 | 149 | assert isinstance(argument, (BinOp, Index)) 150 | 151 | 152 | def test_get_multiple_keys(): 153 | assert parse('lol: kek[a]# kek: lol[a]', ['lol', 'kek']) == [ParsedComment(key='lol', command='kek', arguments=['a']), ParsedComment(key='kek', command='lol', arguments=['a'])] 154 | assert parse('lol: kek[a]# kek: lol[a]', ['lol', 'KEK'], ignore_case=True) == [ParsedComment(key='lol', command='kek', arguments=['a']), ParsedComment(key='kek', command='lol', arguments=['a'])] 155 | 156 | 157 | def test_ignore_case(): 158 | assert parse('KEY: action', 'key', ignore_case=True) == [ParsedComment(key='KEY', command='action', arguments=[])] 159 | assert parse('lol: kek[a]# kek: lol[a]', ['lol', 'KEK'], ignore_case=True) == [ParsedComment(key='lol', command='kek', arguments=['a']), ParsedComment(key='kek', command='lol', arguments=['a'])] 160 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 |
2 | ⓘ 3 | 4 | [![Downloads](https://static.pepy.tech/badge/metacode/month)](https://pepy.tech/project/metacode) 5 | [![Downloads](https://static.pepy.tech/badge/metacode)](https://pepy.tech/project/metacode) 6 | [![Coverage Status](https://coveralls.io/repos/github/pomponchik/metacode/badge.svg?branch=main)](https://coveralls.io/github/pomponchik/metacode?branch=main) 7 | [![Lines of code](https://sloc.xyz/github/pomponchik/metacode/?category=code)](https://github.com/boyter/scc/) 8 | [![Hits-of-Code](https://hitsofcode.com/github/pomponchik/metacode?branch=main&label=Hits-of-Code&exclude=docs/)](https://hitsofcode.com/github/pomponchik/metacode/view?branch=main) 9 | [![Test-Package](https://github.com/pomponchik/metacode/actions/workflows/tests_and_coverage.yml/badge.svg)](https://github.com/pomponchik/metacode/actions/workflows/tests_and_coverage.yml) 10 | [![Python versions](https://img.shields.io/pypi/pyversions/metacode.svg)](https://pypi.python.org/pypi/metacode) 11 | [![PyPI version](https://badge.fury.io/py/metacode.svg)](https://badge.fury.io/py/metacode) 12 | [![Checked with mypy](http://www.mypy-lang.org/static/mypy_badge.svg)](http://mypy-lang.org/) 13 | [![Ruff](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/ruff/main/assets/badge/v2.json)](https://github.com/astral-sh/ruff) 14 | [![DeepWiki](https://deepwiki.com/badge.svg)](https://deepwiki.com/pomponchik/metacode) 15 | 16 |
17 | 18 | 19 | ![logo](https://raw.githubusercontent.com/pomponchik/metacode/develop/docs/assets/logo_3.svg) 20 | 21 | Many source code analysis tools use comments in a special format to mark it up. This is an important part of the Python ecosystem, but there is still no single standard around it. This library offers such a standard. 22 | 23 | 24 | ## Table of contents 25 | 26 | - [**Why?**](#why) 27 | - [**The language**](#the-language) 28 | - [**Installation**](#installation) 29 | - [**Usage**](#usage) 30 | - [**What about other languages?**](#what-about-other-languages) 31 | 32 | 33 | ## Why? 34 | 35 | In the Python ecosystem, there are many tools dealing with source code: linters, test coverage collection systems, and many others. Many of them use special comments, and as a rule, the style of these comments is very similar. Here are some examples: 36 | 37 | - [`Ruff`](https://docs.astral.sh/ruff/linter/#error-suppression), [`Vulture`](https://github.com/jendrikseipp/vulture?tab=readme-ov-file#flake8-noqa-comments) —> `# noqa`, `# noqa: E741, F841`. 38 | - [`Black`](https://black.readthedocs.io/en/stable/usage_and_configuration/the_basics.html#ignoring-sections) and [`Ruff`](https://docs.astral.sh/ruff/formatter/#format-suppression) —> `# fmt: on`, `# fmt: off`. 39 | - [`Mypy`](https://discuss.python.org/t/ignore-mypy-specific-type-errors/58535) —> `# type: ignore`, `type: ignore[error-code]`. 40 | - [`Coverage`](https://coverage.readthedocs.io/en/7.13.0/excluding.html#default-exclusions) —> `# pragma: no cover`, `# pragma: no branch`. 41 | - [`Isort`](https://pycqa.github.io/isort/docs/configuration/action_comments.html) —> `# isort: skip`, `# isort: off`. 42 | - [`Bandit`](https://bandit.readthedocs.io/en/latest/config.html#suppressing-individual-lines) —> `# nosec`. 43 | 44 | But you know what? *There is no single standard for such comments*. 45 | 46 | The internal implementation of reading such comments is also different. Someone uses regular expressions, someone uses even more primitive string processing tools, and someone uses full-fledged parsers, including the Python parser or even written from scratch. 47 | 48 | As a result, as a user, you need to remember the rules by which comments are written for each specific tool. And at the same time, you can't be sure that things like double comments (when you want to leave 2 comments for different tools in one line of code) will work in principle. And as the creator of such tools, you are faced with a seemingly simple task — just to read a comment — and find out for yourself that it suddenly turns out to be quite difficult, and there are many possible mistakes. 49 | 50 | This is exactly the problem that this library solves. It describes a [simple and intuitive standard](https://xkcd.com/927/) for action comments, and also offers a ready-made parser that creators of other tools can use. The standard offered by this library is based entirely on a subset of the Python syntax and can be easily reimplemented even if you do not want to use this library directly. 51 | 52 | 53 | ## The language 54 | 55 | So, this library offers a language for action comments. Its syntax is a subset of Python syntax, but without Python semantics, as full-fledged execution does not occur. The purpose of the language is simply to provide the developer with the content of the comment in a convenient way, if it is written in a compatible format. If the comment format is not compatible with the parser, it is ignored. 56 | 57 | From the point of view of the language, any meaningful comment can consist of 3 elements: 58 | 59 | - **Key**. This is usually the name of the specific tool for which this comment is intended, but in some cases it may be something else. This can be any string allowed as an [identifier](https://docs.python.org/3/reference/lexical_analysis.html#identifiers) in Python. 60 | - **Action**. The short name of the action that you want to link to this line. Also, only the allowed Python identifier. 61 | - **List of arguments**. These are often some kind of identifiers of specific linting rules or other arguments associated with this action. The list of possible data types described below. 62 | 63 | Consider a comment designed to ignore a specific mypy rule: 64 | 65 | ``` 66 | # type: ignore[error-code] 67 | └-key-┘└action┴-arguments┘ 68 | ``` 69 | 70 | > ↑ The key here is the word `type`, that is, what you see before the colon. The action is the `ignore` word, that is, what comes before the square brackets, but after the colon. Finally, the list of arguments is what is in square brackets, in this case, there is only one argument in it: `error-code`. 71 | 72 | Simplified writing is also possible, without a list of arguments: 73 | 74 | ``` 75 | # type: ignore 76 | └-key-┘└action┘ 77 | ``` 78 | 79 | > ↑ In this case, the parser assumes that there is an argument list, but it is empty. 80 | 81 | The number of arguments in the list is unlimited, they can be separated by commas. Here are the valid data types for arguments: 82 | 83 | - [Valid Python identifiers](https://docs.python.org/3/reference/lexical_analysis.html#identifiers). They are interpreted as strings. 84 | - Two valid Python identifiers, separated by the `-` symbol, like this: `error-code`. There can also be any number of spaces between them, they will be ignored. Interpreted as a single string. 85 | - String literals. 86 | - Numeric literals (`int`, `float`, `complex`). 87 | - Boolean literals (`True` and `False`). 88 | - `None`. 89 | - `...` ([ellipsis](https://docs.python.org/dev/library/constants.html#Ellipsis)). 90 | - Any other Python-compatible code. This is disabled by default, but you can force the mode of reading such code and get descriptions for any inserts of such code in the form of [`AST` objects](https://docs.python.org/3/library/ast.html#ast.AST), after which you can somehow process it yourself. 91 | 92 | The syntax of all these data types is completely similar to the Python original (except that you can't use multi-line writing options). Over time, it is possible to extend the possible syntax of `metacode`, but this template will always be supported. 93 | 94 | There can be several comments in the `metacode` format. In this case, they should be interspersed with the `#` symbol, as if each subsequent comment is a comment on the previous one. You can also add regular text comments, they will just be ignored by the parser if they are not in `metacode` format: 95 | 96 | ``` python 97 | # type: ignore # <- This is a comment for mypy! # fmt: off # <- And this is a comment for Ruff! 98 | ``` 99 | 100 | If you scroll through this text [above](#why) to the examples of action comments from various tools, you may notice that the syntax of most of them (but not all) is it can be described using `metacode`, and if not, it can be easily adapted to `metacode`. Read on to learn how to use a ready-made parser in practice. 101 | 102 | 103 | ## Installation 104 | 105 | Install it: 106 | 107 | ```bash 108 | pip install metacode 109 | ``` 110 | 111 | You can also quickly try out this and other packages without having to install using [instld](https://github.com/pomponchik/instld). 112 | 113 | 114 | ## Usage 115 | 116 | The parser offered by this library is just one function that is imported like this: 117 | 118 | ```python 119 | from metacode import parse 120 | ``` 121 | 122 | To use it, you need to extract the text of the comment in some third-party way (preferably, but not necessarily, without the `#` symbol at the beginning) and pass it, and the expected key must also be passed as the second argument. As a result, you will receive a list of the contents of all the comments that were parsed: 123 | 124 | ```python 125 | print(parse('type: ignore[error-code]', 'type')) 126 | #> [ParsedComment(key='type', command='ignore', arguments=['error-code'])] 127 | print(parse('type: ignore[error-code] # type: not_ignore[another-error]', 'type')) 128 | #> [ParsedComment(key='type', command='ignore', arguments=['error-code']), ParsedComment(key='type', command='not_ignore', arguments=['another-error'])] 129 | ``` 130 | 131 | As you can see, the `parse()` function returns a list of `ParsedComment` objects. Here are the fields of this type's objects and their expected types: 132 | 133 | ```python 134 | key: str 135 | command: str 136 | arguments: List[Optional[Union[str, int, float, complex, bool, EllipsisType, AST]]] 137 | ``` 138 | 139 | > ↑ Please note that you are transmitting a key, which means that the result is returned filtered by this key. This way you can read only those comments that relate to your tool, ignoring the rest. 140 | 141 | By default, an argument in a comment must be of one of the strictly allowed types. However, you can enable reading of arbitrary other types, in which case they will be transmitted in the [`AST` node](https://docs.python.org/3/library/ast.html#ast.AST) format. To do this, pass `allow_ast=True`: 142 | 143 | ```python 144 | print(parse('key: action[a + b]', 'key', allow_ast=True)) 145 | #> [ParsedComment(key='key', command='action', arguments=[])] 146 | ``` 147 | 148 | > ↑ If you do not pass `allow_ast=True`, a `metacode.errors.UnknownArgumentTypeError` exception will be raised. When processing an argument, you can also raise this exception for an AST node of a format that your tool does not expect. 149 | 150 | > ⚠️ Be careful when writing code that analyzes the AST. Different versions of the Python interpreter can generate different AST based on the same code, so don't forget to test your code (for example, using [matrix](https://docs.github.com/en/actions/how-tos/write-workflows/choose-what-workflows-do/run-job-variations) or [tox](https://tox.wiki/)) well. Otherwise, it is better to use standard `metacode` argument types. 151 | 152 | You can allow your users to write keys in any case. To do this, pass `ignore_case=True`: 153 | 154 | ```python 155 | print(parse('KEY: action', 'key', ignore_case=True)) 156 | #> [ParsedComment(key='KEY', command='action', arguments=[])] 157 | ``` 158 | 159 | You can also easily add support for several different keys. To do this, pass a list of keys instead of one key: 160 | 161 | ```python 162 | print(parse('key: action # other_key: other_action', ['key', 'other_key'])) 163 | #> [ParsedComment(key='key', command='action', arguments=[]), ParsedComment(key='other_key', command='other_action', arguments=[])] 164 | ``` 165 | 166 | Well, now we can read the comments. But what if we want to record? There is another function for this: `insert()`: 167 | 168 | ```python 169 | from metacode import insert, ParsedComment 170 | ``` 171 | 172 | You send the comment you want to insert there, as well as the current comment (empty if there is no comment, or starting with # if there is), and you get a ready-made new comment text: 173 | 174 | ```python 175 | print(insert(ParsedComment(key='key', command='command', arguments=['lol', 'lol-kek']), '')) 176 | # key: command[lol, 'lol-kek'] 177 | print(insert(ParsedComment(key='key', command='command', arguments=['lol', 'lol-kek']), '# some existing text')) 178 | # key: command[lol, 'lol-kek'] # some existing text 179 | ``` 180 | 181 | As you can see, our comment is inserted before the existing comment. However, you can do the opposite: 182 | 183 | ```python 184 | print(insert(ParsedComment(key='key', command='command', arguments=['lol', 'lol-kek']), '# some existing text', at_end=True)) 185 | # some existing text # key: command[lol, 'lol-kek'] 186 | ``` 187 | 188 | > ⚠️ Be careful: AST nodes can be read, but cannot be written. 189 | 190 | 191 | ## What about other languages? 192 | 193 | If you are writing your Python-related tool not in Python, as is currently fashionable, but in some other language, such as Rust, you may want to adhere to the `metacode` standard for machine-readable comments, however, you cannot directly use the ready-made parser described [above](#usage). What to do? 194 | 195 | The proposed `metacode` language is a syntactic subset of Python. The original `metacode` parser allows you to read arbitrary arguments written in Python as AST nodes. The rules for such parsing are determined by the specific version of the interpreter that `metacode` runs under, and they cannot be strictly standardized, since [Python syntax](https://docs.python.org/3/reference/grammar.html) is gradually evolving in an unpredictable direction. However, you can use a "safe" subset of the valid syntax by implementing your parser based on this [`EBNF`](https://en.wikipedia.org/wiki/Extended_Backus%E2%80%93Naur_form) grammar: 196 | 197 | ``` 198 | line ::= element { "#" element } 199 | element ::= statement | ignored_content 200 | statement ::= key ":" action [ "[" arguments "]" ] 201 | ignored_content ::= ? any sequence of characters excluding "#" ? 202 | 203 | key ::= identifier 204 | action ::= identifier { "-" identifier } 205 | arguments ::= argument { "," argument } 206 | 207 | argument ::= hyphenated_identifier 208 | | identifier 209 | | string_literal 210 | | complex_literal 211 | | number_literal 212 | | "True" | "False" | "None" | "..." 213 | 214 | hyphenated_identifier ::= identifier "-" identifier 215 | identifier ::= ? python-style identifier ? 216 | string_literal ::= ? python-style string ? 217 | number_literal ::= ? python-style number ? 218 | complex_literal ::= ? python-style complex number ? 219 | ``` 220 | 221 | If you suddenly implement your ready-made open-source parser of this grammar in a language other than Python, please [let me know](https://github.com/pomponchik/metacode/issues). This information can be added to this text. 222 | -------------------------------------------------------------------------------- /docs/assets/logo_3.svg: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 21 | 41 | 49 | 51 | 57 | 58 | 59 | 62 | 69 | 76 | 83 | 90 | 97 | 104 | 111 | 118 | 125 | 132 | 139 | 146 | 153 | 160 | 167 | 174 | 181 | 188 | 195 | 203 | 210 | 217 | 224 | 225 | 227 | 228 | 246 | 247 | 251 | 255 | 259 | 263 | 267 | 271 | 275 | 279 | 283 | 287 | 291 | 295 | 299 | 303 | 307 | 311 | 315 | 319 | 323 | 327 | 331 | 335 | 339 | 343 | 344 | 349 | 350 | 351 | 357 | 358 | 361 | 365 | 369 | 373 | 379 | 380 | 381 | --------------------------------------------------------------------------------