├── .github └── workflows │ └── core.yml ├── .gitignore ├── LICENSE ├── MANIFEST.in ├── README.rst ├── SECURITY.rst ├── gast ├── __init__.py ├── ast2.py ├── ast3.py ├── astn.py ├── gast.py ├── unparser.py └── version.py ├── setup.py ├── tests ├── __init__.py ├── test_api.py ├── test_compat.py ├── test_py3_12.py ├── test_self.py └── test_unparser.py └── tox.ini /.github/workflows/core.yml: -------------------------------------------------------------------------------- 1 | name: core 2 | 3 | on: 4 | push: 5 | branches: 6 | - master 7 | pull_request: 8 | branches: 9 | - master 10 | jobs: 11 | build: 12 | strategy: 13 | matrix: 14 | python-version: ["3.8", "3.9", "3.10", "3.11", "3.12", "3.13.0-beta.1"] 15 | include: 16 | - python-version: 3.6 17 | os: ubuntu-20.04 18 | - python-version: 3.7 19 | os: ubuntu-20.04 20 | runs-on: ${{ matrix.os || 'ubuntu-latest' }} 21 | 22 | steps: 23 | - uses: actions/checkout@v3 24 | - name: Setup Python ${{ matrix.python-version }} 25 | uses: actions/setup-python@v4 26 | with: 27 | python-version: ${{ matrix.python-version }} 28 | - name: Install dependencies 29 | run: | 30 | python -m pip install --upgrade pip 31 | pip install setuptools 32 | pip install pytest 33 | - name: Setup 34 | run: | 35 | python setup.py install 36 | - name: Testing sequential 37 | run: | 38 | pytest 39 | 40 | 41 | build-27: 42 | runs-on: ubuntu-20.04 43 | container: 44 | image: python:2.7.18-buster 45 | env: 46 | py27: 2.7 47 | steps: 48 | - name: Checkout code 49 | uses: actions/checkout@v3 50 | - name: Install dependencies 51 | run: | 52 | python -m pip install --upgrade pip 53 | pip install pytest 54 | - name: Setup 55 | run: | 56 | python setup.py install 57 | - name: Testing sequential 58 | run: | 59 | pytest 60 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | # Byte-compiled / optimized / DLL files 2 | __pycache__/ 3 | *.py[cod] 4 | *$py.class 5 | 6 | # Distribution / packaging 7 | build/ 8 | dist/ 9 | *.egg-info/ 10 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | Copyright (c) 2016, Serge Guelton 2 | All rights reserved. 3 | 4 | Redistribution and use in source and binary forms, with or without 5 | modification, are permitted provided that the following conditions are met: 6 | 7 | Redistributions of source code must retain the above copyright notice, this 8 | list of conditions and the following disclaimer. 9 | 10 | Redistributions in binary form must reproduce the above copyright notice, 11 | this list of conditions and the following disclaimer in the documentation 12 | and/or other materials provided with the distribution. 13 | 14 | Neither the name of HPCProject, Serge Guelton nor the names of its 15 | contributors may be used to endorse or promote products derived from this 16 | software without specific prior written permission. 17 | 18 | THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND 19 | ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED 20 | WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE 21 | DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE 22 | FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL 23 | DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR 24 | SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER 25 | CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, 26 | OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE 27 | OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 28 | 29 | 30 | -------------------------------------------------------------------------------- /MANIFEST.in: -------------------------------------------------------------------------------- 1 | recursive-include tests *.py 2 | include LICENSE 3 | -------------------------------------------------------------------------------- /README.rst: -------------------------------------------------------------------------------- 1 | GAST, daou naer! 2 | ================ 3 | 4 | A generic AST to represent Python2 and Python3's Abstract Syntax Tree(AST). 5 | 6 | GAST provides a compatibility layer between the AST of various Python versions, 7 | as produced by ``ast.parse`` from the standard ``ast`` module. 8 | 9 | Basic Usage 10 | ----------- 11 | 12 | .. code:: python 13 | 14 | >>> import ast, gast 15 | >>> code = open('file.py').read() 16 | >>> tree = ast.parse(code) 17 | >>> gtree = gast.ast_to_gast(tree) 18 | >>> ... # process gtree 19 | >>> tree = gast.gast_to_ast(gtree) 20 | >>> ... # do stuff specific to tree 21 | 22 | API 23 | --- 24 | 25 | ``gast`` provides the same API as the ``ast`` module. All functions and classes 26 | from the ``ast`` module are also available in the ``gast`` module, and work 27 | accordingly on the ``gast`` tree. 28 | 29 | Three notable exceptions: 30 | 31 | 1. ``gast.parse`` directly produces a GAST node. It's equivalent to running 32 | ``gast.ast_to_gast`` on the output of ``ast.parse``. 33 | 34 | 2. ``gast.dump`` dumps the ``gast`` common representation, not the original 35 | one. 36 | 37 | 3. ``gast.gast_to_ast`` and ``gast.ast_to_gast`` can be used to convert 38 | from one ast to the other, back and forth. 39 | 40 | Version Compatibility 41 | --------------------- 42 | 43 | GAST is tested using ``tox`` and Travis on the following Python versions: 44 | 45 | - 2.7 46 | - 3.4 47 | - 3.5 48 | - 3.6 49 | - 3.7 50 | - 3.8 51 | - 3.9 52 | - 3.10 53 | - 3.11 54 | 55 | 56 | AST Changes 57 | ----------- 58 | 59 | 60 | Python3 61 | ******* 62 | 63 | The AST used by GAST is the same as the one used in Python3.9, with the 64 | notable exception of ``ast.arg`` being replaced by an ``ast.Name`` with an 65 | ``ast.Param`` context. 66 | 67 | The ``name`` field of ``ExceptHandler`` is represented as an ``ast.Name`` with 68 | an ``ast.Store`` context and not a ``str``. 69 | 70 | For minor version before 3.9, please note that ``ExtSlice`` and ``Index`` are 71 | not used. 72 | 73 | For minor version before 3.8, please note that ``Ellipsis``, ``Num``, ``Str``, 74 | ``Bytes`` and ``NamedConstant`` are represented as ``Constant``. 75 | 76 | Python2 77 | ******* 78 | 79 | To cope with Python3 features, several nodes from the Python2 AST are extended 80 | with some new attributes/children, or represented by different nodes 81 | 82 | - ``ModuleDef`` nodes have a ``type_ignores`` attribute. 83 | 84 | - ``FunctionDef`` nodes have a ``returns`` attribute and a ``type_comment`` 85 | attribute. 86 | 87 | - ``ClassDef`` nodes have a ``keywords`` attribute. 88 | 89 | - ``With``'s ``context_expr`` and ``optional_vars`` fields are hold in a 90 | ``withitem`` object. 91 | 92 | - ``For`` nodes have a ``type_comment`` attribute. 93 | 94 | - ``Raise``'s ``type``, ``inst`` and ``tback`` fields are hold in a single 95 | ``exc`` field, using the transformation ``raise E, V, T => raise E(V).with_traceback(T)``. 96 | 97 | - ``TryExcept`` and ``TryFinally`` nodes are merged in the ``Try`` node. 98 | 99 | - ``arguments`` nodes have a ``kwonlyargs`` and ``kw_defaults`` attributes. 100 | 101 | - ``Call`` nodes loose their ``starargs`` attribute, replaced by an 102 | argument wrapped in a ``Starred`` node. They also loose their ``kwargs`` 103 | attribute, wrapped in a ``keyword`` node with the identifier set to 104 | ``None``, as done in Python3. 105 | 106 | - ``comprehension`` nodes have an ``async`` attribute (that is always set 107 | to 0). 108 | 109 | - ``Ellipsis``, ``Num`` and ``Str`` nodes are represented as ``Constant``. 110 | 111 | - ``Subscript`` which don't have any ``Slice`` node as ``slice`` attribute (and 112 | ``Ellipsis`` are not ``Slice`` nodes) no longer hold an ``ExtSlice`` but an 113 | ``Index(Tuple(...))`` instead. 114 | 115 | 116 | Pit Falls 117 | ********* 118 | 119 | - In Python3, ``None``, ``True`` and ``False`` are parsed as ``Constant`` 120 | while they are parsed as regular ``Name`` in Python2. 121 | 122 | ASDL 123 | **** 124 | 125 | This closely matches the one from https://docs.python.org/3/library/ast.html#abstract-grammar, with a few 126 | trade-offs to cope with legacy ASTs. 127 | 128 | .. code:: 129 | 130 | -- ASDL's six builtin types are identifier, int, string, bytes, object, singleton 131 | 132 | module Python 133 | { 134 | mod = Module(stmt* body, type_ignore *type_ignores) 135 | | Interactive(stmt* body) 136 | | Expression(expr body) 137 | | FunctionType(expr* argtypes, expr returns) 138 | 139 | -- not really an actual node but useful in Jython's typesystem. 140 | | Suite(stmt* body) 141 | 142 | stmt = FunctionDef(identifier name, arguments args, 143 | stmt* body, expr* decorator_list, expr? returns, 144 | string? type_comment, type_param* type_params) 145 | | AsyncFunctionDef(identifier name, arguments args, 146 | stmt* body, expr* decorator_list, expr? returns, 147 | string? type_comment, type_param* type_params) 148 | 149 | | ClassDef(identifier name, 150 | expr* bases, 151 | keyword* keywords, 152 | stmt* body, 153 | expr* decorator_list, 154 | type_param* type_params) 155 | | Return(expr? value) 156 | 157 | | Delete(expr* targets) 158 | | Assign(expr* targets, expr value, string? type_comment) 159 | | TypeAlias(expr name, type_param* type_params, expr value) 160 | | AugAssign(expr target, operator op, expr value) 161 | -- 'simple' indicates that we annotate simple name without parens 162 | | AnnAssign(expr target, expr annotation, expr? value, int simple) 163 | 164 | -- not sure if bool is allowed, can always use int 165 | | Print(expr? dest, expr* values, bool nl) 166 | 167 | -- use 'orelse' because else is a keyword in target languages 168 | | For(expr target, expr iter, stmt* body, stmt* orelse, string? type_comment) 169 | | AsyncFor(expr target, expr iter, stmt* body, stmt* orelse, string? type_comment) 170 | | While(expr test, stmt* body, stmt* orelse) 171 | | If(expr test, stmt* body, stmt* orelse) 172 | | With(withitem* items, stmt* body, string? type_comment) 173 | | AsyncWith(withitem* items, stmt* body, string? type_comment) 174 | 175 | | Match(expr subject, match_case* cases) 176 | 177 | | Raise(expr? exc, expr? cause) 178 | | Try(stmt* body, excepthandler* handlers, stmt* orelse, stmt* finalbody) 179 | | TryStar(stmt* body, excepthandler* handlers, stmt* orelse, stmt* finalbody) 180 | | Assert(expr test, expr? msg) 181 | 182 | | Import(alias* names) 183 | | ImportFrom(identifier? module, alias* names, int? level) 184 | 185 | -- Doesn't capture requirement that locals must be 186 | -- defined if globals is 187 | -- still supports use as a function! 188 | | Exec(expr body, expr? globals, expr? locals) 189 | 190 | | Global(identifier* names) 191 | | Nonlocal(identifier* names) 192 | | Expr(expr value) 193 | | Pass | Break | Continue 194 | 195 | -- XXX Jython will be different 196 | -- col_offset is the byte offset in the utf8 string the parser uses 197 | attributes (int lineno, int col_offset) 198 | 199 | -- BoolOp() can use left & right? 200 | expr = BoolOp(boolop op, expr* values) 201 | | NamedExpr(expr target, expr value) 202 | | BinOp(expr left, operator op, expr right) 203 | | UnaryOp(unaryop op, expr operand) 204 | | Lambda(arguments args, expr body) 205 | | IfExp(expr test, expr body, expr orelse) 206 | | Dict(expr* keys, expr* values) 207 | | Set(expr* elts) 208 | | ListComp(expr elt, comprehension* generators) 209 | | SetComp(expr elt, comprehension* generators) 210 | | DictComp(expr key, expr value, comprehension* generators) 211 | | GeneratorExp(expr elt, comprehension* generators) 212 | -- the grammar constrains where yield expressions can occur 213 | | Await(expr value) 214 | | Yield(expr? value) 215 | | YieldFrom(expr value) 216 | -- need sequences for compare to distinguish between 217 | -- x < 4 < 3 and (x < 4) < 3 218 | | Compare(expr left, cmpop* ops, expr* comparators) 219 | | Call(expr func, expr* args, keyword* keywords) 220 | | Repr(expr value) 221 | | FormattedValue(expr value, int? conversion, expr? format_spec) 222 | | JoinedStr(expr* values) 223 | | Constant(constant value, string? kind) 224 | 225 | -- the following expression can appear in assignment context 226 | | Attribute(expr value, identifier attr, expr_context ctx) 227 | | Subscript(expr value, slice slice, expr_context ctx) 228 | | Starred(expr value, expr_context ctx) 229 | | Name(identifier id, expr_context ctx, expr? annotation, 230 | string? type_comment) 231 | | List(expr* elts, expr_context ctx) 232 | | Tuple(expr* elts, expr_context ctx) 233 | 234 | -- col_offset is the byte offset in the utf8 string the parser uses 235 | attributes (int lineno, int col_offset) 236 | 237 | expr_context = Load | Store | Del | AugLoad | AugStore | Param 238 | 239 | slice = Slice(expr? lower, expr? upper, expr? step) 240 | | ExtSlice(slice* dims) 241 | | Index(expr value) 242 | 243 | boolop = And | Or 244 | 245 | operator = Add | Sub | Mult | MatMult | Div | Mod | Pow | LShift 246 | | RShift | BitOr | BitXor | BitAnd | FloorDiv 247 | 248 | unaryop = Invert | Not | UAdd | USub 249 | 250 | cmpop = Eq | NotEq | Lt | LtE | Gt | GtE | Is | IsNot | In | NotIn 251 | 252 | comprehension = (expr target, expr iter, expr* ifs, int is_async) 253 | 254 | excepthandler = ExceptHandler(expr? type, expr? name, stmt* body) 255 | attributes (int lineno, int col_offset, int? end_lineno, int? end_col_offset) 256 | 257 | arguments = (expr* args, expr* posonlyargs, expr? vararg, expr* kwonlyargs, 258 | expr* kw_defaults, expr? kwarg, expr* defaults) 259 | 260 | -- keyword arguments supplied to call (NULL identifier for **kwargs) 261 | keyword = (identifier? arg, expr value) 262 | 263 | -- import name with optional 'as' alias. 264 | alias = (identifier name, identifier? asname) 265 | attributes (int lineno, int col_offset, int? end_lineno, int? end_col_offset) 266 | 267 | withitem = (expr context_expr, expr? optional_vars) 268 | 269 | match_case = (pattern pattern, expr? guard, stmt* body) 270 | 271 | pattern = MatchValue(expr value) 272 | | MatchSingleton(constant value) 273 | | MatchSequence(pattern* patterns) 274 | | MatchMapping(expr* keys, pattern* patterns, identifier? rest) 275 | | MatchClass(expr cls, pattern* patterns, identifier* kwd_attrs, pattern* kwd_patterns) 276 | 277 | | MatchStar(identifier? name) 278 | -- The optional "rest" MatchMapping parameter handles capturing extra mapping keys 279 | 280 | | MatchAs(pattern? pattern, identifier? name) 281 | | MatchOr(pattern* patterns) 282 | 283 | attributes (int lineno, int col_offset, int end_lineno, int end_col_offset) 284 | 285 | type_ignore = TypeIgnore(int lineno, string tag) 286 | 287 | type_param = TypeVar(identifier name, expr? bound) 288 | | ParamSpec(identifier name) 289 | | TypeVarTuple(identifier name) 290 | attributes (int lineno, int col_offset, int end_lineno, int end_col_offset) 291 | } 292 | 293 | 294 | Reporting Bugs 295 | -------------- 296 | 297 | Bugs can be reported through `GitHub issues `_. 298 | 299 | Reporting Security Issues 300 | ------------------------- 301 | 302 | If for some reason, you think your bug is security-related and should be subject 303 | to responsible disclosure, don't hesitate to `contact the maintainer 304 | `_ directly. 305 | -------------------------------------------------------------------------------- /SECURITY.rst: -------------------------------------------------------------------------------- 1 | Reporting Security Issues 2 | ------------------------- 3 | 4 | If for some reason, you think your bug is security-related and should be subject 5 | to responsible disclosure, don't hesitate to `contact the maintainer 6 | `_ directly. 7 | 8 | -------------------------------------------------------------------------------- /gast/__init__.py: -------------------------------------------------------------------------------- 1 | from .gast import * 2 | from .version import __version__ 3 | from ast import NodeVisitor, NodeTransformer, iter_fields 4 | -------------------------------------------------------------------------------- /gast/ast2.py: -------------------------------------------------------------------------------- 1 | from astn import AstToGAst, GAstToAst 2 | import ast 3 | import gast 4 | 5 | 6 | class Ast2ToGAst(AstToGAst): 7 | 8 | # mod 9 | def visit_Module(self, node): 10 | new_node = gast.Module( 11 | self._visit(node.body), 12 | [] # type_ignores 13 | ) 14 | return new_node 15 | 16 | # stmt 17 | def visit_FunctionDef(self, node): 18 | new_node = gast.FunctionDef( 19 | self._visit(node.name), 20 | self._visit(node.args), 21 | self._visit(node.body), 22 | self._visit(node.decorator_list), 23 | None, # returns 24 | None, # type_comment 25 | [], # type_params 26 | ) 27 | gast.copy_location(new_node, node) 28 | new_node.end_lineno = new_node.end_col_offset = None 29 | return new_node 30 | 31 | def visit_ClassDef(self, node): 32 | new_node = gast.ClassDef( 33 | self._visit(node.name), 34 | self._visit(node.bases), 35 | [], # keywords 36 | self._visit(node.body), 37 | self._visit(node.decorator_list), 38 | [], # type_params 39 | ) 40 | 41 | gast.copy_location(new_node, node) 42 | new_node.end_lineno = new_node.end_col_offset = None 43 | return new_node 44 | 45 | def visit_Assign(self, node): 46 | new_node = gast.Assign( 47 | self._visit(node.targets), 48 | self._visit(node.value), 49 | None, # type_comment 50 | ) 51 | 52 | gast.copy_location(new_node, node) 53 | new_node.end_lineno = new_node.end_col_offset = None 54 | return new_node 55 | 56 | def visit_For(self, node): 57 | new_node = gast.For( 58 | self._visit(node.target), 59 | self._visit(node.iter), 60 | self._visit(node.body), 61 | self._visit(node.orelse), 62 | [] # type_comment 63 | ) 64 | gast.copy_location(new_node, node) 65 | new_node.end_lineno = new_node.end_col_offset = None 66 | return new_node 67 | 68 | def visit_With(self, node): 69 | new_node = gast.With( 70 | [gast.withitem( 71 | self._visit(node.context_expr), 72 | self._visit(node.optional_vars) 73 | )], 74 | self._visit(node.body), 75 | None, # type_comment 76 | ) 77 | gast.copy_location(new_node, node) 78 | new_node.end_lineno = new_node.end_col_offset = None 79 | return new_node 80 | 81 | def visit_Raise(self, node): 82 | ntype = self._visit(node.type) 83 | ninst = self._visit(node.inst) 84 | ntback = self._visit(node.tback) 85 | 86 | what = ntype 87 | 88 | if ninst is not None: 89 | what = gast.Call(ntype, [ninst], []) 90 | gast.copy_location(what, node) 91 | what.end_lineno = what.end_col_offset = None 92 | 93 | if ntback is not None: 94 | attr = gast.Attribute(what, 'with_traceback', gast.Load()) 95 | gast.copy_location(attr, node) 96 | attr.end_lineno = attr.end_col_offset = None 97 | 98 | what = gast.Call( 99 | attr, 100 | [ntback], 101 | [] 102 | ) 103 | gast.copy_location(what, node) 104 | what.end_lineno = what.end_col_offset = None 105 | 106 | new_node = gast.Raise(what, None) 107 | 108 | gast.copy_location(new_node, node) 109 | new_node.end_lineno = new_node.end_col_offset = None 110 | return new_node 111 | 112 | def visit_TryExcept(self, node): 113 | new_node = gast.Try( 114 | self._visit(node.body), 115 | self._visit(node.handlers), 116 | self._visit(node.orelse), 117 | [] # finalbody 118 | ) 119 | gast.copy_location(new_node, node) 120 | new_node.end_lineno = new_node.end_col_offset = None 121 | return new_node 122 | 123 | def visit_TryFinally(self, node): 124 | new_node = gast.Try( 125 | self._visit(node.body), 126 | [], # handlers 127 | [], # orelse 128 | self._visit(node.finalbody) 129 | ) 130 | gast.copy_location(new_node, node) 131 | new_node.end_lineno = new_node.end_col_offset = None 132 | return new_node 133 | 134 | # expr 135 | 136 | def visit_Name(self, node): 137 | new_node = gast.Name( 138 | self._visit(node.id), 139 | self._visit(node.ctx), 140 | None, 141 | None, 142 | ) 143 | gast.copy_location(new_node, node) 144 | new_node.end_lineno = new_node.end_col_offset = None 145 | return new_node 146 | 147 | def visit_Num(self, node): 148 | new_node = gast.Constant( 149 | node.n, 150 | None, 151 | ) 152 | gast.copy_location(new_node, node) 153 | new_node.end_lineno = new_node.end_col_offset = None 154 | return new_node 155 | 156 | def visit_Subscript(self, node): 157 | new_slice = self._visit(node.slice) 158 | new_node = gast.Subscript( 159 | self._visit(node.value), 160 | new_slice, 161 | self._visit(node.ctx), 162 | ) 163 | gast.copy_location(new_node, node) 164 | new_node.end_lineno = new_node.end_col_offset = None 165 | return new_node 166 | 167 | def visit_Ellipsis(self, node): 168 | new_node = gast.Constant( 169 | Ellipsis, 170 | None, 171 | ) 172 | gast.copy_location(new_node, node) 173 | new_node.end_lineno = new_node.end_col_offset = None 174 | return new_node 175 | 176 | def visit_Index(self, node): 177 | return self._visit(node.value) 178 | 179 | def visit_ExtSlice(self, node): 180 | new_dims = self._visit(node.dims) 181 | new_node = gast.Tuple(new_dims, gast.Load()) 182 | gast.copy_location(new_node, node) 183 | new_node.end_lineno = new_node.end_col_offset = None 184 | return new_node 185 | 186 | def visit_Str(self, node): 187 | new_node = gast.Constant( 188 | node.s, 189 | None, 190 | ) 191 | gast.copy_location(new_node, node) 192 | new_node.end_lineno = new_node.end_col_offset = None 193 | return new_node 194 | 195 | def visit_Call(self, node): 196 | if node.starargs: 197 | star = gast.Starred(self._visit(node.starargs), gast.Load()) 198 | gast.copy_location(star, node) 199 | star.end_lineno = star.end_col_offset = None 200 | starred = [star] 201 | else: 202 | starred = [] 203 | 204 | if node.kwargs: 205 | kwargs = [gast.keyword(None, self._visit(node.kwargs))] 206 | else: 207 | kwargs = [] 208 | 209 | new_node = gast.Call( 210 | self._visit(node.func), 211 | self._visit(node.args) + starred, 212 | self._visit(node.keywords) + kwargs, 213 | ) 214 | gast.copy_location(new_node, node) 215 | new_node.end_lineno = new_node.end_col_offset = None 216 | return new_node 217 | 218 | def visit_comprehension(self, node): 219 | new_node = gast.comprehension( 220 | target=self._visit(node.target), 221 | iter=self._visit(node.iter), 222 | ifs=self._visit(node.ifs), 223 | is_async=0, 224 | ) 225 | gast.copy_location(new_node, node) 226 | new_node.end_lineno = new_node.end_col_offset = None 227 | return new_node 228 | 229 | # arguments 230 | def visit_arguments(self, node): 231 | # missing locations for vararg and kwarg set at function level 232 | if node.vararg: 233 | vararg = ast.Name(node.vararg, ast.Param()) 234 | else: 235 | vararg = None 236 | 237 | if node.kwarg: 238 | kwarg = ast.Name(node.kwarg, ast.Param()) 239 | else: 240 | kwarg = None 241 | 242 | if node.vararg: 243 | vararg = ast.Name(node.vararg, ast.Param()) 244 | else: 245 | vararg = None 246 | 247 | new_node = gast.arguments( 248 | self._visit(node.args), 249 | [], # posonlyargs 250 | self._visit(vararg), 251 | [], # kwonlyargs 252 | [], # kw_defaults 253 | self._visit(kwarg), 254 | self._visit(node.defaults), 255 | ) 256 | return new_node 257 | 258 | def visit_alias(self, node): 259 | new_node = gast.alias( 260 | self._visit(node.name), 261 | self._visit(node.asname), 262 | ) 263 | new_node.lineno = new_node.col_offset = None 264 | new_node.end_lineno = new_node.end_col_offset = None 265 | return new_node 266 | 267 | 268 | class GAstToAst2(GAstToAst): 269 | 270 | # mod 271 | def visit_Module(self, node): 272 | new_node = ast.Module(self._visit(node.body)) 273 | return new_node 274 | 275 | # stmt 276 | def visit_FunctionDef(self, node): 277 | new_node = ast.FunctionDef( 278 | self._visit(node.name), 279 | self._visit(node.args), 280 | self._visit(node.body), 281 | self._visit(node.decorator_list), 282 | ) 283 | # because node.args doesn't have any location to copy from 284 | if node.args.vararg: 285 | ast.copy_location(node.args.vararg, node) 286 | if node.args.kwarg: 287 | ast.copy_location(node.args.kwarg, node) 288 | 289 | ast.copy_location(new_node, node) 290 | return new_node 291 | 292 | def visit_ClassDef(self, node): 293 | new_node = ast.ClassDef( 294 | self._visit(node.name), 295 | self._visit(node.bases), 296 | self._visit(node.body), 297 | self._visit(node.decorator_list), 298 | ) 299 | 300 | ast.copy_location(new_node, node) 301 | return new_node 302 | 303 | def visit_Assign(self, node): 304 | new_node = ast.Assign( 305 | self._visit(node.targets), 306 | self._visit(node.value), 307 | ) 308 | 309 | ast.copy_location(new_node, node) 310 | return new_node 311 | 312 | def visit_For(self, node): 313 | new_node = ast.For( 314 | self._visit(node.target), 315 | self._visit(node.iter), 316 | self._visit(node.body), 317 | self._visit(node.orelse), 318 | ) 319 | 320 | ast.copy_location(new_node, node) 321 | return new_node 322 | 323 | def visit_With(self, node): 324 | new_node = ast.With( 325 | self._visit(node.items[0].context_expr), 326 | self._visit(node.items[0].optional_vars), 327 | self._visit(node.body) 328 | ) 329 | ast.copy_location(new_node, node) 330 | return new_node 331 | 332 | def visit_Raise(self, node): 333 | if isinstance(node.exc, gast.Call) and \ 334 | isinstance(node.exc.func, gast.Attribute) and \ 335 | node.exc.func.attr == 'with_traceback': 336 | raised = self._visit(node.exc.func.value) 337 | traceback = self._visit(node.exc.args[0]) 338 | else: 339 | raised = self._visit(node.exc) 340 | traceback = None 341 | new_node = ast.Raise(raised, None, traceback) 342 | ast.copy_location(new_node, node) 343 | return new_node 344 | 345 | def visit_Try(self, node): 346 | if node.finalbody: 347 | new_node = ast.TryFinally( 348 | self._visit(node.body), 349 | self._visit(node.finalbody) 350 | ) 351 | else: 352 | new_node = ast.TryExcept( 353 | self._visit(node.body), 354 | self._visit(node.handlers), 355 | self._visit(node.orelse), 356 | ) 357 | ast.copy_location(new_node, node) 358 | return new_node 359 | 360 | # expr 361 | 362 | def visit_Name(self, node): 363 | new_node = ast.Name( 364 | self._visit(node.id), 365 | self._visit(node.ctx), 366 | ) 367 | ast.copy_location(new_node, node) 368 | return new_node 369 | 370 | def visit_Constant(self, node): 371 | if isinstance(node.value, (bool, int, long, float, complex)): 372 | new_node = ast.Num(node.value) 373 | elif node.value is Ellipsis: 374 | new_node = ast.Ellipsis() 375 | else: 376 | new_node = ast.Str(node.value) 377 | ast.copy_location(new_node, node) 378 | return new_node 379 | 380 | def visit_Subscript(self, node): 381 | def adjust_slice(s): 382 | if isinstance(s, (ast.Slice, ast.Ellipsis)): 383 | return s 384 | else: 385 | return ast.Index(s) 386 | if isinstance(node.slice, gast.Tuple): 387 | new_slice = ast.ExtSlice([adjust_slice(self._visit(elt)) 388 | for elt in node.slice.elts]) 389 | else: 390 | new_slice = adjust_slice(self._visit(node.slice)) 391 | ast.copy_location(new_slice, node.slice) 392 | 393 | new_node = ast.Subscript( 394 | self._visit(node.value), 395 | new_slice, 396 | self._visit(node.ctx), 397 | ) 398 | ast.copy_location(new_node, node) 399 | new_node.end_lineno = new_node.end_col_offset = None 400 | return new_node 401 | 402 | def visit_Call(self, node): 403 | if node.args and isinstance(node.args[-1], gast.Starred): 404 | args = node.args[:-1] 405 | starargs = node.args[-1].value 406 | else: 407 | args = node.args 408 | starargs = None 409 | 410 | if node.keywords and node.keywords[-1].arg is None: 411 | keywords = node.keywords[:-1] 412 | kwargs = node.keywords[-1].value 413 | else: 414 | keywords = node.keywords 415 | kwargs = None 416 | 417 | new_node = ast.Call( 418 | self._visit(node.func), 419 | self._visit(args), 420 | self._visit(keywords), 421 | self._visit(starargs), 422 | self._visit(kwargs), 423 | ) 424 | ast.copy_location(new_node, node) 425 | return new_node 426 | 427 | def visit_arg(self, node): 428 | new_node = ast.Name(node.arg, ast.Param()) 429 | ast.copy_location(new_node, node) 430 | return new_node 431 | 432 | # arguments 433 | def visit_arguments(self, node): 434 | vararg = node.vararg and node.vararg.id 435 | kwarg = node.kwarg and node.kwarg.id 436 | 437 | new_node = ast.arguments( 438 | self._visit(node.args), 439 | self._visit(vararg), 440 | self._visit(kwarg), 441 | self._visit(node.defaults), 442 | ) 443 | return new_node 444 | 445 | def visit_alias(self, node): 446 | new_node = ast.alias( 447 | self._visit(node.name), 448 | self._visit(node.asname) 449 | ) 450 | return new_node 451 | 452 | 453 | def ast_to_gast(node): 454 | return Ast2ToGAst().visit(node) 455 | 456 | 457 | def gast_to_ast(node): 458 | return GAstToAst2().visit(node) 459 | -------------------------------------------------------------------------------- /gast/ast3.py: -------------------------------------------------------------------------------- 1 | from gast.astn import AstToGAst, GAstToAst 2 | import gast 3 | import ast 4 | import sys 5 | 6 | 7 | class Ast3ToGAst(AstToGAst): 8 | if sys.version_info.minor < 10: 9 | 10 | def visit_alias(self, node): 11 | new_node = gast.alias( 12 | self._visit(node.name), 13 | self._visit(node.asname), 14 | ) 15 | new_node.lineno = new_node.col_offset = None 16 | new_node.end_lineno = new_node.end_col_offset = None 17 | return new_node 18 | 19 | if sys.version_info.minor < 9: 20 | 21 | def visit_ExtSlice(self, node): 22 | new_node = gast.Tuple(self._visit(node.dims), gast.Load()) 23 | return gast.copy_location(new_node, node) 24 | 25 | def visit_Index(self, node): 26 | return self._visit(node.value) 27 | 28 | def visit_Assign(self, node): 29 | new_node = gast.Assign( 30 | self._visit(node.targets), 31 | self._visit(node.value), 32 | None, # type_comment 33 | ) 34 | 35 | gast.copy_location(new_node, node) 36 | new_node.end_lineno = new_node.end_col_offset = None 37 | return new_node 38 | 39 | if sys.version_info.minor < 8: 40 | def visit_Module(self, node): 41 | new_node = gast.Module( 42 | self._visit(node.body), 43 | [] # type_ignores 44 | ) 45 | return new_node 46 | 47 | def visit_Num(self, node): 48 | new_node = gast.Constant( 49 | node.n, 50 | None, 51 | ) 52 | return gast.copy_location(new_node, node) 53 | 54 | def visit_Ellipsis(self, node): 55 | new_node = gast.Constant( 56 | Ellipsis, 57 | None, 58 | ) 59 | gast.copy_location(new_node, node) 60 | new_node.end_lineno = new_node.end_col_offset = None 61 | return new_node 62 | 63 | def visit_Str(self, node): 64 | new_node = gast.Constant( 65 | node.s, 66 | None, 67 | ) 68 | return gast.copy_location(new_node, node) 69 | 70 | def visit_Bytes(self, node): 71 | new_node = gast.Constant( 72 | node.s, 73 | None, 74 | ) 75 | return gast.copy_location(new_node, node) 76 | 77 | def visit_FunctionDef(self, node): 78 | new_node = gast.FunctionDef( 79 | self._visit(node.name), 80 | self._visit(node.args), 81 | self._visit(node.body), 82 | self._visit(node.decorator_list), 83 | self._visit(node.returns), 84 | None, # type_comment 85 | [], # type_params 86 | ) 87 | return gast.copy_location(new_node, node) 88 | 89 | def visit_AsyncFunctionDef(self, node): 90 | new_node = gast.AsyncFunctionDef( 91 | self._visit(node.name), 92 | self._visit(node.args), 93 | self._visit(node.body), 94 | self._visit(node.decorator_list), 95 | self._visit(node.returns), 96 | None, # type_comment 97 | [], # type_params 98 | ) 99 | return gast.copy_location(new_node, node) 100 | 101 | def visit_For(self, node): 102 | new_node = gast.For( 103 | self._visit(node.target), 104 | self._visit(node.iter), 105 | self._visit(node.body), 106 | self._visit(node.orelse), 107 | None, # type_comment 108 | ) 109 | return gast.copy_location(new_node, node) 110 | 111 | def visit_AsyncFor(self, node): 112 | new_node = gast.AsyncFor( 113 | self._visit(node.target), 114 | self._visit(node.iter), 115 | self._visit(node.body), 116 | self._visit(node.orelse), 117 | None, # type_comment 118 | ) 119 | return gast.copy_location(new_node, node) 120 | 121 | def visit_With(self, node): 122 | new_node = gast.With( 123 | self._visit(node.items), 124 | self._visit(node.body), 125 | None, # type_comment 126 | ) 127 | return gast.copy_location(new_node, node) 128 | 129 | def visit_AsyncWith(self, node): 130 | new_node = gast.AsyncWith( 131 | self._visit(node.items), 132 | self._visit(node.body), 133 | None, # type_comment 134 | ) 135 | return gast.copy_location(new_node, node) 136 | 137 | def visit_Call(self, node): 138 | if sys.version_info.minor < 5: 139 | if node.starargs: 140 | star = gast.Starred(self._visit(node.starargs), 141 | gast.Load()) 142 | gast.copy_location(star, node) 143 | starred = [star] 144 | else: 145 | starred = [] 146 | 147 | if node.kwargs: 148 | kw = gast.keyword(None, self._visit(node.kwargs)) 149 | gast.copy_location(kw, node.kwargs) 150 | kwargs = [kw] 151 | else: 152 | kwargs = [] 153 | else: 154 | starred = kwargs = [] 155 | 156 | new_node = gast.Call( 157 | self._visit(node.func), 158 | self._visit(node.args) + starred, 159 | self._visit(node.keywords) + kwargs, 160 | ) 161 | return gast.copy_location(new_node, node) 162 | 163 | def visit_NameConstant(self, node): 164 | if node.value is None: 165 | new_node = gast.Constant(None, None) 166 | elif node.value is True: 167 | new_node = gast.Constant(True, None) 168 | elif node.value is False: 169 | new_node = gast.Constant(False, None) 170 | return gast.copy_location(new_node, node) 171 | 172 | def visit_arguments(self, node): 173 | new_node = gast.arguments( 174 | self._visit(node.args), 175 | [], # posonlyargs 176 | self._visit(node.vararg), 177 | self._visit(node.kwonlyargs), 178 | self._visit(node.kw_defaults), 179 | self._visit(node.kwarg), 180 | self._visit(node.defaults), 181 | ) 182 | return gast.copy_location(new_node, node) 183 | 184 | def visit_Name(self, node): 185 | new_node = gast.Name( 186 | node.id, # micro-optimization here, don't call self._visit 187 | self._visit(node.ctx), 188 | None, 189 | None, 190 | ) 191 | return ast.copy_location(new_node, node) 192 | 193 | def visit_arg(self, node): 194 | if sys.version_info.minor < 8: 195 | extra_arg = None 196 | else: 197 | extra_arg = self._visit(node.type_comment) 198 | 199 | new_node = gast.Name( 200 | node.arg, # micro-optimization here, don't call self._visit 201 | gast.Param(), 202 | self._visit(node.annotation), 203 | extra_arg # type_comment 204 | ) 205 | return ast.copy_location(new_node, node) 206 | 207 | def visit_ExceptHandler(self, node): 208 | if node.name: 209 | new_node = gast.ExceptHandler( 210 | self._visit(node.type), 211 | gast.Name(node.name, gast.Store(), None, None), 212 | self._visit(node.body)) 213 | return ast.copy_location(new_node, node) 214 | else: 215 | return self.generic_visit(node) 216 | 217 | if sys.version_info.minor < 6: 218 | 219 | def visit_comprehension(self, node): 220 | new_node = gast.comprehension( 221 | target=self._visit(node.target), 222 | iter=self._visit(node.iter), 223 | ifs=self._visit(node.ifs), 224 | is_async=0, 225 | ) 226 | return ast.copy_location(new_node, node) 227 | 228 | if 8 <= sys.version_info.minor < 12: 229 | def visit_FunctionDef(self, node): 230 | new_node = gast.FunctionDef( 231 | self._visit(node.name), 232 | self._visit(node.args), 233 | self._visit(node.body), 234 | self._visit(node.decorator_list), 235 | self._visit(node.returns), 236 | self._visit(node.type_comment), 237 | [], # type_params 238 | ) 239 | return gast.copy_location(new_node, node) 240 | 241 | def visit_AsyncFunctionDef(self, node): 242 | new_node = gast.AsyncFunctionDef( 243 | self._visit(node.name), 244 | self._visit(node.args), 245 | self._visit(node.body), 246 | self._visit(node.decorator_list), 247 | self._visit(node.returns), 248 | self._visit(node.type_comment), 249 | [], # type_params 250 | ) 251 | return gast.copy_location(new_node, node) 252 | 253 | if sys.version_info.minor < 12: 254 | 255 | def visit_ClassDef(self, node): 256 | new_node = gast.ClassDef( 257 | self._visit(node.name), 258 | self._visit(node.bases), 259 | self._visit(node.keywords), 260 | self._visit(node.body), 261 | self._visit(node.decorator_list), 262 | [], # type_params 263 | ) 264 | return gast.copy_location(new_node, node) 265 | 266 | 267 | class GAstToAst3(GAstToAst): 268 | if sys.version_info.minor < 10: 269 | def visit_alias(self, node): 270 | new_node = ast.alias( 271 | self._visit(node.name), 272 | self._visit(node.asname) 273 | ) 274 | return new_node 275 | 276 | if sys.version_info.minor < 9: 277 | def visit_Subscript(self, node): 278 | def adjust_slice(s): 279 | if isinstance(s, ast.Slice): 280 | return s 281 | else: 282 | return ast.Index(s) 283 | if isinstance(node.slice, gast.Tuple): 284 | if any(isinstance(elt, gast.slice) for elt in node.slice.elts): 285 | new_slice = ast.ExtSlice( 286 | [adjust_slice(x) for x in 287 | self._visit(node.slice.elts)]) 288 | else: 289 | value = ast.Tuple(self._visit(node.slice.elts), ast.Load()) 290 | ast.copy_location(value, node.slice) 291 | new_slice = ast.Index(value) 292 | else: 293 | new_slice = adjust_slice(self._visit(node.slice)) 294 | ast.copy_location(new_slice, node.slice) 295 | 296 | new_node = ast.Subscript( 297 | self._visit(node.value), 298 | new_slice, 299 | self._visit(node.ctx), 300 | ) 301 | return ast.copy_location(new_node, node) 302 | 303 | def visit_Assign(self, node): 304 | new_node = ast.Assign( 305 | self._visit(node.targets), 306 | self._visit(node.value), 307 | ) 308 | 309 | return ast.copy_location(new_node, node) 310 | 311 | if sys.version_info.minor < 8: 312 | 313 | def visit_Module(self, node): 314 | new_node = ast.Module(self._visit(node.body)) 315 | return new_node 316 | 317 | def visit_Constant(self, node): 318 | if node.value is None: 319 | new_node = ast.NameConstant(node.value) 320 | elif node.value is Ellipsis: 321 | new_node = ast.Ellipsis() 322 | elif isinstance(node.value, bool): 323 | new_node = ast.NameConstant(node.value) 324 | elif isinstance(node.value, (int, float, complex)): 325 | new_node = ast.Num(node.value) 326 | elif isinstance(node.value, str): 327 | new_node = ast.Str(node.value) 328 | else: 329 | new_node = ast.Bytes(node.value) 330 | return ast.copy_location(new_node, node) 331 | 332 | def _make_arg(self, node): 333 | if node is None: 334 | return None 335 | 336 | if sys.version_info.minor < 8: 337 | extra_args = tuple() 338 | else: 339 | extra_args = self._visit(node.type_comment), 340 | 341 | new_node = ast.arg( 342 | self._visit(node.id), 343 | self._visit(node.annotation), 344 | *extra_args 345 | ) 346 | return ast.copy_location(new_node, node) 347 | 348 | def visit_Name(self, node): 349 | new_node = ast.Name( 350 | self._visit(node.id), 351 | self._visit(node.ctx), 352 | ) 353 | return ast.copy_location(new_node, node) 354 | 355 | def visit_ExceptHandler(self, node): 356 | if node.name: 357 | new_node = ast.ExceptHandler( 358 | self._visit(node.type), 359 | node.name.id, 360 | self._visit(node.body)) 361 | return ast.copy_location(new_node, node) 362 | else: 363 | return self.generic_visit(node) 364 | 365 | if sys.version_info.minor < 5: 366 | 367 | def visit_Call(self, node): 368 | if node.args and isinstance(node.args[-1], gast.Starred): 369 | args = node.args[:-1] 370 | starargs = node.args[-1].value 371 | else: 372 | args = node.args 373 | starargs = None 374 | 375 | if node.keywords and node.keywords[-1].arg is None: 376 | keywords = node.keywords[:-1] 377 | kwargs = node.keywords[-1].value 378 | else: 379 | keywords = node.keywords 380 | kwargs = None 381 | 382 | new_node = ast.Call( 383 | self._visit(node.func), 384 | self._visit(args), 385 | self._visit(keywords), 386 | self._visit(starargs), 387 | self._visit(kwargs), 388 | ) 389 | return ast.copy_location(new_node, node) 390 | 391 | def visit_ClassDef(self, node): 392 | self.generic_visit(node) 393 | new_node = ast.ClassDef( 394 | name=self._visit(node.name), 395 | bases=self._visit(node.bases), 396 | keywords=self._visit(node.keywords), 397 | body=self._visit(node.body), 398 | decorator_list=self._visit(node.decorator_list), 399 | starargs=None, 400 | kwargs=None, 401 | ) 402 | return ast.copy_location(new_node, node) 403 | 404 | elif sys.version_info.minor < 8: 405 | 406 | def visit_FunctionDef(self, node): 407 | new_node = ast.FunctionDef( 408 | self._visit(node.name), 409 | self._visit(node.args), 410 | self._visit(node.body), 411 | self._visit(node.decorator_list), 412 | self._visit(node.returns), 413 | ) 414 | return ast.copy_location(new_node, node) 415 | 416 | def visit_AsyncFunctionDef(self, node): 417 | new_node = ast.AsyncFunctionDef( 418 | self._visit(node.name), 419 | self._visit(node.args), 420 | self._visit(node.body), 421 | self._visit(node.decorator_list), 422 | self._visit(node.returns), 423 | ) 424 | return ast.copy_location(new_node, node) 425 | 426 | def visit_For(self, node): 427 | new_node = ast.For( 428 | self._visit(node.target), 429 | self._visit(node.iter), 430 | self._visit(node.body), 431 | self._visit(node.orelse), 432 | ) 433 | return ast.copy_location(new_node, node) 434 | 435 | def visit_AsyncFor(self, node): 436 | new_node = ast.AsyncFor( 437 | self._visit(node.target), 438 | self._visit(node.iter), 439 | self._visit(node.body), 440 | self._visit(node.orelse), 441 | None, # type_comment 442 | ) 443 | return ast.copy_location(new_node, node) 444 | 445 | def visit_With(self, node): 446 | new_node = ast.With( 447 | self._visit(node.items), 448 | self._visit(node.body), 449 | ) 450 | return ast.copy_location(new_node, node) 451 | 452 | def visit_AsyncWith(self, node): 453 | new_node = ast.AsyncWith( 454 | self._visit(node.items), 455 | self._visit(node.body), 456 | ) 457 | return ast.copy_location(new_node, node) 458 | 459 | def visit_Call(self, node): 460 | new_node = ast.Call( 461 | self._visit(node.func), 462 | self._visit(node.args), 463 | self._visit(node.keywords), 464 | ) 465 | return ast.copy_location(new_node, node) 466 | if 5 <= sys.version_info.minor < 12: 467 | def visit_ClassDef(self, node): 468 | new_node = ast.ClassDef( 469 | self._visit(node.name), 470 | self._visit(node.bases), 471 | self._visit(node.keywords), 472 | self._visit(node.body), 473 | self._visit(node.decorator_list), 474 | ) 475 | return ast.copy_location(new_node, node) 476 | 477 | if 8 <= sys.version_info.minor < 12: 478 | def visit_FunctionDef(self, node): 479 | new_node = ast.FunctionDef( 480 | self._visit(node.name), 481 | self._visit(node.args), 482 | self._visit(node.body), 483 | self._visit(node.decorator_list), 484 | self._visit(node.returns), 485 | self._visit(node.type_comment), 486 | ) 487 | return ast.copy_location(new_node, node) 488 | 489 | def visit_AsyncFunctionDef(self, node): 490 | new_node = ast.AsyncFunctionDef( 491 | self._visit(node.name), 492 | self._visit(node.args), 493 | self._visit(node.body), 494 | self._visit(node.decorator_list), 495 | self._visit(node.returns), 496 | self._visit(node.type_comment), 497 | ) 498 | return ast.copy_location(new_node, node) 499 | 500 | 501 | 502 | def visit_arguments(self, node): 503 | extra_args = [self._make_arg(node.vararg), 504 | [self._make_arg(n) for n in node.kwonlyargs], 505 | self._visit(node.kw_defaults), 506 | self._make_arg(node.kwarg), 507 | self._visit(node.defaults), ] 508 | if sys.version_info.minor >= 8: 509 | new_node = ast.arguments( 510 | [self._make_arg(arg) for arg in node.posonlyargs], 511 | [self._make_arg(n) for n in node.args], 512 | *extra_args 513 | ) 514 | else: 515 | new_node = ast.arguments( 516 | [self._make_arg(n) for n in node.args], 517 | *extra_args 518 | ) 519 | return new_node 520 | 521 | 522 | def ast_to_gast(node): 523 | return Ast3ToGAst().visit(node) 524 | 525 | 526 | def gast_to_ast(node): 527 | return GAstToAst3().visit(node) 528 | -------------------------------------------------------------------------------- /gast/astn.py: -------------------------------------------------------------------------------- 1 | import ast 2 | import gast 3 | 4 | 5 | def _generate_translators(to): 6 | 7 | class Translator(ast.NodeTransformer): 8 | 9 | def _visit(self, node): 10 | if isinstance(node, ast.AST): 11 | return self.visit(node) 12 | elif isinstance(node, list): 13 | return [self._visit(n) for n in node] 14 | else: 15 | return node 16 | 17 | def generic_visit(self, node): 18 | class_name = type(node).__name__ 19 | if not hasattr(to, class_name): 20 | # handle nodes that are not part of the AST 21 | return 22 | cls = getattr(to, class_name) 23 | new_node = cls( 24 | **{ 25 | field: self._visit(getattr(node, field)) 26 | for field in node._fields 27 | } 28 | ) 29 | 30 | for attr in node._attributes: 31 | try: 32 | setattr(new_node, attr, getattr(node, attr)) 33 | except AttributeError: 34 | pass 35 | return new_node 36 | 37 | return Translator 38 | 39 | 40 | AstToGAst = _generate_translators(gast) 41 | 42 | GAstToAst = _generate_translators(ast) 43 | -------------------------------------------------------------------------------- /gast/gast.py: -------------------------------------------------------------------------------- 1 | import sys as _sys 2 | import ast as _ast 3 | from ast import boolop, cmpop, excepthandler, expr, expr_context, operator 4 | from ast import slice, stmt, unaryop, mod, AST 5 | from ast import iter_child_nodes, walk 6 | 7 | try: 8 | from ast import TypeIgnore 9 | except ImportError: 10 | class TypeIgnore(AST): 11 | pass 12 | 13 | try: 14 | from ast import pattern 15 | except ImportError: 16 | class pattern(AST): 17 | pass 18 | 19 | 20 | try: 21 | from ast import type_param 22 | except ImportError: 23 | class type_param(AST): 24 | pass 25 | 26 | 27 | def _make_node(Name, Fields, Attributes, Bases): 28 | 29 | # This constructor is used a lot during conversion from ast to gast, 30 | # then as the primary way to build ast nodes. So we tried to optimized it 31 | # for speed and not for readability. 32 | def create_node(self, *args, **kwargs): 33 | if len(args) > len(Fields): 34 | raise TypeError( 35 | "{} constructor takes at most {} positional arguments". 36 | format(Name, len(Fields))) 37 | 38 | # it's faster to iterate rather than zipping or enumerate 39 | for i in range(len(args)): 40 | setattr(self, Fields[i], args[i]) 41 | if kwargs: # cold branch 42 | self.__dict__.update(kwargs) 43 | 44 | setattr(_sys.modules[__name__], 45 | Name, 46 | type(Name, 47 | Bases, 48 | {'__init__': create_node, 49 | '_fields': Fields, 50 | '_attributes': Attributes})) 51 | 52 | 53 | _nodes = ( 54 | # mod 55 | ('Module', (('body', 'type_ignores'), (), (mod,))), 56 | ('Interactive', (('body',), (), (mod,))), 57 | ('Expression', (('body',), (), (mod,))), 58 | ('FunctionType', (('argtypes', 'returns'), (), (mod,))), 59 | ('Suite', (('body',), (), (mod,))), 60 | 61 | # stmt 62 | ('FunctionDef', (('name', 'args', 'body', 'decorator_list', 'returns', 63 | 'type_comment', 'type_params'), 64 | ('lineno', 'col_offset', 'end_lineno', 'end_col_offset',), 65 | (stmt,))), 66 | ('AsyncFunctionDef', (('name', 'args', 'body', 'decorator_list', 'returns', 67 | 'type_comment', 'type_params',), 68 | ('lineno', 'col_offset', 69 | 'end_lineno', 'end_col_offset',), 70 | (stmt,))), 71 | ('ClassDef', (('name', 'bases', 'keywords', 'body', 'decorator_list', 72 | 'type_params',), 73 | ('lineno', 'col_offset', 'end_lineno', 'end_col_offset',), 74 | (stmt,))), 75 | ('Return', (('value',), 76 | ('lineno', 'col_offset', 'end_lineno', 'end_col_offset',), 77 | (stmt,))), 78 | ('Delete', (('targets',), 79 | ('lineno', 'col_offset', 'end_lineno', 'end_col_offset',), 80 | (stmt,))), 81 | ('Assign', (('targets', 'value', 'type_comment'), 82 | ('lineno', 'col_offset', 'end_lineno', 'end_col_offset',), 83 | (stmt,))), 84 | ('TypeAlias', (('name', 'type_params', 'value'), 85 | ('lineno', 'col_offset', 'end_lineno', 'end_col_offset',), 86 | (stmt,))), 87 | ('AugAssign', (('target', 'op', 'value',), 88 | ('lineno', 'col_offset', 'end_lineno', 'end_col_offset',), 89 | (stmt,))), 90 | ('AnnAssign', (('target', 'annotation', 'value', 'simple',), 91 | ('lineno', 'col_offset', 'end_lineno', 'end_col_offset',), 92 | (stmt,))), 93 | ('Print', (('dest', 'values', 'nl',), 94 | ('lineno', 'col_offset', 'end_lineno', 'end_col_offset',), 95 | (stmt,))), 96 | ('For', (('target', 'iter', 'body', 'orelse', 'type_comment'), 97 | ('lineno', 'col_offset', 'end_lineno', 'end_col_offset',), 98 | (stmt,))), 99 | ('AsyncFor', (('target', 'iter', 'body', 'orelse', 'type_comment'), 100 | ('lineno', 'col_offset', 'end_lineno', 'end_col_offset',), 101 | (stmt,))), 102 | ('While', (('test', 'body', 'orelse',), 103 | ('lineno', 'col_offset', 'end_lineno', 'end_col_offset',), 104 | (stmt,))), 105 | ('If', (('test', 'body', 'orelse',), 106 | ('lineno', 'col_offset', 'end_lineno', 'end_col_offset',), 107 | (stmt,))), 108 | ('With', (('items', 'body', 'type_comment'), 109 | ('lineno', 'col_offset', 'end_lineno', 'end_col_offset',), 110 | (stmt,))), 111 | ('AsyncWith', (('items', 'body', 'type_comment'), 112 | ('lineno', 'col_offset', 'end_lineno', 'end_col_offset',), 113 | (stmt,))), 114 | ('Match', (('subject', 'cases'), 115 | ('lineno', 'col_offset', 'end_lineno', 'end_col_offset',), 116 | (stmt,))), 117 | ('Raise', (('exc', 'cause',), 118 | ('lineno', 'col_offset', 'end_lineno', 'end_col_offset',), 119 | (stmt,))), 120 | ('Try', (('body', 'handlers', 'orelse', 'finalbody',), 121 | ('lineno', 'col_offset', 'end_lineno', 'end_col_offset',), 122 | (stmt,))), 123 | ('TryStar', (('body', 'handlers', 'orelse', 'finalbody',), 124 | ('lineno', 'col_offset', 'end_lineno', 'end_col_offset',), 125 | (stmt,))), 126 | ('Assert', (('test', 'msg',), 127 | ('lineno', 'col_offset', 'end_lineno', 'end_col_offset',), 128 | (stmt,))), 129 | ('Import', (('names',), 130 | ('lineno', 'col_offset', 'end_lineno', 'end_col_offset',), 131 | (stmt,))), 132 | ('ImportFrom', (('module', 'names', 'level',), 133 | ('lineno', 'col_offset', 'end_lineno', 'end_col_offset',), 134 | (stmt,))), 135 | ('Exec', (('body', 'globals', 'locals',), 136 | ('lineno', 'col_offset', 'end_lineno', 'end_col_offset',), 137 | (stmt,))), 138 | ('Global', (('names',), 139 | ('lineno', 'col_offset', 'end_lineno', 'end_col_offset',), 140 | (stmt,))), 141 | ('Nonlocal', (('names',), 142 | ('lineno', 'col_offset', 'end_lineno', 'end_col_offset',), 143 | (stmt,))), 144 | ('Expr', (('value',), 145 | ('lineno', 'col_offset', 'end_lineno', 'end_col_offset',), 146 | (stmt,))), 147 | ('Pass', ((), ('lineno', 'col_offset', 'end_lineno', 'end_col_offset',), 148 | (stmt,))), 149 | ('Break', ((), ('lineno', 'col_offset', 'end_lineno', 'end_col_offset',), 150 | (stmt,))), 151 | ('Continue', ((), 152 | ('lineno', 'col_offset', 'end_lineno', 'end_col_offset',), 153 | (stmt,))), 154 | 155 | # expr 156 | 157 | ('BoolOp', (('op', 'values',), 158 | ('lineno', 'col_offset', 'end_lineno', 'end_col_offset',), 159 | (expr,))), 160 | ('NamedExpr', (('target', 'value',), 161 | ('lineno', 'col_offset', 'end_lineno', 'end_col_offset',), 162 | (expr,))), 163 | ('BinOp', (('left', 'op', 'right',), 164 | ('lineno', 'col_offset', 'end_lineno', 'end_col_offset',), 165 | (expr,))), 166 | ('UnaryOp', (('op', 'operand',), 167 | ('lineno', 'col_offset', 'end_lineno', 'end_col_offset',), 168 | (expr,))), 169 | ('Lambda', (('args', 'body',), 170 | ('lineno', 'col_offset', 'end_lineno', 'end_col_offset',), 171 | (expr,))), 172 | ('IfExp', (('test', 'body', 'orelse',), 173 | ('lineno', 'col_offset', 'end_lineno', 'end_col_offset',), 174 | (expr,))), 175 | ('Dict', (('keys', 'values',), 176 | ('lineno', 'col_offset', 'end_lineno', 'end_col_offset',), 177 | (expr,))), 178 | ('Set', (('elts',), 179 | ('lineno', 'col_offset', 'end_lineno', 'end_col_offset',), 180 | (expr,))), 181 | ('ListComp', (('elt', 'generators',), 182 | ('lineno', 'col_offset', 'end_lineno', 'end_col_offset',), 183 | (expr,))), 184 | ('SetComp', (('elt', 'generators',), 185 | ('lineno', 'col_offset', 'end_lineno', 'end_col_offset',), 186 | (expr,))), 187 | ('DictComp', (('key', 'value', 'generators',), 188 | ('lineno', 'col_offset', 'end_lineno', 'end_col_offset',), 189 | (expr,))), 190 | ('GeneratorExp', (('elt', 'generators',), 191 | ('lineno', 'col_offset', 192 | 'end_lineno', 'end_col_offset',), 193 | (expr,))), 194 | ('Await', (('value',), 195 | ('lineno', 'col_offset', 'end_lineno', 'end_col_offset',), 196 | (expr,))), 197 | ('Yield', (('value',), 198 | ('lineno', 'col_offset', 'end_lineno', 'end_col_offset',), 199 | (expr,))), 200 | ('YieldFrom', (('value',), 201 | ('lineno', 'col_offset', 'end_lineno', 'end_col_offset',), 202 | (expr,))), 203 | ('Compare', (('left', 'ops', 'comparators',), 204 | ('lineno', 'col_offset', 'end_lineno', 'end_col_offset',), 205 | (expr,))), 206 | ('Call', (('func', 'args', 'keywords',), 207 | ('lineno', 'col_offset', 'end_lineno', 'end_col_offset',), 208 | (expr,))), 209 | ('Repr', (('value',), 210 | ('lineno', 'col_offset', 'end_lineno', 'end_col_offset',), 211 | (expr,))), 212 | ('FormattedValue', (('value', 'conversion', 'format_spec',), 213 | ('lineno', 'col_offset', 214 | 'end_lineno', 'end_col_offset',), 215 | (expr,))), 216 | ('JoinedStr', (('values',), 217 | ('lineno', 'col_offset', 'end_lineno', 'end_col_offset',), 218 | (expr,))), 219 | ('Constant', (('value', 'kind'), 220 | ('lineno', 'col_offset', 'end_lineno', 'end_col_offset',), 221 | (expr,))), 222 | ('Attribute', (('value', 'attr', 'ctx',), 223 | ('lineno', 'col_offset', 'end_lineno', 'end_col_offset',), 224 | (expr,))), 225 | ('Subscript', (('value', 'slice', 'ctx',), 226 | ('lineno', 'col_offset', 'end_lineno', 'end_col_offset',), 227 | (expr,))), 228 | ('Starred', (('value', 'ctx',), 229 | ('lineno', 'col_offset', 'end_lineno', 'end_col_offset',), 230 | (expr,))), 231 | ('Name', (('id', 'ctx', 'annotation', 'type_comment'), 232 | ('lineno', 'col_offset', 'end_lineno', 'end_col_offset',), 233 | (expr,))), 234 | ('List', (('elts', 'ctx',), 235 | ('lineno', 'col_offset', 'end_lineno', 'end_col_offset',), 236 | (expr,))), 237 | ('Tuple', (('elts', 'ctx',), 238 | ('lineno', 'col_offset', 'end_lineno', 'end_col_offset',), 239 | (expr,))), 240 | 241 | # expr_context 242 | ('Load', ((), (), (expr_context,))), 243 | ('Store', ((), (), (expr_context,))), 244 | ('Del', ((), (), (expr_context,))), 245 | ('AugLoad', ((), (), (expr_context,))), 246 | ('AugStore', ((), (), (expr_context,))), 247 | ('Param', ((), (), (expr_context,))), 248 | 249 | # slice 250 | ('Slice', (('lower', 'upper', 'step'), 251 | ('lineno', 'col_offset', 'end_lineno', 'end_col_offset',), 252 | (slice,))), 253 | 254 | # boolop 255 | ('And', ((), (), (boolop,))), 256 | ('Or', ((), (), (boolop,))), 257 | 258 | # operator 259 | ('Add', ((), (), (operator,))), 260 | ('Sub', ((), (), (operator,))), 261 | ('Mult', ((), (), (operator,))), 262 | ('MatMult', ((), (), (operator,))), 263 | ('Div', ((), (), (operator,))), 264 | ('Mod', ((), (), (operator,))), 265 | ('Pow', ((), (), (operator,))), 266 | ('LShift', ((), (), (operator,))), 267 | ('RShift', ((), (), (operator,))), 268 | ('BitOr', ((), (), (operator,))), 269 | ('BitXor', ((), (), (operator,))), 270 | ('BitAnd', ((), (), (operator,))), 271 | ('FloorDiv', ((), (), (operator,))), 272 | 273 | # unaryop 274 | ('Invert', ((), (), (unaryop, AST,))), 275 | ('Not', ((), (), (unaryop, AST,))), 276 | ('UAdd', ((), (), (unaryop, AST,))), 277 | ('USub', ((), (), (unaryop, AST,))), 278 | 279 | # cmpop 280 | ('Eq', ((), (), (cmpop,))), 281 | ('NotEq', ((), (), (cmpop,))), 282 | ('Lt', ((), (), (cmpop,))), 283 | ('LtE', ((), (), (cmpop,))), 284 | ('Gt', ((), (), (cmpop,))), 285 | ('GtE', ((), (), (cmpop,))), 286 | ('Is', ((), (), (cmpop,))), 287 | ('IsNot', ((), (), (cmpop,))), 288 | ('In', ((), (), (cmpop,))), 289 | ('NotIn', ((), (), (cmpop,))), 290 | 291 | # comprehension 292 | ('comprehension', (('target', 'iter', 'ifs', 'is_async'), (), (AST,))), 293 | 294 | # excepthandler 295 | ('ExceptHandler', (('type', 'name', 'body'), 296 | ('lineno', 'col_offset', 297 | 'end_lineno', 'end_col_offset'), 298 | (excepthandler,))), 299 | 300 | # arguments 301 | ('arguments', (('args', 'posonlyargs', 'vararg', 'kwonlyargs', 302 | 'kw_defaults', 'kwarg', 'defaults'), (), (AST,))), 303 | 304 | # keyword 305 | ('keyword', (('arg', 'value'), 306 | ('lineno', 'col_offset', 'end_lineno', 'end_col_offset'), 307 | (AST,))), 308 | 309 | # alias 310 | ('alias', (('name', 'asname'), 311 | ('lineno', 'col_offset', 'end_lineno', 'end_col_offset'), 312 | (AST,))), 313 | 314 | # withitem 315 | ('withitem', (('context_expr', 'optional_vars'), (), (AST,))), 316 | 317 | # match_case 318 | ('match_case', (('pattern', 'guard', 'body'), (), (AST,))), 319 | 320 | # pattern 321 | ('MatchValue', (('value',), 322 | ('lineno', 'col_offset', 'end_lineno', 'end_col_offset'), 323 | (pattern,))), 324 | ('MatchSingleton', (('value',), 325 | ('lineno', 'col_offset', 326 | 'end_lineno', 'end_col_offset'), 327 | (pattern,))), 328 | ('MatchSequence', (('patterns',), 329 | ('lineno', 'col_offset', 330 | 'end_lineno', 'end_col_offset'), 331 | (pattern,))), 332 | ('MatchMapping', (('keys', 'patterns', 'rest'), 333 | ('lineno', 'col_offset', 334 | 'end_lineno', 'end_col_offset'), 335 | (pattern,))), 336 | ('MatchClass', (('cls', 'patterns', 'kwd_attrs', 'kwd_patterns'), 337 | ('lineno', 'col_offset', 338 | 'end_lineno', 'end_col_offset'), 339 | (pattern,))), 340 | ('MatchStar', (('name',), 341 | ('lineno', 'col_offset', 342 | 'end_lineno', 'end_col_offset'), 343 | (pattern,))), 344 | ('MatchAs', (('pattern', 'name'), 345 | ('lineno', 'col_offset', 346 | 'end_lineno', 'end_col_offset'), 347 | (pattern,))), 348 | ('MatchOr', (('patterns',), 349 | ('lineno', 'col_offset', 350 | 'end_lineno', 'end_col_offset'), 351 | (pattern,))), 352 | 353 | # type_ignore 354 | ('type_ignore', ((), ('lineno', 'tag'), (TypeIgnore,))), 355 | 356 | # type_param 357 | ('TypeVar', (('name', 'bound',), 358 | ('lineno', 'col_offset', 359 | 'end_lineno', 'end_col_offset'), 360 | (type_param,))), 361 | ('ParamSpec', (('name',), 362 | ('lineno', 'col_offset', 363 | 'end_lineno', 'end_col_offset'), 364 | (type_param,))), 365 | ('TypeVarTuple', (('name',), 366 | ('lineno', 'col_offset', 367 | 'end_lineno', 'end_col_offset'), 368 | (type_param,))), 369 | ) 370 | 371 | 372 | 373 | 374 | for name, descr in _nodes: 375 | _make_node(name, *descr) 376 | 377 | if _sys.version_info.major == 2: 378 | from .ast2 import ast_to_gast, gast_to_ast 379 | if _sys.version_info.major == 3: 380 | from .ast3 import ast_to_gast, gast_to_ast 381 | 382 | 383 | def parse(*args, **kwargs): 384 | return ast_to_gast(_ast.parse(*args, **kwargs)) 385 | 386 | 387 | def unparse(gast_obj): 388 | from .unparser import unparse 389 | return unparse(gast_obj) 390 | 391 | 392 | def literal_eval(node_or_string): 393 | if isinstance(node_or_string, AST): 394 | node_or_string = gast_to_ast(node_or_string) 395 | return _ast.literal_eval(node_or_string) 396 | 397 | 398 | def get_docstring(node, clean=True): 399 | if not isinstance(node, (AsyncFunctionDef, FunctionDef, ClassDef, Module)): 400 | raise TypeError("%r can't have docstrings" % node.__class__.__name__) 401 | if not(node.body and isinstance(node.body[0], Expr)): 402 | return None 403 | node = node.body[0].value 404 | if isinstance(node, Constant) and isinstance(node.value, str): 405 | text = node.value 406 | else: 407 | return None 408 | if clean: 409 | import inspect 410 | text = inspect.cleandoc(text) 411 | return text 412 | 413 | 414 | # the following are directly imported from python3.8's Lib/ast.py # 415 | 416 | def copy_location(new_node, old_node): 417 | """ 418 | Copy source location (`lineno`, `col_offset`, `end_lineno`, and 419 | `end_col_offset` attributes) from *old_node* to *new_node* if possible, 420 | and return *new_node*. 421 | """ 422 | for attr in 'lineno', 'col_offset', 'end_lineno', 'end_col_offset': 423 | if attr in old_node._attributes and attr in new_node._attributes \ 424 | and hasattr(old_node, attr): 425 | setattr(new_node, attr, getattr(old_node, attr)) 426 | return new_node 427 | 428 | 429 | def fix_missing_locations(node): 430 | """ 431 | When you compile a node tree with compile(), the compiler expects lineno 432 | and col_offset attributes for every node that supports them. This is 433 | rather tedious to fill in for generated nodes, so this helper adds these 434 | attributes recursively where not already set, by setting them to the values 435 | of the parent node. It works recursively starting at *node*. 436 | """ 437 | def _fix(node, lineno, col_offset, end_lineno, end_col_offset): 438 | if 'lineno' in node._attributes: 439 | if not hasattr(node, 'lineno'): 440 | node.lineno = lineno 441 | else: 442 | lineno = node.lineno 443 | if 'end_lineno' in node._attributes: 444 | if not hasattr(node, 'end_lineno'): 445 | node.end_lineno = end_lineno 446 | else: 447 | end_lineno = node.end_lineno 448 | if 'col_offset' in node._attributes: 449 | if not hasattr(node, 'col_offset'): 450 | node.col_offset = col_offset 451 | else: 452 | col_offset = node.col_offset 453 | if 'end_col_offset' in node._attributes: 454 | if not hasattr(node, 'end_col_offset'): 455 | node.end_col_offset = end_col_offset 456 | else: 457 | end_col_offset = node.end_col_offset 458 | for child in iter_child_nodes(node): 459 | _fix(child, lineno, col_offset, end_lineno, end_col_offset) 460 | _fix(node, 1, 0, 1, 0) 461 | return node 462 | 463 | 464 | if _sys.version_info.major == 3 and _sys.version_info.minor >= 8: 465 | get_source_segment = _ast.get_source_segment 466 | else: 467 | # No end_lineno no end_col_offset info set for those version, so always 468 | # return None 469 | def get_source_segment(source, node, padded=False): 470 | return None 471 | 472 | 473 | def increment_lineno(node, n=1): 474 | """ 475 | Increment the line number and end line number of each node in the tree 476 | starting at *node* by *n*. This is useful to "move code" to a different 477 | location in a file. 478 | """ 479 | for child in walk(node): 480 | if 'lineno' in child._attributes: 481 | child.lineno = (getattr(child, 'lineno', 0) or 0) + n 482 | if 'end_lineno' in child._attributes: 483 | child.end_lineno = (getattr(child, 'end_lineno', 0) or 0) + n 484 | return node 485 | 486 | if _sys.version_info.major == 3 and _sys.version_info.minor >= 13: 487 | dump = _ast.dump 488 | else: 489 | # Code import from Lib/ast.py 490 | # 491 | # minor changes: getattr(x, y, ...) is None => getattr(x, y, 42) is None 492 | # 493 | def dump( 494 | node, annotate_fields=True, include_attributes=False, 495 | # *, # removed for compatibility with python2 :-/ 496 | indent=None, show_empty=False, 497 | ): 498 | """ 499 | Return a formatted dump of the tree in node. This is mainly useful for 500 | debugging purposes. If annotate_fields is true (by default), 501 | the returned string will show the names and the values for fields. 502 | If annotate_fields is false, the result string will be more compact by 503 | omitting unambiguous field names. Attributes such as line 504 | numbers and column offsets are not dumped by default. If this is wanted, 505 | include_attributes can be set to true. If indent is a non-negative 506 | integer or string, then the tree will be pretty-printed with that indent 507 | level. None (the default) selects the single line representation. 508 | If show_empty is False, then empty lists and fields that are None 509 | will be omitted from the output for better readability. 510 | """ 511 | def _format(node, level=0): 512 | if indent is not None: 513 | level += 1 514 | prefix = '\n' + indent * level 515 | sep = ',\n' + indent * level 516 | else: 517 | prefix = '' 518 | sep = ', ' 519 | if isinstance(node, AST): 520 | cls = type(node) 521 | args = [] 522 | args_buffer = [] 523 | allsimple = True 524 | keywords = annotate_fields 525 | for name in node._fields: 526 | try: 527 | value = getattr(node, name) 528 | except AttributeError: 529 | keywords = True 530 | continue 531 | if value is None and getattr(cls, name, 42) is None: 532 | keywords = True 533 | continue 534 | if ( 535 | not show_empty 536 | and (value is None or value == []) 537 | # Special cases: 538 | # `Constant(value=None)` and `MatchSingleton(value=None)` 539 | and not isinstance(node, (Constant, MatchSingleton)) 540 | ): 541 | args_buffer.append(repr(value)) 542 | continue 543 | elif not keywords: 544 | args.extend(args_buffer) 545 | args_buffer = [] 546 | value, simple = _format(value, level) 547 | allsimple = allsimple and simple 548 | if keywords: 549 | args.append('%s=%s' % (name, value)) 550 | else: 551 | args.append(value) 552 | if include_attributes and node._attributes: 553 | for name in node._attributes: 554 | try: 555 | value = getattr(node, name) 556 | except AttributeError: 557 | continue 558 | if value is None and getattr(cls, name, 42) is None: 559 | continue 560 | value, simple = _format(value, level) 561 | allsimple = allsimple and simple 562 | args.append('%s=%s' % (name, value)) 563 | if allsimple and len(args) <= 3: 564 | return '%s(%s)' % (node.__class__.__name__, ', '.join(args)), not args 565 | return '%s(%s%s)' % (node.__class__.__name__, prefix, sep.join(args)), False 566 | elif isinstance(node, list): 567 | if not node: 568 | return '[]', True 569 | return '[%s%s]' % (prefix, sep.join(_format(x, level)[0] for x in node)), False 570 | return repr(node), True 571 | 572 | if not isinstance(node, AST): 573 | raise TypeError('expected AST, got %r' % node.__class__.__name__) 574 | if indent is not None and not isinstance(indent, str): 575 | indent = ' ' * indent 576 | return _format(node)[0] 577 | -------------------------------------------------------------------------------- /gast/unparser.py: -------------------------------------------------------------------------------- 1 | # Code copied from Lib/ast.py from cpython 3.10 and slightly adjusted for gast 2 | # 3 | # PYTHON SOFTWARE FOUNDATION LICENSE VERSION 2 4 | # -------------------------------------------- 5 | # 6 | # 1. This LICENSE AGREEMENT is between the Python Software Foundation 7 | # ("PSF"), and the Individual or Organization ("Licensee") accessing and 8 | # otherwise using this software ("Python") in source or binary form and 9 | # its associated documentation. 10 | # 11 | # 2. Subject to the terms and conditions of this License Agreement, PSF hereby 12 | # grants Licensee a nonexclusive, royalty-free, world-wide license to reproduce, 13 | # analyze, test, perform and/or display publicly, prepare derivative works, 14 | # distribute, and otherwise use Python alone or in any derivative version, 15 | # provided, however, that PSF's License Agreement and PSF's notice of copyright, 16 | # i.e., "Copyright (c) 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009, 2010, 17 | # 2011, 2012, 2013, 2014, 2015, 2016, 2017, 2018, 2019, 2020 Python Software Foundation; 18 | # All Rights Reserved" are retained in Python alone or in any derivative version 19 | # prepared by Licensee. 20 | # 21 | # 3. In the event Licensee prepares a derivative work that is based on 22 | # or incorporates Python or any part thereof, and wants to make 23 | # the derivative work available to others as provided herein, then 24 | # Licensee hereby agrees to include in any such work a brief summary of 25 | # the changes made to Python. 26 | # 27 | # 4. PSF is making Python available to Licensee on an "AS IS" 28 | # basis. PSF MAKES NO REPRESENTATIONS OR WARRANTIES, EXPRESS OR 29 | # IMPLIED. BY WAY OF EXAMPLE, BUT NOT LIMITATION, PSF MAKES NO AND 30 | # DISCLAIMS ANY REPRESENTATION OR WARRANTY OF MERCHANTABILITY OR FITNESS 31 | # FOR ANY PARTICULAR PURPOSE OR THAT THE USE OF PYTHON WILL NOT 32 | # INFRINGE ANY THIRD PARTY RIGHTS. 33 | # 34 | # 5. PSF SHALL NOT BE LIABLE TO LICENSEE OR ANY OTHER USERS OF PYTHON 35 | # FOR ANY INCIDENTAL, SPECIAL, OR CONSEQUENTIAL DAMAGES OR LOSS AS 36 | # A RESULT OF MODIFYING, DISTRIBUTING, OR OTHERWISE USING PYTHON, 37 | # OR ANY DERIVATIVE THEREOF, EVEN IF ADVISED OF THE POSSIBILITY THEREOF. 38 | # 39 | # 6. This License Agreement will automatically terminate upon a material 40 | # breach of its terms and conditions. 41 | # 42 | # 7. Nothing in this License Agreement shall be deemed to create any 43 | # relationship of agency, partnership, or joint venture between PSF and 44 | # Licensee. This License Agreement does not grant permission to use PSF 45 | # trademarks or trade name in a trademark sense to endorse or promote 46 | # products or services of Licensee, or any third party. 47 | # 48 | # 8. By copying, installing or otherwise using Python, Licensee 49 | # agrees to be bound by the terms and conditions of this License 50 | # Agreement. 51 | 52 | import sys 53 | from . import * 54 | from contextlib import contextmanager 55 | from string import printable 56 | 57 | 58 | class nullcontext(object): 59 | def __init__(self, enter_result=None): 60 | self.enter_result = enter_result 61 | 62 | def __enter__(self): 63 | return self.enter_result 64 | 65 | def __exit__(self, *excinfo): 66 | pass 67 | 68 | 69 | # Large float and imaginary literals get turned into infinities in the AST. 70 | # We unparse those infinities to INFSTR. 71 | _INFSTR = "1e" + repr(sys.float_info.max_10_exp + 1) 72 | 73 | class _Precedence(object): 74 | """Precedence table that originated from python grammar.""" 75 | 76 | TUPLE = 1 77 | YIELD = 2 # 'yield', 'yield from' 78 | TEST = 3 # 'if'-'else', 'lambda' 79 | OR = 4 # 'or' 80 | AND = 5 # 'and' 81 | NOT = 6 # 'not' 82 | CMP = 7 # '<', '>', '==', '>=', '<=', '!=', 83 | # 'in', 'not in', 'is', 'is not' 84 | EXPR = 8 85 | BOR = EXPR # '|' 86 | BXOR = 9 # '^' 87 | BAND = 10 # '&' 88 | SHIFT = 11 # '<<', '>>' 89 | ARITH = 12 # '+', '-' 90 | TERM = 13 # '*', '@', '/', '%', '//' 91 | FACTOR = 14 # unary '+', '-', '~' 92 | POWER = 15 # '**' 93 | AWAIT = 16 # 'await' 94 | ATOM = 17 95 | 96 | 97 | _SINGLE_QUOTES = ("'", '"') 98 | _MULTI_QUOTES = ('"""', "'''") 99 | _ALL_QUOTES = _SINGLE_QUOTES + _MULTI_QUOTES 100 | 101 | class _Unparser(NodeVisitor): 102 | """Methods in this class recursively traverse an AST and 103 | output source code for the abstract syntax; original formatting 104 | is disregarded.""" 105 | 106 | def __init__(self, _avoid_backslashes=False): 107 | self._source = [] 108 | self._buffer = [] 109 | self._precedences = {} 110 | self._type_ignores = {} 111 | self._indent = 0 112 | self._avoid_backslashes = _avoid_backslashes 113 | 114 | def interleave(self, inter, f, seq): 115 | """Call f on each item in seq, calling inter() in between.""" 116 | seq = iter(seq) 117 | try: 118 | f(next(seq)) 119 | except StopIteration: 120 | pass 121 | else: 122 | for x in seq: 123 | inter() 124 | f(x) 125 | 126 | def items_view(self, traverser, items): 127 | """Traverse and separate the given *items* with a comma and append it to 128 | the buffer. If *items* is a single item sequence, a trailing comma 129 | will be added.""" 130 | if len(items) == 1: 131 | traverser(items[0]) 132 | self.write(",") 133 | else: 134 | self.interleave(lambda: self.write(", "), traverser, items) 135 | 136 | def maybe_newline(self): 137 | """Adds a newline if it isn't the start of generated source""" 138 | if self._source: 139 | self.write("\n") 140 | 141 | def fill(self, text=""): 142 | """Indent a piece of text and append it, according to the current 143 | indentation level""" 144 | self.maybe_newline() 145 | self.write(" " * self._indent + text) 146 | 147 | def write(self, text): 148 | """Append a piece of text""" 149 | self._source.append(text) 150 | 151 | def buffer_writer(self, text): 152 | self._buffer.append(text) 153 | 154 | @property 155 | def buffer(self): 156 | value = "".join(self._buffer) 157 | self._buffer.clear() 158 | return value 159 | 160 | @contextmanager 161 | def block(self, extra = None): 162 | """A context manager for preparing the source for blocks. It adds 163 | the character':', increases the indentation on enter and decreases 164 | the indentation on exit. If *extra* is given, it will be directly 165 | appended after the colon character. 166 | """ 167 | self.write(":") 168 | if extra: 169 | self.write(extra) 170 | self._indent += 1 171 | yield 172 | self._indent -= 1 173 | 174 | @contextmanager 175 | def delimit(self, start, end): 176 | """A context manager for preparing the source for expressions. It adds 177 | *start* to the buffer and enters, after exit it adds *end*.""" 178 | 179 | self.write(start) 180 | yield 181 | self.write(end) 182 | 183 | def delimit_if(self, start, end, condition): 184 | if condition: 185 | return self.delimit(start, end) 186 | else: 187 | return nullcontext() 188 | 189 | def require_parens(self, precedence, node): 190 | """Shortcut to adding precedence related parens""" 191 | return self.delimit_if("(", ")", self.get_precedence(node) > precedence) 192 | 193 | def get_precedence(self, node): 194 | return self._precedences.get(node, _Precedence.TEST) 195 | 196 | def set_precedence(self, precedence, *nodes): 197 | for node in nodes: 198 | self._precedences[node] = precedence 199 | 200 | def get_raw_docstring(self, node): 201 | """If a docstring node is found in the body of the *node* parameter, 202 | return that docstring node, None otherwise. 203 | 204 | Logic mirrored from ``_PyAST_GetDocString``.""" 205 | if not isinstance( 206 | node, (AsyncFunctionDef, FunctionDef, ClassDef, Module) 207 | ) or len(node.body) < 1: 208 | return None 209 | node = node.body[0] 210 | if not isinstance(node, Expr): 211 | return None 212 | node = node.value 213 | if isinstance(node, Constant) and isinstance(node.value, str): 214 | return node 215 | 216 | def get_type_comment(self, node): 217 | comment = self._type_ignores.get(node.lineno) or node.type_comment 218 | if comment is not None: 219 | return " # type: {}".format(comment) 220 | 221 | def traverse(self, node): 222 | if isinstance(node, list): 223 | for item in node: 224 | self.traverse(item) 225 | else: 226 | super(_Unparser, self).visit(node) 227 | 228 | # Note: as visit() resets the output text, do NOT rely on 229 | # NodeVisitor.generic_visit to handle any nodes (as it calls back in to 230 | # the subclass visit() method, which resets self._source to an empty list) 231 | def visit(self, node): 232 | """Outputs a source code string that, if converted back to an ast 233 | (using ast.parse) will generate an AST equivalent to *node*""" 234 | self._source = [] 235 | self.traverse(node) 236 | return "".join(self._source) 237 | 238 | def _write_docstring_and_traverse_body(self, node): 239 | docstring = self.get_raw_docstring(node) 240 | if docstring: 241 | self._write_docstring(docstring) 242 | self.traverse(node.body[1:]) 243 | else: 244 | self.traverse(node.body) 245 | 246 | def visit_Module(self, node): 247 | self._type_ignores = { 248 | ignore.lineno: "ignore{}".format(ignore.tag) 249 | for ignore in node.type_ignores 250 | } 251 | self._write_docstring_and_traverse_body(node) 252 | self._type_ignores.clear() 253 | 254 | def visit_FunctionType(self, node): 255 | with self.delimit("(", ")"): 256 | self.interleave( 257 | lambda: self.write(", "), self.traverse, node.argtypes 258 | ) 259 | 260 | self.write(" -> ") 261 | self.traverse(node.returns) 262 | 263 | def visit_Expr(self, node): 264 | self.fill() 265 | self.set_precedence(_Precedence.YIELD, node.value) 266 | self.traverse(node.value) 267 | 268 | def visit_NamedExpr(self, node): 269 | with self.require_parens(_Precedence.TUPLE, node): 270 | self.set_precedence(_Precedence.ATOM, node.target, node.value) 271 | self.traverse(node.target) 272 | self.write(" := ") 273 | self.traverse(node.value) 274 | 275 | def visit_Import(self, node): 276 | self.fill("import ") 277 | self.interleave(lambda: self.write(", "), self.traverse, node.names) 278 | 279 | def visit_ImportFrom(self, node): 280 | self.fill("from ") 281 | self.write("." * node.level) 282 | if node.module: 283 | self.write(node.module) 284 | self.write(" import ") 285 | self.interleave(lambda: self.write(", "), self.traverse, node.names) 286 | 287 | def visit_Assign(self, node): 288 | self.fill() 289 | for target in node.targets: 290 | self.traverse(target) 291 | self.write(" = ") 292 | self.traverse(node.value) 293 | type_comment = self.get_type_comment(node) 294 | if type_comment: 295 | self.write(type_comment) 296 | 297 | def visit_AugAssign(self, node): 298 | self.fill() 299 | self.traverse(node.target) 300 | self.write(" " + self.binop[node.op.__class__.__name__] + "= ") 301 | self.traverse(node.value) 302 | 303 | def visit_AnnAssign(self, node): 304 | self.fill() 305 | with self.delimit_if("(", ")", not node.simple and isinstance(node.target, Name)): 306 | self.traverse(node.target) 307 | self.write(": ") 308 | self.traverse(node.annotation) 309 | if node.value: 310 | self.write(" = ") 311 | self.traverse(node.value) 312 | 313 | def visit_Return(self, node): 314 | self.fill("return") 315 | if node.value: 316 | self.write(" ") 317 | self.traverse(node.value) 318 | 319 | def visit_Pass(self, node): 320 | self.fill("pass") 321 | 322 | def visit_Break(self, node): 323 | self.fill("break") 324 | 325 | def visit_Continue(self, node): 326 | self.fill("continue") 327 | 328 | def visit_Delete(self, node): 329 | self.fill("del ") 330 | self.interleave(lambda: self.write(", "), self.traverse, node.targets) 331 | 332 | def visit_Assert(self, node): 333 | self.fill("assert ") 334 | self.traverse(node.test) 335 | if node.msg: 336 | self.write(", ") 337 | self.traverse(node.msg) 338 | 339 | def visit_Global(self, node): 340 | self.fill("global ") 341 | self.interleave(lambda: self.write(", "), self.write, node.names) 342 | 343 | def visit_Nonlocal(self, node): 344 | self.fill("nonlocal ") 345 | self.interleave(lambda: self.write(", "), self.write, node.names) 346 | 347 | def visit_Await(self, node): 348 | with self.require_parens(_Precedence.AWAIT, node): 349 | self.write("await") 350 | if node.value: 351 | self.write(" ") 352 | self.set_precedence(_Precedence.ATOM, node.value) 353 | self.traverse(node.value) 354 | 355 | def visit_Yield(self, node): 356 | with self.require_parens(_Precedence.YIELD, node): 357 | self.write("yield") 358 | if node.value: 359 | self.write(" ") 360 | self.set_precedence(_Precedence.ATOM, node.value) 361 | self.traverse(node.value) 362 | 363 | def visit_YieldFrom(self, node): 364 | with self.require_parens(_Precedence.YIELD, node): 365 | self.write("yield from ") 366 | if not node.value: 367 | raise ValueError("Node can't be used without a value attribute.") 368 | self.set_precedence(_Precedence.ATOM, node.value) 369 | self.traverse(node.value) 370 | 371 | def visit_Raise(self, node): 372 | self.fill("raise") 373 | if not node.exc: 374 | if node.cause: 375 | raise ValueError("Node can't use cause without an exception.") 376 | return 377 | self.write(" ") 378 | self.traverse(node.exc) 379 | if node.cause: 380 | self.write(" from ") 381 | self.traverse(node.cause) 382 | 383 | def visit_Try(self, node): 384 | self.fill("try") 385 | with self.block(): 386 | self.traverse(node.body) 387 | for ex in node.handlers: 388 | self.traverse(ex) 389 | if node.orelse: 390 | self.fill("else") 391 | with self.block(): 392 | self.traverse(node.orelse) 393 | if node.finalbody: 394 | self.fill("finally") 395 | with self.block(): 396 | self.traverse(node.finalbody) 397 | 398 | def visit_ExceptHandler(self, node): 399 | self.fill("except") 400 | if node.type: 401 | self.write(" ") 402 | self.traverse(node.type) 403 | if node.name: 404 | self.write(" as ") 405 | self.write(node.name.id) 406 | with self.block(): 407 | self.traverse(node.body) 408 | 409 | def visit_ClassDef(self, node): 410 | self.maybe_newline() 411 | for deco in node.decorator_list: 412 | self.fill("@") 413 | self.traverse(deco) 414 | self.fill("class " + node.name) 415 | if hasattr(node, "type_params"): 416 | self._type_params_helper(node.type_params) 417 | with self.delimit_if("(", ")", condition = node.bases or node.keywords): 418 | comma = False 419 | for e in node.bases: 420 | if comma: 421 | self.write(", ") 422 | else: 423 | comma = True 424 | self.traverse(e) 425 | for e in node.keywords: 426 | if comma: 427 | self.write(", ") 428 | else: 429 | comma = True 430 | self.traverse(e) 431 | 432 | with self.block(): 433 | self._write_docstring_and_traverse_body(node) 434 | 435 | def visit_FunctionDef(self, node): 436 | self._function_helper(node, "def") 437 | 438 | def visit_AsyncFunctionDef(self, node): 439 | self._function_helper(node, "async def") 440 | 441 | def _function_helper(self, node, fill_suffix): 442 | self.maybe_newline() 443 | for deco in node.decorator_list: 444 | self.fill("@") 445 | self.traverse(deco) 446 | def_str = fill_suffix + " " + node.name 447 | self.fill(def_str) 448 | if hasattr(node, "type_params"): 449 | self._type_params_helper(node.type_params) 450 | with self.delimit("(", ")"): 451 | self.traverse(node.args) 452 | if node.returns: 453 | self.write(" -> ") 454 | self.traverse(node.returns) 455 | with self.block(extra=self.get_type_comment(node)): 456 | self._write_docstring_and_traverse_body(node) 457 | 458 | def _type_params_helper(self, type_params): 459 | if type_params is not None and len(type_params) > 0: 460 | with self.delimit("[", "]"): 461 | self.interleave(lambda: self.write(", "), self.traverse, type_params) 462 | 463 | def visit_TypeVar(self, node): 464 | self.write(node.name) 465 | if node.bound: 466 | self.write(": ") 467 | self.traverse(node.bound) 468 | 469 | def visit_TypeVarTuple(self, node): 470 | self.write("*" + node.name) 471 | 472 | def visit_ParamSpec(self, node): 473 | self.write("**" + node.name) 474 | 475 | def visit_TypeAlias(self, node): 476 | self.fill("type ") 477 | self.traverse(node.name) 478 | self._type_params_helper(node.type_params) 479 | self.write(" = ") 480 | self.traverse(node.value) 481 | 482 | def visit_For(self, node): 483 | self._for_helper("for ", node) 484 | 485 | def visit_AsyncFor(self, node): 486 | self._for_helper("async for ", node) 487 | 488 | def _for_helper(self, fill, node): 489 | self.fill(fill) 490 | self.traverse(node.target) 491 | self.write(" in ") 492 | self.traverse(node.iter) 493 | with self.block(extra=self.get_type_comment(node)): 494 | self.traverse(node.body) 495 | if node.orelse: 496 | self.fill("else") 497 | with self.block(): 498 | self.traverse(node.orelse) 499 | 500 | def visit_If(self, node): 501 | self.fill("if ") 502 | self.traverse(node.test) 503 | with self.block(): 504 | self.traverse(node.body) 505 | # collapse nested ifs into equivalent elifs. 506 | while node.orelse and len(node.orelse) == 1 and isinstance(node.orelse[0], If): 507 | node = node.orelse[0] 508 | self.fill("elif ") 509 | self.traverse(node.test) 510 | with self.block(): 511 | self.traverse(node.body) 512 | # final else 513 | if node.orelse: 514 | self.fill("else") 515 | with self.block(): 516 | self.traverse(node.orelse) 517 | 518 | def visit_While(self, node): 519 | self.fill("while ") 520 | self.traverse(node.test) 521 | with self.block(): 522 | self.traverse(node.body) 523 | if node.orelse: 524 | self.fill("else") 525 | with self.block(): 526 | self.traverse(node.orelse) 527 | 528 | def visit_With(self, node): 529 | self.fill("with ") 530 | self.interleave(lambda: self.write(", "), self.traverse, node.items) 531 | with self.block(extra=self.get_type_comment(node)): 532 | self.traverse(node.body) 533 | 534 | def visit_AsyncWith(self, node): 535 | self.fill("async with ") 536 | self.interleave(lambda: self.write(", "), self.traverse, node.items) 537 | with self.block(extra=self.get_type_comment(node)): 538 | self.traverse(node.body) 539 | 540 | def _str_literal_helper( 541 | self, string, quote_types=_ALL_QUOTES, escape_special_whitespace=False 542 | ): 543 | """Helper for writing string literals, minimizing escapes. 544 | Returns the tuple (string literal to write, possible quote types). 545 | """ 546 | def escape_char(c): 547 | # \n and \t are non-printable, but we only escape them if 548 | # escape_special_whitespace is True 549 | if not escape_special_whitespace and c in "\n\t": 550 | return c 551 | # Always escape backslashes and other non-printable characters 552 | if c == "\\" or not all(cc in printable for cc in c): 553 | return c.encode("unicode_escape").decode("ascii") 554 | return c 555 | 556 | escaped_string = "".join(map(escape_char, string)) 557 | possible_quotes = quote_types 558 | if "\n" in escaped_string: 559 | possible_quotes = [q for q in possible_quotes if q in _MULTI_QUOTES] 560 | possible_quotes = [q for q in possible_quotes if q not in escaped_string] 561 | if not possible_quotes: 562 | # If there aren't any possible_quotes, fallback to using repr 563 | # on the original string. Try to use a quote from quote_types, 564 | # e.g., so that we use triple quotes for docstrings. 565 | string = repr(string) 566 | quote = next((q for q in quote_types if string[0] in q), string[0]) 567 | return string[1:-1], [quote] 568 | if escaped_string: 569 | # Sort so that we prefer '''"''' over """\"""" 570 | possible_quotes.sort(key=lambda q: q[0] == escaped_string[-1]) 571 | # If we're using triple quotes and we'd need to escape a final 572 | # quote, escape it 573 | if possible_quotes[0][0] == escaped_string[-1]: 574 | assert len(possible_quotes[0]) == 3 575 | escaped_string = escaped_string[:-1] + "\\" + escaped_string[-1] 576 | return escaped_string, possible_quotes 577 | 578 | def _write_str_avoiding_backslashes(self, string, quote_types=_ALL_QUOTES): 579 | """Write string literal value with a best effort attempt to avoid backslashes.""" 580 | string, quote_types = self._str_literal_helper(string, quote_types=quote_types) 581 | quote_type = quote_types[0] 582 | self.write("{0}{1}{0}".format(quote_type, string)) 583 | 584 | def visit_JoinedStr(self, node): 585 | self.write("f") 586 | if self._avoid_backslashes: 587 | self._fstring_JoinedStr(node, self.buffer_writer) 588 | self._write_str_avoiding_backslashes(self.buffer) 589 | return 590 | 591 | # If we don't need to avoid backslashes globally (i.e., we only need 592 | # to avoid them inside FormattedValues), it's cosmetically preferred 593 | # to use escaped whitespace. That is, it's preferred to use backslashes 594 | # for cases like: f"{x}\n". To accomplish this, we keep track of what 595 | # in our buffer corresponds to FormattedValues and what corresponds to 596 | # Constant parts of the f-string, and allow escapes accordingly. 597 | buffer = [] 598 | for value in node.values: 599 | meth = getattr(self, "_fstring_" + type(value).__name__) 600 | meth(value, self.buffer_writer) 601 | buffer.append((self.buffer, isinstance(value, Constant))) 602 | new_buffer = [] 603 | quote_types = _ALL_QUOTES 604 | for value, is_constant in buffer: 605 | # Repeatedly narrow down the list of possible quote_types 606 | value, quote_types = self._str_literal_helper( 607 | value, quote_types=quote_types, 608 | escape_special_whitespace=is_constant 609 | ) 610 | new_buffer.append(value) 611 | value = "".join(new_buffer) 612 | quote_type = quote_types[0] 613 | self.write("{0}{1}{0}".format(quote_type, value)) 614 | 615 | def visit_FormattedValue(self, node): 616 | self.write("f") 617 | self._fstring_FormattedValue(node, self.buffer_writer) 618 | self._write_str_avoiding_backslashes(self.buffer) 619 | 620 | def _fstring_JoinedStr(self, node, write): 621 | for value in node.values: 622 | meth = getattr(self, "_fstring_" + type(value).__name__) 623 | meth(value, write) 624 | 625 | def _fstring_Constant(self, node, write): 626 | if not isinstance(node.value, str): 627 | raise ValueError("Constants inside JoinedStr should be a string.") 628 | value = node.value.replace("{", "{{").replace("}", "}}") 629 | write(value) 630 | 631 | def _fstring_FormattedValue(self, node, write): 632 | write("{") 633 | unparser = type(self)(_avoid_backslashes=True) 634 | unparser.set_precedence(_Precedence.TEST + 1, node.value) 635 | expr = unparser.visit(node.value) 636 | if expr.startswith("{"): 637 | write(" ") # Separate pair of opening brackets as "{ {" 638 | if "\\" in expr: 639 | raise ValueError("Unable to avoid backslash in f-string expression part") 640 | write(expr) 641 | if node.conversion != -1: 642 | conversion = chr(node.conversion) 643 | if conversion not in "sra": 644 | raise ValueError("Unknown f-string conversion.") 645 | write("!{}".format(conversion)) 646 | if node.format_spec: 647 | write(":") 648 | meth = getattr(self, "_fstring_" + type(node.format_spec).__name__) 649 | meth(node.format_spec, write) 650 | write("}") 651 | 652 | def visit_Name(self, node): 653 | self.write(node.id) 654 | 655 | def _write_docstring(self, node): 656 | self.fill() 657 | if node.kind == "u": 658 | self.write("u") 659 | self._write_str_avoiding_backslashes(node.value, quote_types=_MULTI_QUOTES) 660 | 661 | def _write_constant(self, value): 662 | if isinstance(value, (float, complex)): 663 | # Substitute overflowing decimal literal for AST infinities, 664 | # and inf - inf for NaNs. 665 | self.write( 666 | repr(value) 667 | .replace("inf", _INFSTR) 668 | .replace("nan", "({0}-{0})".format(_INFSTR)) 669 | ) 670 | elif self._avoid_backslashes and isinstance(value, str): 671 | self._write_str_avoiding_backslashes(value) 672 | else: 673 | self.write(repr(value)) 674 | 675 | def visit_Constant(self, node): 676 | value = node.value 677 | if isinstance(value, tuple): 678 | with self.delimit("(", ")"): 679 | self.items_view(self._write_constant, value) 680 | elif value is Ellipsis: 681 | self.write("...") 682 | else: 683 | if node.kind == "u": 684 | self.write("u") 685 | self._write_constant(node.value) 686 | 687 | def visit_List(self, node): 688 | with self.delimit("[", "]"): 689 | self.interleave(lambda: self.write(", "), self.traverse, node.elts) 690 | 691 | def visit_ListComp(self, node): 692 | with self.delimit("[", "]"): 693 | self.traverse(node.elt) 694 | for gen in node.generators: 695 | self.traverse(gen) 696 | 697 | def visit_GeneratorExp(self, node): 698 | with self.delimit("(", ")"): 699 | self.traverse(node.elt) 700 | for gen in node.generators: 701 | self.traverse(gen) 702 | 703 | def visit_SetComp(self, node): 704 | with self.delimit("{", "}"): 705 | self.traverse(node.elt) 706 | for gen in node.generators: 707 | self.traverse(gen) 708 | 709 | def visit_DictComp(self, node): 710 | with self.delimit("{", "}"): 711 | self.traverse(node.key) 712 | self.write(": ") 713 | self.traverse(node.value) 714 | for gen in node.generators: 715 | self.traverse(gen) 716 | 717 | def visit_comprehension(self, node): 718 | if node.is_async: 719 | self.write(" async for ") 720 | else: 721 | self.write(" for ") 722 | self.set_precedence(_Precedence.TUPLE, node.target) 723 | self.traverse(node.target) 724 | self.write(" in ") 725 | self.set_precedence(_Precedence.TEST + 1, node.iter, *node.ifs) 726 | self.traverse(node.iter) 727 | for if_clause in node.ifs: 728 | self.write(" if ") 729 | self.traverse(if_clause) 730 | 731 | def visit_IfExp(self, node): 732 | with self.require_parens(_Precedence.TEST, node): 733 | self.set_precedence(_Precedence.TEST + 1, node.body, node.test) 734 | self.traverse(node.body) 735 | self.write(" if ") 736 | self.traverse(node.test) 737 | self.write(" else ") 738 | self.set_precedence(_Precedence.TEST, node.orelse) 739 | self.traverse(node.orelse) 740 | 741 | def visit_Set(self, node): 742 | if node.elts: 743 | with self.delimit("{", "}"): 744 | self.interleave(lambda: self.write(", "), self.traverse, node.elts) 745 | else: 746 | # `{}` would be interpreted as a dictionary literal, and 747 | # `set` might be shadowed. Thus: 748 | self.write('{*()}') 749 | 750 | def visit_Dict(self, node): 751 | def write_key_value_pair(k, v): 752 | self.traverse(k) 753 | self.write(": ") 754 | self.traverse(v) 755 | 756 | def write_item(item): 757 | k, v = item 758 | if k is None: 759 | # for dictionary unpacking operator in dicts {**{'y': 2}} 760 | # see PEP 448 for details 761 | self.write("**") 762 | self.set_precedence(_Precedence.EXPR, v) 763 | self.traverse(v) 764 | else: 765 | write_key_value_pair(k, v) 766 | 767 | with self.delimit("{", "}"): 768 | self.interleave( 769 | lambda: self.write(", "), write_item, zip(node.keys, node.values) 770 | ) 771 | 772 | def visit_Tuple(self, node): 773 | with self.delimit("(", ")"): 774 | self.items_view(self.traverse, node.elts) 775 | 776 | unop = {"Invert": "~", "Not": "not", "UAdd": "+", "USub": "-"} 777 | unop_precedence = { 778 | "not": _Precedence.NOT, 779 | "~": _Precedence.FACTOR, 780 | "+": _Precedence.FACTOR, 781 | "-": _Precedence.FACTOR, 782 | } 783 | 784 | def visit_UnaryOp(self, node): 785 | operator = self.unop[node.op.__class__.__name__] 786 | operator_precedence = self.unop_precedence[operator] 787 | with self.require_parens(operator_precedence, node): 788 | self.write(operator) 789 | # factor prefixes (+, -, ~) shouldn't be seperated 790 | # from the value they belong, (e.g: +1 instead of + 1) 791 | if operator_precedence is not _Precedence.FACTOR: 792 | self.write(" ") 793 | self.set_precedence(operator_precedence, node.operand) 794 | self.traverse(node.operand) 795 | 796 | binop = { 797 | "Add": "+", 798 | "Sub": "-", 799 | "Mult": "*", 800 | "MatMult": "@", 801 | "Div": "/", 802 | "Mod": "%", 803 | "LShift": "<<", 804 | "RShift": ">>", 805 | "BitOr": "|", 806 | "BitXor": "^", 807 | "BitAnd": "&", 808 | "FloorDiv": "//", 809 | "Pow": "**", 810 | } 811 | 812 | binop_precedence = { 813 | "+": _Precedence.ARITH, 814 | "-": _Precedence.ARITH, 815 | "*": _Precedence.TERM, 816 | "@": _Precedence.TERM, 817 | "/": _Precedence.TERM, 818 | "%": _Precedence.TERM, 819 | "<<": _Precedence.SHIFT, 820 | ">>": _Precedence.SHIFT, 821 | "|": _Precedence.BOR, 822 | "^": _Precedence.BXOR, 823 | "&": _Precedence.BAND, 824 | "//": _Precedence.TERM, 825 | "**": _Precedence.POWER, 826 | } 827 | 828 | binop_rassoc = frozenset(("**",)) 829 | def visit_BinOp(self, node): 830 | operator = self.binop[node.op.__class__.__name__] 831 | operator_precedence = self.binop_precedence[operator] 832 | with self.require_parens(operator_precedence, node): 833 | if operator in self.binop_rassoc: 834 | left_precedence = operator_precedence + 1 835 | right_precedence = operator_precedence 836 | else: 837 | left_precedence = operator_precedence 838 | right_precedence = operator_precedence + 1 839 | 840 | self.set_precedence(left_precedence, node.left) 841 | self.traverse(node.left) 842 | self.write(" {} ".format(operator)) 843 | self.set_precedence(right_precedence, node.right) 844 | self.traverse(node.right) 845 | 846 | cmpops = { 847 | "Eq": "==", 848 | "NotEq": "!=", 849 | "Lt": "<", 850 | "LtE": "<=", 851 | "Gt": ">", 852 | "GtE": ">=", 853 | "Is": "is", 854 | "IsNot": "is not", 855 | "In": "in", 856 | "NotIn": "not in", 857 | } 858 | 859 | def visit_Compare(self, node): 860 | with self.require_parens(_Precedence.CMP, node): 861 | self.set_precedence(_Precedence.CMP + 1, node.left, *node.comparators) 862 | self.traverse(node.left) 863 | for o, e in zip(node.ops, node.comparators): 864 | self.write(" " + self.cmpops[o.__class__.__name__] + " ") 865 | self.traverse(e) 866 | 867 | boolops = {"And": "and", "Or": "or"} 868 | boolop_precedence = {"and": _Precedence.AND, "or": _Precedence.OR} 869 | 870 | def visit_BoolOp(self, node): 871 | operator = self.boolops[node.op.__class__.__name__] 872 | operator_precedence = [self.boolop_precedence[operator]] 873 | 874 | def increasing_level_traverse(node): 875 | operator_precedence[0] += 1 876 | self.set_precedence(operator_precedence[0], node) 877 | self.traverse(node) 878 | 879 | with self.require_parens(operator_precedence[0], node): 880 | s = " {} ".format(operator) 881 | self.interleave(lambda: self.write(s), increasing_level_traverse, node.values) 882 | 883 | def visit_Attribute(self, node): 884 | self.set_precedence(_Precedence.ATOM, node.value) 885 | self.traverse(node.value) 886 | # Special case: 3.__abs__() is a syntax error, so if node.value 887 | # is an integer literal then we need to either parenthesize 888 | # it or add an extra space to get 3 .__abs__(). 889 | if isinstance(node.value, Constant) and isinstance(node.value.value, int): 890 | self.write(" ") 891 | self.write(".") 892 | self.write(node.attr) 893 | 894 | def visit_Call(self, node): 895 | self.set_precedence(_Precedence.ATOM, node.func) 896 | self.traverse(node.func) 897 | with self.delimit("(", ")"): 898 | comma = False 899 | for e in node.args: 900 | if comma: 901 | self.write(", ") 902 | else: 903 | comma = True 904 | self.traverse(e) 905 | for e in node.keywords: 906 | if comma: 907 | self.write(", ") 908 | else: 909 | comma = True 910 | self.traverse(e) 911 | 912 | def visit_Subscript(self, node): 913 | def is_simple_tuple(slice_value): 914 | # when unparsing a non-empty tuple, the parentheses can be safely 915 | # omitted if there aren't any elements that explicitly requires 916 | # parentheses (such as starred expressions). 917 | return ( 918 | isinstance(slice_value, Tuple) 919 | and slice_value.elts 920 | and not any(isinstance(elt, Starred) for elt in slice_value.elts) 921 | ) 922 | 923 | self.set_precedence(_Precedence.ATOM, node.value) 924 | self.traverse(node.value) 925 | with self.delimit("[", "]"): 926 | if is_simple_tuple(node.slice): 927 | self.items_view(self.traverse, node.slice.elts) 928 | else: 929 | self.traverse(node.slice) 930 | 931 | def visit_Starred(self, node): 932 | self.write("*") 933 | self.set_precedence(_Precedence.EXPR, node.value) 934 | self.traverse(node.value) 935 | 936 | def visit_Ellipsis(self, node): 937 | self.write("...") 938 | 939 | def visit_Slice(self, node): 940 | if node.lower: 941 | self.traverse(node.lower) 942 | self.write(":") 943 | if node.upper: 944 | self.traverse(node.upper) 945 | if node.step: 946 | self.write(":") 947 | self.traverse(node.step) 948 | 949 | def visit_Match(self, node): 950 | self.fill("match ") 951 | self.traverse(node.subject) 952 | with self.block(): 953 | for case in node.cases: 954 | self.traverse(case) 955 | 956 | def visit_arg(self, node): 957 | self.write(node.arg.id) 958 | if node.annotation: 959 | self.write(": ") 960 | self.traverse(node.annotation) 961 | 962 | def visit_arguments(self, node): 963 | first = True 964 | # normal arguments 965 | all_args = node.posonlyargs + node.args 966 | defaults = [None] * (len(all_args) - len(node.defaults)) + node.defaults 967 | for index, elements in enumerate(zip(all_args, defaults), 1): 968 | a, d = elements 969 | if first: 970 | first = False 971 | else: 972 | self.write(", ") 973 | self.traverse(a) 974 | if d: 975 | self.write("=") 976 | self.traverse(d) 977 | if index == len(node.posonlyargs): 978 | self.write(", /") 979 | 980 | # varargs, or bare '*' if no varargs but keyword-only arguments present 981 | if node.vararg or node.kwonlyargs: 982 | if first: 983 | first = False 984 | else: 985 | self.write(", ") 986 | self.write("*") 987 | if node.vararg: 988 | self.write(node.vararg.id) 989 | if node.vararg.annotation: 990 | self.write(": ") 991 | self.traverse(node.vararg.annotation) 992 | 993 | # keyword-only arguments 994 | if node.kwonlyargs: 995 | for a, d in zip(node.kwonlyargs, node.kw_defaults): 996 | self.write(", ") 997 | self.traverse(a) 998 | if d: 999 | self.write("=") 1000 | self.traverse(d) 1001 | 1002 | # kwargs 1003 | if node.kwarg: 1004 | if first: 1005 | first = False 1006 | else: 1007 | self.write(", ") 1008 | self.write("**" + node.kwarg.id) 1009 | if node.kwarg.annotation: 1010 | self.write(": ") 1011 | self.traverse(node.kwarg.annotation) 1012 | 1013 | def visit_keyword(self, node): 1014 | if node.arg is None: 1015 | self.write("**") 1016 | else: 1017 | self.write(node.arg) 1018 | self.write("=") 1019 | self.traverse(node.value) 1020 | 1021 | def visit_Lambda(self, node): 1022 | with self.require_parens(_Precedence.TEST, node): 1023 | self.write("lambda ") 1024 | self.traverse(node.args) 1025 | self.write(": ") 1026 | self.set_precedence(_Precedence.TEST, node.body) 1027 | self.traverse(node.body) 1028 | 1029 | def visit_alias(self, node): 1030 | self.write(node.name) 1031 | if node.asname: 1032 | self.write(" as " + node.asname) 1033 | 1034 | def visit_withitem(self, node): 1035 | self.traverse(node.context_expr) 1036 | if node.optional_vars: 1037 | self.write(" as ") 1038 | self.traverse(node.optional_vars) 1039 | 1040 | def visit_match_case(self, node): 1041 | self.fill("case ") 1042 | self.traverse(node.pattern) 1043 | if node.guard: 1044 | self.write(" if ") 1045 | self.traverse(node.guard) 1046 | with self.block(): 1047 | self.traverse(node.body) 1048 | 1049 | def visit_MatchValue(self, node): 1050 | self.traverse(node.value) 1051 | 1052 | def visit_MatchSingleton(self, node): 1053 | self._write_constant(node.value) 1054 | 1055 | def visit_MatchSequence(self, node): 1056 | with self.delimit("[", "]"): 1057 | self.interleave( 1058 | lambda: self.write(", "), self.traverse, node.patterns 1059 | ) 1060 | 1061 | def visit_MatchStar(self, node): 1062 | name = node.name 1063 | if name is None: 1064 | name = "_" 1065 | self.write("*{}".format(name)) 1066 | 1067 | def visit_MatchMapping(self, node): 1068 | def write_key_pattern_pair(pair): 1069 | k, p = pair 1070 | self.traverse(k) 1071 | self.write(": ") 1072 | self.traverse(p) 1073 | 1074 | with self.delimit("{", "}"): 1075 | keys = node.keys 1076 | self.interleave( 1077 | lambda: self.write(", "), 1078 | write_key_pattern_pair, 1079 | zip(keys, node.patterns, strict=True), 1080 | ) 1081 | rest = node.rest 1082 | if rest is not None: 1083 | if keys: 1084 | self.write(", ") 1085 | self.write("**{}".format(rest)) 1086 | 1087 | def visit_MatchClass(self, node): 1088 | self.set_precedence(_Precedence.ATOM, node.cls) 1089 | self.traverse(node.cls) 1090 | with self.delimit("(", ")"): 1091 | patterns = node.patterns 1092 | self.interleave( 1093 | lambda: self.write(", "), self.traverse, patterns 1094 | ) 1095 | attrs = node.kwd_attrs 1096 | if attrs: 1097 | def write_attr_pattern(pair): 1098 | attr, pattern = pair 1099 | self.write("{}=".format(attr)) 1100 | self.traverse(pattern) 1101 | 1102 | if patterns: 1103 | self.write(", ") 1104 | self.interleave( 1105 | lambda: self.write(", "), 1106 | write_attr_pattern, 1107 | zip(attrs, node.kwd_patterns, strict=True), 1108 | ) 1109 | 1110 | def visit_MatchAs(self, node): 1111 | name = node.name 1112 | pattern = node.pattern 1113 | if name is None: 1114 | self.write("_") 1115 | elif pattern is None: 1116 | self.write(node.name) 1117 | else: 1118 | with self.require_parens(_Precedence.TEST, node): 1119 | self.set_precedence(_Precedence.BOR, node.pattern) 1120 | self.traverse(node.pattern) 1121 | self.write(" as {}".format(node.name)) 1122 | 1123 | def visit_MatchOr(self, node): 1124 | with self.require_parens(_Precedence.BOR, node): 1125 | self.set_precedence(_Precedence.BOR + 1, *node.patterns) 1126 | self.interleave(lambda: self.write(" | "), self.traverse, node.patterns) 1127 | 1128 | def unparse(ast_obj): 1129 | unparser = _Unparser() 1130 | return unparser.visit(ast_obj) 1131 | -------------------------------------------------------------------------------- /gast/version.py: -------------------------------------------------------------------------------- 1 | __version__ = '0.6.0' 2 | -------------------------------------------------------------------------------- /setup.py: -------------------------------------------------------------------------------- 1 | # Use the newer `setuptools.setup()`, if available. 2 | try: 3 | from setuptools import setup 4 | kw = { 5 | 'test_suite': 'tests', 6 | } 7 | except ImportError: 8 | from distutils.core import setup 9 | kw = {} 10 | 11 | import os 12 | 13 | versionfile = os.path.join('gast', 'version.py') 14 | exec(open(versionfile).read()) 15 | 16 | setup(name='gast', # gast, daou naer! 17 | version=__version__, 18 | packages=['gast'], 19 | description='Python AST that abstracts the underlying Python version', 20 | long_description=''' 21 | A generic AST to represent Python2 and Python3's Abstract Syntax Tree(AST). 22 | 23 | GAST provides a compatibility layer between the AST of various Python versions, 24 | as produced by ``ast.parse`` from the standard ``ast`` module.''', 25 | author='serge-sans-paille', 26 | author_email='serge.guelton@telecom-bretagne.eu', 27 | url='https://github.com/serge-sans-paille/gast/', 28 | license="BSD 3-Clause", 29 | classifiers=['Development Status :: 4 - Beta', 30 | 'Environment :: Console', 31 | 'Intended Audience :: Developers', 32 | 'License :: OSI Approved :: BSD License', 33 | 'Natural Language :: English', 34 | 'Programming Language :: Python :: 2', 35 | 'Programming Language :: Python :: 2.7', 36 | 'Programming Language :: Python :: 3', 37 | 'Programming Language :: Python :: 3.4', 38 | 'Programming Language :: Python :: 3.5', 39 | 'Programming Language :: Python :: 3.6', 40 | 'Programming Language :: Python :: 3.7', 41 | 'Programming Language :: Python :: 3.8', 42 | 'Programming Language :: Python :: 3.9', 43 | 'Programming Language :: Python :: 3.10', 44 | 'Programming Language :: Python :: 3.11', 45 | ], 46 | python_requires='>=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*', 47 | **kw 48 | ) 49 | -------------------------------------------------------------------------------- /tests/__init__.py: -------------------------------------------------------------------------------- 1 | """Package containing unit tests for gast.""" 2 | -------------------------------------------------------------------------------- /tests/test_api.py: -------------------------------------------------------------------------------- 1 | import unittest 2 | 3 | import ast 4 | import gast 5 | import sys 6 | 7 | 8 | def dump(node): 9 | return gast.dump(node, show_empty=True) 10 | 11 | 12 | class APITestCase(unittest.TestCase): 13 | 14 | def test_literal_eval_string(self): 15 | code = "1, 3" 16 | self.assertEqual(ast.literal_eval(code), 17 | gast.literal_eval(code)) 18 | 19 | def test_literal_eval_code(self): 20 | code = "[1, 3]" 21 | tree = ast.parse(code, mode='eval') 22 | gtree = gast.parse(code, mode='eval') 23 | self.assertEqual(ast.literal_eval(tree), 24 | gast.literal_eval(gtree)) 25 | 26 | def test_parse(self): 27 | code = ''' 28 | def foo(x=1, *args, **kwargs): 29 | return x + y +len(args) + len(kwargs) 30 | ''' 31 | gast.parse(code) 32 | 33 | def test_unparse(self): 34 | code = 'def foo(x=1): return x' 35 | self.assertEqual(gast.unparse(gast.parse(code)), 36 | 'def foo(x=1):\n return x') 37 | 38 | def test_dump(self): 39 | code = 'lambda x: x' 40 | tree = gast.parse(code, mode='eval') 41 | zdump = dump(tree) 42 | norm = ("Expression(body=Lambda(args=arguments(args=[Name(" 43 | "id='x', ctx=Param(), " 44 | "annotation=None, type_comment=None)], posonlyargs=[], " 45 | "vararg=None, kwonlyargs=[], kw_defaults=[], kwarg=None, " 46 | "defaults=[]), body=Name(id='x', ctx=Load(), " 47 | "annotation=None, type_comment=None)" 48 | "))") 49 | self.assertEqual(zdump, norm) 50 | 51 | def test_walk(self): 52 | code = 'x + 1' 53 | tree = gast.parse(code, mode='eval') 54 | zdump = dump(tree) 55 | norm = ("Expression(body=BinOp(left=Name(id='x', ctx=Load(), " 56 | "annotation=None, type_comment=None), op=Add(), " 57 | "right=Constant(value=1, kind=None)))") 58 | self.assertEqual(zdump, norm) 59 | self.assertEqual(len(list(gast.walk(tree))), 6) 60 | 61 | def test_iter_fields(self): 62 | tree = gast.Constant(value=1, kind=None) 63 | self.assertEqual({name for name, _ in gast.iter_fields(tree)}, 64 | {'value', 'kind'}) 65 | 66 | def test_iter_child_nodes(self): 67 | tree = gast.UnaryOp(gast.USub(), gast.Constant(value=1, kind=None)) 68 | self.assertEqual(len(list(gast.iter_fields(tree))), 69 | 2) 70 | 71 | def test_increment_lineno(self): 72 | tree = gast.Constant(value=1, kind=None) 73 | tree.lineno = 1 74 | gast.increment_lineno(tree) 75 | self.assertEqual(tree.lineno, 2) 76 | 77 | def test_get_source_segment(self): 78 | code = 'x + 1' 79 | tree = gast.parse(code) 80 | source = gast.get_source_segment(code, tree.body[0].value.left) 81 | if sys.version_info >= (3, 8): 82 | self.assertEqual(source, 'x') 83 | else: 84 | self.assertEqual(source, None) 85 | 86 | 87 | def test_get_source_segment_padded(self): 88 | code = 'if 1:\n if 2:\n 3' 89 | tree = gast.parse(code) 90 | if_tree = tree.body[0].body[0] 91 | source_nopadding = gast.get_source_segment(code, if_tree, padded=False) 92 | if sys.version_info >= (3, 8): 93 | self.assertEqual(source_nopadding, 'if 2:\n 3') 94 | else: 95 | self.assertEqual(source_nopadding, None) 96 | source_padding = gast.get_source_segment(code, if_tree, padded=True) 97 | if sys.version_info >= (3, 8): 98 | self.assertEqual(source_padding, ' if 2:\n 3') 99 | else: 100 | self.assertEqual(source_padding, None) 101 | 102 | def test_get_docstring_function(self): 103 | code = 'def foo(): "foo"' 104 | tree = gast.parse(code) 105 | func = tree.body[0] 106 | docs = gast.get_docstring(func) 107 | self.assertEqual(docs, "foo") 108 | 109 | if sys.version_info >= (3, 5): 110 | def test_get_docstring_asyncfunction(self): 111 | code = 'async def foo(): "foo"' 112 | tree = gast.parse(code) 113 | func = tree.body[0] 114 | docs = gast.get_docstring(func) 115 | self.assertEqual(docs, "foo") 116 | 117 | def test_get_docstring_module(self): 118 | code = '"foo"' 119 | tree = gast.parse(code) 120 | docs = gast.get_docstring(tree) 121 | self.assertEqual(docs, "foo") 122 | 123 | def test_get_docstring_class(self): 124 | code = 'class foo: "foo"' 125 | tree = gast.parse(code) 126 | cls = tree.body[0] 127 | docs = gast.get_docstring(cls) 128 | self.assertEqual(docs, "foo") 129 | 130 | def test_get_docstring_expr(self): 131 | code = '1' 132 | tree = gast.parse(code) 133 | func = tree 134 | docs = gast.get_docstring(func) 135 | self.assertEqual(docs, None) 136 | 137 | def test_copy_location(self): 138 | tree = gast.Constant(value=1, kind=None) 139 | tree.lineno = 1 140 | tree.col_offset = 2 141 | 142 | node = gast.Constant(value=2, kind=None) 143 | gast.copy_location(node, tree) 144 | self.assertEqual(node.lineno, tree.lineno) 145 | self.assertEqual(node.col_offset, tree.col_offset) 146 | 147 | def test_fix_missing_locations(self): 148 | node = gast.Constant(value=6, kind=None) 149 | tree = gast.UnaryOp(gast.USub(), node) 150 | tree.lineno = 1 151 | tree.col_offset = 2 152 | gast.fix_missing_locations(tree) 153 | self.assertEqual(node.lineno, tree.lineno) 154 | self.assertEqual(node.col_offset, tree.col_offset) 155 | 156 | def test_NodeTransformer(self): 157 | node = gast.Constant(value=6, kind=None) 158 | tree = gast.UnaryOp(gast.USub(), node) 159 | 160 | class Trans(gast.NodeTransformer): 161 | 162 | def visit_Constant(self, node): 163 | node.value *= 2 164 | return node 165 | 166 | tree = Trans().visit(tree) 167 | 168 | self.assertEqual(node.value, 12) 169 | 170 | def test_NodeVisitor(self): 171 | node = gast.Constant(value=6, kind=None) 172 | tree = gast.UnaryOp(gast.USub(), node) 173 | 174 | class Vis(gast.NodeTransformer): 175 | 176 | def __init__(self): 177 | self.state = [] 178 | 179 | def visit_Constant(self, node): 180 | self.state.append(node.value) 181 | 182 | vis = Vis() 183 | vis.visit(tree) 184 | 185 | self.assertEqual(vis.state, [6]) 186 | 187 | def test_NodeConstructor(self): 188 | node0 = gast.Name() 189 | load = gast.Load() 190 | node1 = gast.Name('id', load, None, None) 191 | node2 = gast.Name('id', load, None, type_comment=None) 192 | with self.assertRaises(TypeError): 193 | node1 = gast.Name('id', 'ctx', 'annotation', 'type_comment', 194 | 'random_field') 195 | for field in gast.Name._fields: 196 | self.assertEqual(getattr(node1, field), getattr(node2, field)) 197 | 198 | 199 | if __name__ == '__main__': 200 | unittest.main() 201 | -------------------------------------------------------------------------------- /tests/test_compat.py: -------------------------------------------------------------------------------- 1 | import unittest 2 | 3 | import gast 4 | import sys 5 | 6 | 7 | def dump(node): 8 | return gast.dump(node, show_empty=True) 9 | 10 | 11 | class CompatTestCase(unittest.TestCase): 12 | 13 | def __init__(self, *args, **kwargs): 14 | unittest.TestCase.__init__(self, *args, **kwargs) 15 | self.maxDiff = None 16 | 17 | if sys.version_info.major == 2: 18 | 19 | def test_FunctionDef(self): 20 | code = 'def foo((x, y)): return x, y' 21 | tree = gast.parse(code) 22 | compile(gast.gast_to_ast(tree), '', 'exec') 23 | norm = ("Module(body=[FunctionDef(name='foo', args=arguments(args=" 24 | "[Tuple(elts=[Name(id='x', ctx=Store(), annotation=None, " 25 | "type_comment=None), Name(id='y', ctx=Store(), " 26 | "annotation=None, type_comment=None)], ctx=Store())], " 27 | "posonlyargs=[], vararg=None, " 28 | "kwonlyargs=[], kw_defaults=[], kwarg=None, defaults=[]), " 29 | "body=[Return(value=Tuple(elts=[Name(id='x', ctx=Load(), " 30 | "annotation=None, type_comment=None), " 31 | "Name(id='y', ctx=Load(), " 32 | "annotation=None, type_comment=None" 33 | ")], ctx=Load()))], decorator_list=" 34 | "[], returns=None, type_comment=None, type_params=[])], " 35 | "type_ignores=[])") 36 | self.assertEqual(dump(tree), norm) 37 | 38 | else: 39 | 40 | def test_ArgAnnotation(self): 41 | code = 'def foo(x:int): pass' 42 | tree = gast.parse(code) 43 | compile(gast.gast_to_ast(tree), '', 'exec') 44 | norm = ("Module(body=[FunctionDef(name='foo', args=arguments(args=" 45 | "[Name(id='x', ctx=Param(), annotation=Name" 46 | "(id='int', ctx=Load(), annotation=None, type_comment=None" 47 | "), type_comment=None)], posonlyargs=" 48 | "[], vararg=None, kwonlyargs=[], kw_defaults=[], kwarg=" 49 | "None, defaults=[]), body=[Pass()], decorator_list=[], " 50 | "returns=None, type_comment=None, type_params=[])], " 51 | "type_ignores=[])") 52 | self.assertEqual(dump(tree), norm) 53 | 54 | def test_KeywordOnlyArgument(self): 55 | code = 'def foo(*, x=1): pass' 56 | tree = gast.parse(code) 57 | compile(gast.gast_to_ast(tree), '', 'exec') 58 | norm = ("Module(body=[FunctionDef(name='foo', args=arguments(args=" 59 | "[], posonlyargs=[], vararg=None, kwonlyargs=[Name" 60 | "(id='x', ctx=Param(), annotation=None, type_comment=None" 61 | ")], kw_defaults=[Constant(value=1, kind=None)], kwarg=" 62 | "None, defaults=[]), body=[Pass()], decorator_list=[], " 63 | "returns=None, type_comment=None, type_params=[])], " 64 | "type_ignores=[])") 65 | self.assertEqual(dump(tree), norm) 66 | 67 | if sys.version_info.minor >= 6: 68 | 69 | def test_FormattedValue(self): 70 | code = 'e = 1; f"{e}"' 71 | tree = gast.parse(code) 72 | compile(gast.gast_to_ast(tree), '', 'exec') 73 | norm = ("Module(body=[Assign(targets=[Name(id='e', ctx=Store()" 74 | ", annotation=None, type_comment=None" 75 | ")], value=Constant(value=1, kind=None), " 76 | "type_comment=None), Expr(value=" 77 | "JoinedStr(values=[FormattedValue(value=Name(id='e', " 78 | "ctx=Load(), annotation=None, type_comment=None), " 79 | "conversion=-1, format_spec=None)]))], " 80 | "type_ignores=[])") 81 | self.assertEqual(dump(tree), norm) 82 | 83 | def test_JoinedStr(self): 84 | code = 'e = 1; f"e = {e}"' 85 | tree = gast.parse(code) 86 | compile(gast.gast_to_ast(tree), '', 'exec') 87 | norm = ("Module(body=[Assign(targets=[Name(id='e', ctx=Store()" 88 | ", annotation=None, type_comment=None" 89 | ")], value=Constant(value=1, kind=None), " 90 | "type_comment=None), Expr(value=" 91 | "JoinedStr(values=[Constant(value='e = ', kind=None), " 92 | "FormattedValue(value=Name(id='e', ctx=Load(), " 93 | "annotation=None, type_comment=None), " 94 | "conversion=-1, format_spec=None)]))], " 95 | "type_ignores=[])") 96 | self.assertEqual(dump(tree), norm) 97 | 98 | if sys.version_info.minor >= 8: 99 | 100 | def test_TypeIgnore(self): 101 | code = 'def foo(): pass # type: ignore[excuse]' 102 | tree = gast.parse(code, type_comments=True) 103 | compile(gast.gast_to_ast(tree), '', 'exec') 104 | norm = ("Module(body=[FunctionDef(name='foo', args=arguments(" 105 | "args=[], posonlyargs=[], vararg=None, kwonlyargs=[], " 106 | "kw_defaults=[], kwarg=None, defaults=[]), body=[" 107 | "Pass()], decorator_list=[], returns=None, " 108 | "type_comment=None, type_params=[])], type_ignores=" 109 | "[TypeIgnore(lineno=1, tag='[excuse]')])") 110 | self.assertEqual(dump(tree), norm) 111 | 112 | def test_PosonlyArgs(self): 113 | code = 'def foo(a, /, b): pass' 114 | tree = gast.parse(code, type_comments=True) 115 | compile(gast.gast_to_ast(tree), '', 'exec') 116 | norm = ("Module(body=[FunctionDef(name='foo', args=arguments(" 117 | "args=[Name(id='b', ctx=Param(), annotation=None, " 118 | "type_comment=None)], posonlyargs=[Name(id='a', " 119 | "ctx=Param(), annotation=None, type_comment=None)], " 120 | "vararg=None, kwonlyargs=[], kw_defaults=[], " 121 | "kwarg=None, defaults=[]), body=[Pass()], " 122 | "decorator_list=[], returns=None, type_comment=None, " 123 | "type_params=[])], type_ignores=[])") 124 | self.assertEqual(dump(tree), norm) 125 | 126 | def test_NamedExpr(self): 127 | code = '(x := 1) ' 128 | tree = gast.parse(code) 129 | compile(gast.gast_to_ast(tree), '', 'exec') 130 | norm = ("Module(body=[Expr(value=NamedExpr(target=Name(id='x'," 131 | " ctx=Store(), annotation=None, type_comment=None), " 132 | "value=Constant(value=1, kind=None)))], type_ignores=" 133 | "[])") 134 | self.assertEqual(dump(tree), norm) 135 | 136 | if sys.version_info.minor >= 10: 137 | 138 | def test_MatchValue(self): 139 | code = 'match v:\n case "hello":...' 140 | tree = gast.parse(code) 141 | compile(gast.gast_to_ast(tree), '', 'exec') 142 | norm = ("Module(body=[Match(subject=Name(id='v', ctx=Load" 143 | "(), annotation=None, type_comment=None), cases=" 144 | "[match_case(pattern=MatchValue(value=Constant(" 145 | "value='hello', kind=None)), guard=None, body=" 146 | "[Expr(value=Constant(value=Ellipsis, kind=None))]" 147 | ")])], type_ignores=[])" 148 | ) 149 | self.assertEqual(dump(tree), norm) 150 | 151 | def test_MatchSingleton(self): 152 | code = 'match v:\n case None:...' 153 | tree = gast.parse(code) 154 | compile(gast.gast_to_ast(tree), '', 'exec') 155 | norm = ("Module(body=[Match(subject=Name(id='v', ctx=Load" 156 | "(), annotation=None, type_comment=None), cases=[" 157 | "match_case(pattern=MatchSingleton(value=None), " 158 | "guard=None, body=[Expr(value=Constant(value=" 159 | "Ellipsis, kind=None))])])], type_ignores=[])") 160 | self.assertEqual(dump(tree), norm) 161 | 162 | def test_MatchSequence(self): 163 | code = 'match v:\n case a, b:...' 164 | tree = gast.parse(code) 165 | compile(gast.gast_to_ast(tree), '', 'exec') 166 | norm = ("Module(body=[Match(subject=Name(id='v', ctx=Load" 167 | "(), annotation=None, type_comment=None), cases=" 168 | "[match_case(pattern=MatchSequence(patterns=[" 169 | "MatchAs(pattern=None, name='a'), MatchAs(pattern" 170 | "=None, name='b')]), guard=None, body=[Expr(value" 171 | "=Constant(value=Ellipsis, kind=None))])])], " 172 | "type_ignores=[])") 173 | self.assertEqual(dump(tree), norm) 174 | 175 | def test_MatchMapping(self): 176 | code = 'match v:\n case {1: a}:...' 177 | tree = gast.parse(code) 178 | compile(gast.gast_to_ast(tree), '', 'exec') 179 | norm = ("Module(body=[Match(subject=Name(id='v', ctx=Load" 180 | "(), annotation=None, type_comment=None), cases=[" 181 | "match_case(pattern=MatchMapping(keys=[Constant(" 182 | "value=1, kind=None)], patterns=[MatchAs(pattern" 183 | "=None, name='a')], rest=None), guard=None, body=" 184 | "[Expr(value=Constant(value=Ellipsis, kind=None))]" 185 | ")])], type_ignores=[])") 186 | self.assertEqual(dump(tree), norm) 187 | 188 | def test_MatchClass(self): 189 | code = 'match v:\n case Cls(attr=1):...' 190 | tree = gast.parse(code) 191 | compile(gast.gast_to_ast(tree), '', 'exec') 192 | norm = ("Module(body=[Match(subject=Name(id='v', ctx=Load" 193 | "(), annotation=None, type_comment=None), cases=[" 194 | "match_case(pattern=MatchClass(cls=Name(id='Cls'" 195 | ", ctx=Load(), annotation=None, type_comment=None" 196 | "), patterns=[], kwd_attrs=['attr'], kwd_patterns" 197 | "=[MatchValue(value=Constant(value=1, kind=None))" 198 | "]), guard=None, body=[Expr(value=Constant(value=" 199 | "Ellipsis, kind=None))])])], type_ignores=[])") 200 | self.assertEqual(dump(tree), norm) 201 | 202 | def test_MatchStar(self): 203 | code = 'match v:\n case [1, *other]:...' 204 | tree = gast.parse(code) 205 | compile(gast.gast_to_ast(tree), '', 'exec') 206 | norm = ("Module(body=[Match(subject=Name(id='v', ctx=Load" 207 | "(), annotation=None, type_comment=None), cases=[" 208 | "match_case(pattern=MatchSequence(patterns=[" 209 | "MatchValue(value=Constant(value=1, kind=None)), " 210 | "MatchStar(name='other')]), guard=None, body=" 211 | "[Expr(value=Constant(value=Ellipsis, kind=None)" 212 | ")])])], type_ignores=[])") 213 | self.assertEqual(dump(tree), norm) 214 | 215 | def test_MatchAs(self): 216 | code = 'match v:\n case 1, other:...' 217 | tree = gast.parse(code) 218 | compile(gast.gast_to_ast(tree), '', 'exec') 219 | norm = ("Module(body=[Match(subject=Name(id='v', ctx=Load" 220 | "(), annotation=None, type_comment=None), cases=[" 221 | "match_case(pattern=MatchSequence(patterns=[" 222 | "MatchValue(value=Constant(value=1, kind=None)), " 223 | "MatchAs(pattern=None, name='other')]), guard=None" 224 | ", body=[Expr(value=Constant(value=Ellipsis, kind" 225 | "=None))])])], type_ignores=[])") 226 | self.assertEqual(dump(tree), norm) 227 | 228 | def test_MatchOr(self): 229 | code = 'match v:\n case 1 | 2:...' 230 | tree = gast.parse(code) 231 | compile(gast.gast_to_ast(tree), '', 'exec') 232 | norm = ("Module(body=[Match(subject=Name(id='v', ctx=Load" 233 | "(), annotation=None, type_comment=None), cases=[" 234 | "match_case(pattern=MatchOr(patterns=[MatchValue(" 235 | "value=Constant(value=1, kind=None)), MatchValue(" 236 | "value=Constant(value=2, kind=None))]), guard=" 237 | "None, body=[Expr(value=Constant(value=Ellipsis, " 238 | "kind=None))])])], type_ignores=[])") 239 | self.assertEqual(dump(tree), norm) 240 | 241 | 242 | if sys.version_info.minor >= 11: 243 | 244 | def test_TryStar(self): 245 | code = ''' 246 | try: ... 247 | except *ValueError: ...''' 248 | norm = ("Module(body=[TryStar(body=[Expr(value=" 249 | "Constant(value=Ellipsis))], handlers=[" 250 | "ExceptHandler(type=Name(id='ValueError', ctx" 251 | "=Load()), body=[Expr(value=Constant(value=" 252 | "Ellipsis))])], orelse=[], finalbody=[])], " 253 | "type_ignores=[])") 254 | pass 255 | 256 | else: 257 | 258 | def test_Bytes(self): 259 | code = 'b"0012"' 260 | tree = gast.parse(code) 261 | compile(gast.gast_to_ast(tree), '', 'exec') 262 | norm = ("Module(body=[Expr(value=Constant(value=b'0012', " 263 | "kind=None))], type_ignores=[])") 264 | self.assertEqual(dump(tree), norm) 265 | 266 | # common 267 | 268 | def test_TryExcept(self): 269 | code = 'try:pass\nexcept e:pass\nelse:pass' 270 | tree = gast.parse(code) 271 | compile(gast.gast_to_ast(tree), '', 'exec') 272 | norm = ("Module(body=[Try(body=[Pass()], handlers=[ExceptHandler(" 273 | "type=Name(id='e', ctx=Load(), annotation=None, " 274 | "type_comment=None), name=None, body=[Pass()])]" 275 | ", orelse=[Pass()], finalbody=[])], type_ignores=[])") 276 | self.assertEqual(dump(tree), norm) 277 | 278 | def test_TryExceptNamed(self): 279 | code = 'try:pass\nexcept e as f:pass\nelse:pass' 280 | tree = gast.parse(code) 281 | compile(gast.gast_to_ast(tree), '', 'exec') 282 | norm = ("Module(body=[Try(body=[Pass()], handlers=[ExceptHandler(" 283 | "type=Name(id='e', ctx=Load(), annotation=None, " 284 | "type_comment=None), name=Name(id='f', ctx=" 285 | "Store(), annotation=None, type_comment=None), body=[Pass()])]" 286 | ", orelse=[Pass()], finalbody=[])], type_ignores=[])") 287 | self.assertEqual(dump(tree), norm) 288 | 289 | def test_Raise(self): 290 | codes = ('raise Exception', 291 | 'raise "Exception"', 292 | 'raise Exception, "err"', 293 | 'raise Exception("err")', 294 | 'raise E, V, T',) 295 | norms = ("Module(body=[Raise(exc=Name(id='Exception', ctx=Load(), " 296 | "annotation=None, type_comment=None)," 297 | " cause=None)], type_ignores=[])", 298 | 299 | "Module(body=[Raise(exc=Constant(value='Exception', kind=" 300 | "None), cause=None)], type_ignores=[])", 301 | 302 | "Module(body=[Raise(exc=Call(func=Name(id='Exception', " 303 | "ctx=Load(), annotation=None, type_comment=None), " 304 | "args=[Constant(value='err', kind=None)], " 305 | "keywords=[]), cause=None)], type_ignores=[])", 306 | 307 | "Module(body=[Raise(exc=Call(func=Name(id='Exception', " 308 | "ctx=Load(), annotation=None, type_comment=None), " 309 | "args=[Constant(value='err', kind=None)], " 310 | "keywords=[]), cause=None)], type_ignores=[])", 311 | 312 | "Module(body=[Raise(exc=Call(func=Attribute(value=Call(" 313 | "func=Name(id='E', ctx=Load(), annotation=None, " 314 | "type_comment=None), args=[Name(id='V', ctx=" 315 | "Load(), annotation=None, type_comment=None)], keywords=[]), " 316 | "attr='with_traceback', ctx=Load" 317 | "()), args=[Name(id='T', ctx=Load(), annotation=None, " 318 | "type_comment=None)], keywords=[]), " 319 | "cause=None)], type_ignores=[])",) 320 | 321 | if sys.version_info.major == 3: 322 | codes = codes[0], codes[1], codes[3] 323 | norms = norms[0], norms[1], norms[3] 324 | 325 | for code, norm in zip(codes, norms): 326 | tree = gast.parse(code) 327 | compile(gast.gast_to_ast(tree), '', 'exec') 328 | self.assertEqual(dump(tree), norm) 329 | 330 | def test_Call(self): 331 | code = 'foo(x, y=1, *args, **kwargs)' 332 | tree = gast.parse(code) 333 | compile(gast.gast_to_ast(tree), '', 'exec') 334 | norm = ("Module(body=[Expr(value=Call(func=Name(id='foo', ctx=Load" 335 | "(), annotation=None, type_comment=None" 336 | "), args=[Name(id='x', ctx=Load(), " 337 | "annotation=None, type_comment=None), Starred(value=Name(" 338 | "id='args', ctx=Load(), annotation=None, type_comment=None)" 339 | ", ctx=Load())], keywords=[keyword(" 340 | "arg='y', value=Constant(value=1, kind=None)), keyword(arg" 341 | "=None, value=Name(id='kwargs', ctx=Load(), annotation=None, " 342 | "type_comment=None))]))], type_ignores=[])") 343 | self.assertEqual(dump(tree), norm) 344 | 345 | def test_With(self): 346 | code = 'with open("any"): pass' 347 | tree = gast.parse(code) 348 | compile(gast.gast_to_ast(tree), '', 'exec') 349 | norm = ("Module(body=[With(items=[withitem(context_expr=Call(func=" 350 | "Name(id='open', ctx=Load(), annotation=None, " 351 | "type_comment=None), args=[Constant(value='any', " 352 | "kind=None)], keywords=[]), optional_vars=None)], body=[" 353 | "Pass()], type_comment=None)], type_ignores=[])") 354 | self.assertEqual(dump(tree), norm) 355 | 356 | def test_TryFinally(self): 357 | code = 'try:pass\nfinally:pass' 358 | tree = gast.parse(code) 359 | compile(gast.gast_to_ast(tree), '', 'exec') 360 | norm = ("Module(body=[Try(body=[Pass()], handlers=[], orelse=[], " 361 | "finalbody=[Pass()])], type_ignores=[])") 362 | self.assertEqual(dump(tree), norm) 363 | 364 | def test_star_argument(self): 365 | code = 'def foo(*a): pass' 366 | tree = gast.parse(code) 367 | compile(gast.gast_to_ast(tree), '', 'exec') 368 | norm = ("Module(body=[FunctionDef(name='foo', args=arguments(args=[], " 369 | "posonlyargs=[], vararg=Name(id='a', ctx=Param(), " 370 | "annotation=None, type_comment=None), " 371 | "kwonlyargs=[], kw_defaults=[], kwarg=None, defaults=[]), " 372 | "body=[Pass()], decorator_list=[], returns=None, " 373 | "type_comment=None, type_params=[])], type_ignores=[])") 374 | self.assertEqual(dump(tree), norm) 375 | 376 | def test_keyword_argument(self): 377 | code = 'def foo(**a): pass' 378 | tree = gast.parse(code) 379 | compile(gast.gast_to_ast(tree), '', 'exec') 380 | norm = ("Module(body=[FunctionDef(name='foo', args=arguments(args=[], " 381 | "posonlyargs=[], vararg=None, kwonlyargs=[], kw_defaults=[], " 382 | "kwarg=Name(id='a', ctx=Param(), annotation=None, " 383 | "type_comment=None), defaults=[]), body=[Pass()], " 384 | "decorator_list=[], returns=None, type_comment=None, " 385 | "type_params=[])], type_ignores=[])") 386 | self.assertEqual(dump(tree), norm) 387 | 388 | def test_Index(self): 389 | code = 'def foo(a): a[1]' 390 | tree = gast.parse(code) 391 | compile(gast.gast_to_ast(tree), '', 'exec') 392 | norm = ("Module(body=[FunctionDef(name='foo', args=arguments(args=[" 393 | "Name(id='a', ctx=Param(), annotation=None, type_comment=None)" 394 | "], posonlyargs=[], vararg=None, kwonlyargs=[], kw_defaults=[]" 395 | ", kwarg=None, defaults=[]), body=[Expr(value=Subscript(value=" 396 | "Name(id='a', ctx=Load(), annotation=None, type_comment=None)" 397 | ", slice=Constant(value=1, kind=None), ctx=Load()" 398 | "))], decorator_list=[], returns=None, type_comment=None, " 399 | "type_params=[])], type_ignores=[])") 400 | self.assertEqual(dump(tree), norm) 401 | 402 | def test_ExtSlice(self): 403 | code = 'def foo(a): a[:,:]' 404 | tree = gast.parse(code) 405 | compile(gast.gast_to_ast(tree), '', 'exec') 406 | norm = ("Module(body=[FunctionDef(name='foo', args=arguments(args=[" 407 | "Name(id='a', ctx=Param(), annotation=None, type_comment=None)" 408 | "], posonlyargs=[], vararg=None, kwonlyargs=[], kw_defaults=[]" 409 | ", kwarg=None, defaults=[]), body=[Expr(value=Subscript(value=" 410 | "Name(id='a', ctx=Load(), annotation=None, type_comment=None)" 411 | ", slice=Tuple(elts=[Slice(lower=None, upper=None, step=" 412 | "None), Slice(lower=None, upper=None, step=None)], ctx=Load())" 413 | ", ctx=Load()))], decorator_list=[], returns=None, " 414 | "type_comment=None, type_params=[])], type_ignores=[])") 415 | self.assertEqual(dump(tree), norm) 416 | 417 | def test_ExtSlices(self): 418 | code = 'def foo(a): a[1,:]' 419 | tree = gast.parse(code) 420 | compile(gast.gast_to_ast(tree), '', 'exec') 421 | norm = ("Module(body=[FunctionDef(name='foo', args=arguments(args=[" 422 | "Name(id='a', ctx=Param(), annotation=None, type_comment=None)" 423 | "], posonlyargs=[], vararg=None, kwonlyargs=[], kw_defaults=[]" 424 | ", kwarg=None, defaults=[]), body=[Expr(value=Subscript(value=" 425 | "Name(id='a', ctx=Load(), annotation=None, type_comment=None)" 426 | ", slice=Tuple(elts=[Constant(value=1, kind=" 427 | "None), Slice(lower=None, upper=None, step=None)], ctx=Load())" 428 | ", ctx=Load()))], decorator_list=[], returns=None, " 429 | "type_comment=None, type_params=[])], type_ignores=[])") 430 | self.assertEqual(dump(tree), norm) 431 | 432 | def test_Ellipsis(self): 433 | code = 'def foo(a): a[...]' 434 | tree = gast.parse(code) 435 | compile(gast.gast_to_ast(tree), '', 'exec') 436 | norm = ("Module(body=[FunctionDef(name='foo', args=arguments(args=[" 437 | "Name(id='a', ctx=Param(), annotation=None, type_comment=None)" 438 | "], posonlyargs=[], vararg=None, kwonlyargs=[], kw_defaults=[]" 439 | ", kwarg=None, defaults=[]), body=[Expr(value=Subscript(value=" 440 | "Name(id='a', ctx=Load(), annotation=None, type_comment=None)" 441 | ", slice=Constant(value=Ellipsis, kind=None), ctx=Load()))], " 442 | "decorator_list=[], returns=None, type_comment=None, " 443 | "type_params=[])], type_ignores=[])") 444 | self.assertEqual(dump(tree), norm) 445 | 446 | def test_ExtSliceEllipsis(self): 447 | code = 'def foo(a): a[1, ...]' 448 | tree = gast.parse(code) 449 | compile(gast.gast_to_ast(tree), '', 'exec') 450 | norm = ("Module(body=[FunctionDef(name='foo', args=arguments(args=[" 451 | "Name(id='a', ctx=Param(), annotation=None, type_comment=None)" 452 | "], posonlyargs=[], vararg=None, kwonlyargs=[], kw_defaults=[]" 453 | ", kwarg=None, defaults=[]), body=[Expr(value=Subscript(value=" 454 | "Name(id='a', ctx=Load(), annotation=None, type_comment=None)" 455 | ", slice=Tuple(elts=[Constant(value=1, kind=None)" 456 | ", Constant(value=Ellipsis, kind=None)], ctx=Load()), ctx=" 457 | "Load()))], decorator_list=[], returns=None, type_comment=" 458 | "None, type_params=[])], type_ignores=[])") 459 | self.assertEqual(dump(tree), norm) 460 | 461 | def test_ClassDef(self): 462 | code = 'class Foo: pass' 463 | tree = gast.parse(code) 464 | compile(gast.gast_to_ast(tree), '', 'exec') 465 | norm = ("Module(body=[ClassDef(name='Foo', bases=[], keywords=[], " 466 | "body=[Pass()], decorator_list=[], type_params=[])], " 467 | "type_ignores=[])") 468 | self.assertEqual(dump(tree), norm) 469 | 470 | 471 | if __name__ == '__main__': 472 | unittest.main() 473 | -------------------------------------------------------------------------------- /tests/test_py3_12.py: -------------------------------------------------------------------------------- 1 | import unittest 2 | 3 | import ast 4 | import gast 5 | import sys 6 | 7 | def dump(node): 8 | return gast.dump(node, show_empty=True) 9 | 10 | 11 | class Python3_12TestCase(unittest.TestCase): 12 | 13 | def __init__(self, *args, **kwargs): 14 | unittest.TestCase.__init__(self, *args, **kwargs) 15 | self.maxDiff = None 16 | 17 | def test_type_alias(self): 18 | code = "type Point = tuple[float, float]" 19 | tree = gast.parse(code) 20 | compile(gast.gast_to_ast(tree), '', 'exec') 21 | norm = ("Module(body=[TypeAlias(name=Name(id='Point', ctx=Store()," 22 | " annotation=None, type_comment=None), type_params=[], " 23 | "value=Subscript(value=Name(id='tuple', ctx=Load(), " 24 | "annotation=None, type_comment=None), slice=Tuple(elts=[" 25 | "Name(id='float', ctx=Load(), annotation=None, " 26 | "type_comment=None), Name(id='float', ctx=Load(), " 27 | "annotation=None, type_comment=None)], ctx=Load()), " 28 | "ctx=Load()))], type_ignores=[])") 29 | self.assertEqual(dump(tree), norm) 30 | 31 | def test_generic_type_alias(self): 32 | code = "type Point[T] = tuple[T, float]" 33 | tree = gast.parse(code) 34 | compile(gast.gast_to_ast(tree), '', 'exec') 35 | norm = ("Module(body=[TypeAlias(name=Name(id='Point', ctx=Store(), " 36 | "annotation=None, type_comment=None), type_params=[TypeVar(" 37 | "name='T', bound=None)], value=Subscript(value=Name(id='tuple'" 38 | ", ctx=Load(), annotation=None, type_comment=None), " 39 | "slice=Tuple(elts=[Name(id='T', ctx=Load(), annotation=None, " 40 | "type_comment=None), Name(id='float', ctx=Load(), " 41 | "annotation=None, type_comment=None)], ctx=Load()), ctx=Load()" 42 | "))], type_ignores=[])") 43 | self.assertEqual(dump(tree), norm) 44 | 45 | def test_generic_function(self): 46 | code = "def foo[T]():..." 47 | tree = gast.parse(code) 48 | compile(gast.gast_to_ast(tree), '', 'exec') 49 | norm = ("Module(body=[FunctionDef(name='foo', args=arguments(args=[], " 50 | "posonlyargs=[], vararg=None, kwonlyargs=[], kw_defaults=[], " 51 | "kwarg=None, defaults=[]), body=[Expr(value=Constant(value=" 52 | "Ellipsis, kind=None))], decorator_list=[], returns=None, " 53 | "type_comment=None, type_params=[TypeVar(name='T', " 54 | "bound=None)])], type_ignores=[])") 55 | self.assertEqual(dump(tree), norm) 56 | 57 | def test_generic_class(self): 58 | code = "class foo[T]:..." 59 | tree = gast.parse(code) 60 | compile(gast.gast_to_ast(tree), '', 'exec') 61 | norm = ("Module(body=[ClassDef(name='foo', bases=[], keywords=[], " 62 | "body=[Expr(value=Constant(value=Ellipsis, kind=None))], " 63 | "decorator_list=[], type_params=[TypeVar(name='T', bound=None)" 64 | "])], type_ignores=[])") 65 | self.assertEqual(dump(tree), norm) 66 | 67 | if sys.version_info < (3, 12): 68 | del Python3_12TestCase 69 | 70 | if __name__ == '__main__': 71 | unittest.main() 72 | -------------------------------------------------------------------------------- /tests/test_self.py: -------------------------------------------------------------------------------- 1 | import glob 2 | import os 3 | import unittest 4 | 5 | import gast 6 | 7 | 8 | class SelfTestCase(unittest.TestCase): 9 | 10 | def setUp(self): 11 | self.srcs = glob.glob(os.path.join(gast.__path__[0], '*.py')) 12 | 13 | def testParse(self): 14 | for src_py in self.srcs: 15 | with open(src_py) as f: 16 | content = f.read() 17 | gast.parse(content) 18 | 19 | def testCompile(self): 20 | for src_py in self.srcs: 21 | with open(src_py) as f: 22 | content = f.read() 23 | gnode = gast.parse(content) 24 | compile(gast.gast_to_ast(gnode), src_py, 'exec') 25 | 26 | def test_unparse(self): 27 | for src_py in self.srcs: 28 | with open(src_py) as f: 29 | content = f.read() 30 | gnode = gast.parse(content) 31 | gast.unparse(gnode) 32 | 33 | 34 | if __name__ == '__main__': 35 | unittest.main() 36 | -------------------------------------------------------------------------------- /tests/test_unparser.py: -------------------------------------------------------------------------------- 1 | import unittest 2 | 3 | import ast 4 | import gast 5 | import sys 6 | 7 | 8 | class UnparserTestCase(unittest.TestCase): 9 | 10 | def __init__(self, *args, **kwargs): 11 | unittest.TestCase.__init__(self, *args, **kwargs) 12 | self.maxDiff = None 13 | 14 | def assertUnparse(self, code): 15 | normalized_code = ast.unparse(ast.parse(code)) 16 | tree = gast.parse(normalized_code) 17 | compile(gast.gast_to_ast(tree), '', 'exec') 18 | unparsed = gast.unparse(tree) 19 | self.assertEqual(normalized_code, unparsed) 20 | 21 | def test_FunctionDef(self): 22 | self.assertUnparse('def foo(x, y): return x, y') 23 | 24 | def test_BinaryOp(self): 25 | self.assertUnparse('1 + 3') 26 | 27 | if sys.version_info >= (3, 12): 28 | 29 | def test_TypeParameter(self): 30 | self.assertUnparse('type x[T] = list[T]') 31 | 32 | 33 | if sys.version_info < (3, 9): 34 | del UnparserTestCase 35 | 36 | if __name__ == '__main__': 37 | unittest.main() 38 | -------------------------------------------------------------------------------- /tox.ini: -------------------------------------------------------------------------------- 1 | [tox] 2 | envlist = py{27,34,35,36,37,38,39,310, 311, 312},lint 3 | 4 | [testenv] 5 | deps = 6 | pytest 7 | commands=pytest {posargs} 8 | 9 | [testenv:lint] 10 | deps = 11 | pytest-pep8 12 | pytest < 5 13 | commands=pytest --pep8 -m pep8 14 | --------------------------------------------------------------------------------