├── LICENSE
├── README.md
├── pythonparser
└── requirements.txt
/LICENSE:
--------------------------------------------------------------------------------
1 | GNU LESSER GENERAL PUBLIC LICENSE
2 | Version 3, 29 June 2007
3 |
4 | Copyright (C) 2007 Free Software Foundation, Inc.
5 | Everyone is permitted to copy and distribute verbatim copies
6 | of this license document, but changing it is not allowed.
7 |
8 |
9 | This version of the GNU Lesser General Public License incorporates
10 | the terms and conditions of version 3 of the GNU General Public
11 | License, supplemented by the additional permissions listed below.
12 |
13 | 0. Additional Definitions.
14 |
15 | As used herein, "this License" refers to version 3 of the GNU Lesser
16 | General Public License, and the "GNU GPL" refers to version 3 of the GNU
17 | General Public License.
18 |
19 | "The Library" refers to a covered work governed by this License,
20 | other than an Application or a Combined Work as defined below.
21 |
22 | An "Application" is any work that makes use of an interface provided
23 | by the Library, but which is not otherwise based on the Library.
24 | Defining a subclass of a class defined by the Library is deemed a mode
25 | of using an interface provided by the Library.
26 |
27 | A "Combined Work" is a work produced by combining or linking an
28 | Application with the Library. The particular version of the Library
29 | with which the Combined Work was made is also called the "Linked
30 | Version".
31 |
32 | The "Minimal Corresponding Source" for a Combined Work means the
33 | Corresponding Source for the Combined Work, excluding any source code
34 | for portions of the Combined Work that, considered in isolation, are
35 | based on the Application, and not on the Linked Version.
36 |
37 | The "Corresponding Application Code" for a Combined Work means the
38 | object code and/or source code for the Application, including any data
39 | and utility programs needed for reproducing the Combined Work from the
40 | Application, but excluding the System Libraries of the Combined Work.
41 |
42 | 1. Exception to Section 3 of the GNU GPL.
43 |
44 | You may convey a covered work under sections 3 and 4 of this License
45 | without being bound by section 3 of the GNU GPL.
46 |
47 | 2. Conveying Modified Versions.
48 |
49 | If you modify a copy of the Library, and, in your modifications, a
50 | facility refers to a function or data to be supplied by an Application
51 | that uses the facility (other than as an argument passed when the
52 | facility is invoked), then you may convey a copy of the modified
53 | version:
54 |
55 | a) under this License, provided that you make a good faith effort to
56 | ensure that, in the event an Application does not supply the
57 | function or data, the facility still operates, and performs
58 | whatever part of its purpose remains meaningful, or
59 |
60 | b) under the GNU GPL, with none of the additional permissions of
61 | this License applicable to that copy.
62 |
63 | 3. Object Code Incorporating Material from Library Header Files.
64 |
65 | The object code form of an Application may incorporate material from
66 | a header file that is part of the Library. You may convey such object
67 | code under terms of your choice, provided that, if the incorporated
68 | material is not limited to numerical parameters, data structure
69 | layouts and accessors, or small macros, inline functions and templates
70 | (ten or fewer lines in length), you do both of the following:
71 |
72 | a) Give prominent notice with each copy of the object code that the
73 | Library is used in it and that the Library and its use are
74 | covered by this License.
75 |
76 | b) Accompany the object code with a copy of the GNU GPL and this license
77 | document.
78 |
79 | 4. Combined Works.
80 |
81 | You may convey a Combined Work under terms of your choice that,
82 | taken together, effectively do not restrict modification of the
83 | portions of the Library contained in the Combined Work and reverse
84 | engineering for debugging such modifications, if you also do each of
85 | the following:
86 |
87 | a) Give prominent notice with each copy of the Combined Work that
88 | the Library is used in it and that the Library and its use are
89 | covered by this License.
90 |
91 | b) Accompany the Combined Work with a copy of the GNU GPL and this license
92 | document.
93 |
94 | c) For a Combined Work that displays copyright notices during
95 | execution, include the copyright notice for the Library among
96 | these notices, as well as a reference directing the user to the
97 | copies of the GNU GPL and this license document.
98 |
99 | d) Do one of the following:
100 |
101 | 0) Convey the Minimal Corresponding Source under the terms of this
102 | License, and the Corresponding Application Code in a form
103 | suitable for, and under terms that permit, the user to
104 | recombine or relink the Application with a modified version of
105 | the Linked Version to produce a modified Combined Work, in the
106 | manner specified by section 6 of the GNU GPL for conveying
107 | Corresponding Source.
108 |
109 | 1) Use a suitable shared library mechanism for linking with the
110 | Library. A suitable mechanism is one that (a) uses at run time
111 | a copy of the Library already present on the user's computer
112 | system, and (b) will operate properly with a modified version
113 | of the Library that is interface-compatible with the Linked
114 | Version.
115 |
116 | e) Provide Installation Information, but only if you would otherwise
117 | be required to provide such information under section 6 of the
118 | GNU GPL, and only to the extent that such information is
119 | necessary to install and execute a modified version of the
120 | Combined Work produced by recombining or relinking the
121 | Application with a modified version of the Linked Version. (If
122 | you use option 4d0, the Installation Information must accompany
123 | the Minimal Corresponding Source and Corresponding Application
124 | Code. If you use option 4d1, you must provide the Installation
125 | Information in the manner specified by section 6 of the GNU GPL
126 | for conveying Corresponding Source.)
127 |
128 | 5. Combined Libraries.
129 |
130 | You may place library facilities that are a work based on the
131 | Library side by side in a single library together with other library
132 | facilities that are not Applications and are not covered by this
133 | License, and convey such a combined library under terms of your
134 | choice, if you do both of the following:
135 |
136 | a) Accompany the combined library with a copy of the same work based
137 | on the Library, uncombined with any other library facilities,
138 | conveyed under the terms of this License.
139 |
140 | b) Give prominent notice with the combined library that part of it
141 | is a work based on the Library, and explaining where to find the
142 | accompanying uncombined form of the same work.
143 |
144 | 6. Revised Versions of the GNU Lesser General Public License.
145 |
146 | The Free Software Foundation may publish revised and/or new versions
147 | of the GNU Lesser General Public License from time to time. Such new
148 | versions will be similar in spirit to the present version, but may
149 | differ in detail to address new problems or concerns.
150 |
151 | Each version is given a distinguishing version number. If the
152 | Library as you received it specifies that a certain numbered version
153 | of the GNU Lesser General Public License "or any later version"
154 | applies to it, you have the option of following the terms and
155 | conditions either of that published version or of any later version
156 | published by the Free Software Foundation. If the Library as you
157 | received it does not specify a version number of the GNU Lesser
158 | General Public License, you may choose any version of the GNU Lesser
159 | General Public License ever published by the Free Software Foundation.
160 |
161 | If the Library as you received it specifies that a proxy can decide
162 | whether future versions of the GNU Lesser General Public License shall
163 | apply, that proxy's public statement of acceptance of any version is
164 | permanent authorization for you to choose that version for the
165 | Library.
166 |
--------------------------------------------------------------------------------
/README.md:
--------------------------------------------------------------------------------
1 | # pythonparser
2 |
3 | Pythonparser is a python tool that produces a GumTree compatible Python AST. It uses the awesome `parso`module under the hood.
4 |
5 | ## Installation
6 |
7 | Pythonparser requires to use Python 3. To install it, just clone the repository and install the dependencies using
8 |
9 | ~~~
10 | pip3 install -r requirements.txt
11 | ~~~~
12 |
13 | Pythonpaser can be used as a standalone tool like this:
14 |
15 | ~~~
16 | pythonparser /path/to/script.py
17 | ~~~~
18 |
19 |
--------------------------------------------------------------------------------
/pythonparser:
--------------------------------------------------------------------------------
1 | #!/usr/bin/env python3
2 |
3 | # This file is part of pythonparser.
4 |
5 | # pythonparser is free software: you can redistribute it and/or modify
6 | # it under the terms of the GNU Lesser General Public License as published by
7 | # the Free Software Foundation, either version 3 of the License, or
8 | # (at your option) any later version.
9 | #
10 | # pythonparser is distributed in the hope that it will be useful,
11 | # but WITHOUT ANY WARRANTY; without even the implied warranty of
12 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
13 | # GNU Lesser General Public License for more details.
14 | #
15 | # You should have received a copy of the GNU Lesser General Public License
16 | # along with pythonparser. If not, see .
17 | #
18 | # Copyright 2020-2021 Jean-Rémy Falleri
19 |
20 | from xml.dom import minidom
21 | import parso
22 | import sys
23 |
24 | doc = minidom.Document()
25 | positions = [0]
26 |
27 | def main(file):
28 | parso_ast = parso.parse(read_file(file))
29 | gumtree_ast = to_gumtree_node(parso_ast)
30 | doc.appendChild(gumtree_ast)
31 | process_node(parso_ast, gumtree_ast)
32 | xml = doc.toprettyxml()
33 | print(xml)
34 |
35 | def process_node(parso_node, gumtree_node):
36 | if parso_node.type == 'error_node':
37 | sys.exit(parso_node)
38 |
39 | for parso_child in parso_node.children:
40 | gumtree_child = to_gumtree_node(parso_child)
41 | if gumtree_child != None:
42 | gumtree_node.appendChild(gumtree_child)
43 | if hasattr(parso_child, 'children'):
44 | process_node(parso_child, gumtree_child)
45 |
46 | def to_gumtree_node(parso_node):
47 | if parso_node.type in ['keyword', 'newline', 'endmarker']:
48 | return
49 | if parso_node.type == 'operator' and parso_node.value in ['.', '(', ')', '[', ']', ':', ';']:
50 | return
51 | gumtree_node = doc.createElement('tree')
52 | gumtree_node.setAttribute("type", parso_node.type)
53 | start_pos = positions[parso_node.start_pos[0] - 1] + parso_node.start_pos[1]
54 | end_pos = positions[parso_node.end_pos[0] - 1] + parso_node.end_pos[1]
55 | length = end_pos - start_pos
56 | gumtree_node.setAttribute("pos", str(start_pos))
57 | gumtree_node.setAttribute("length", str(length))
58 | if (not hasattr(parso_node, 'children')) or len(parso_node.children) == 0:
59 | gumtree_node.setAttribute("label", parso_node.value)
60 | return gumtree_node
61 |
62 | def read_file(file):
63 | with open(file, 'r') as file:
64 | data = file.read()
65 | index = 0
66 | for chr in data:
67 | index += 1
68 | if chr == '\n':
69 | positions.append(index)
70 | return data
71 |
72 | if __name__ == '__main__':
73 | main(sys.argv[1])
--------------------------------------------------------------------------------
/requirements.txt:
--------------------------------------------------------------------------------
1 | parso
--------------------------------------------------------------------------------