├── LICENSE ├── README.md ├── markdoc.md └── markdoc.py /LICENSE: -------------------------------------------------------------------------------- 1 | The MIT License (MIT) 2 | 3 | Copyright (c) 2016 Chris Emmery 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | 23 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # MarkDoc 2 | 3 | This piece of code can be used to convert 4 | [NumPy-styled](https://sphinxcontrib-napoleon.readthedocs.org/en/latest/example_numpy.html) Python docstrings ([example](https://github.com/scikit-learn/scikit-learn/blob/master/sklearn/neural_network/multilayer_perceptron.py#L134)), 5 | such as those used in [scikit-learn](https://www.scikit-learn.org/), to 6 | Markdown with minimum dependencies. In this way, only the code-contained 7 | documentation needs to be editted, and your documentation on for example 8 | [readthedocs](http://www.readthedocs.org/) can be automatically updated 9 | thereafter with some configuration. 10 | 11 | ## Example 12 | 13 | The converted documentation for this particular code can be found 14 | [here](https://github.com/cmry/markdoc/blob/master/markdoc.md). 15 | 16 | ## Before Use 17 | 18 | The format assumes that your code (or at least your docstring documentation) is 19 | in line with the styleguides of *both* PEP8 and PEP257. There are two ways to 20 | go about making sure this is the case: 21 | 22 | #### Fast 23 | 24 | Just check the example 25 | [here](https://github.com/cmry/markdoc/blob/master/markdoc.py#L304) to get a 26 | quick overview of how the docstrings would have to look in order for them 27 | to parse correctly. 28 | 29 | #### Better 30 | 31 | The styleguides are incorporated into two libraries that can be used to check 32 | the Python files for style errors in terminal, like so: 33 | 34 | ``` shell 35 | pip install pep8 36 | pip install pep257 37 | 38 | pep8 yourpythonfile.py 39 | pep257 yourpythonfile.py 40 | ``` 41 | 42 | These are more conveniently implemented in linter plugins such as `linter-pep8` 43 | and `linter-pep257` for the [linter](https://atom.io/users/AtomLinter/packages) 44 | package in [Atom](http://www.atom.io/), and 45 | [Flake8Lint](https://github.com/dreadatour/Flake8Lint) for 46 | [Sublime Text](https://www.sublimetext.com/) (and pretty much every other IDE). 47 | 48 | 49 | ## Usage 50 | 51 | First clone from GitHub: 52 | 53 | ``` shell 54 | git clone https://www.github.com/cmry/markdoc 55 | ``` 56 | 57 | To use the script within terminal, simply type: 58 | 59 | ``` shell 60 | python3 markdoc.py /dir/to/somefile.py /dir/to/somedoc.md 61 | ``` 62 | 63 | If you want to automatically generate a bunch of documentation and are not 64 | comfortable with writing `.sh` scripts, you can use the code in Python as well: 65 | 66 | ``` python 67 | from markdoc import MarkDoc 68 | md = MarkDoc('/dir/to/somefile.py', '/dir/to/somedoc.md') 69 | ``` 70 | 71 | You can access the converted markdown string in: 72 | 73 | ``` python 74 | # print markdown from class attribute 75 | print(md.markdown) 76 | ``` 77 | 78 | The class runs itself on initialization (calls `self.read`). If you do not 79 | want this, you can add `cold=True` to the class initialization, like so: 80 | 81 | ``` python 82 | md = MarkDoc('file.py', 'doc.md', cold=True) 83 | ``` 84 | 85 | ## Planned Fixes 86 | 87 | - Handle consecutive classes within the same file. 88 | - Fix inherrited classes being handled correctly (no `__init__`, no `object`). 89 | - Link class methods from table to their documentation. 90 | - Might not handle decorators neatly. 91 | 92 | ## Docstring Issues 93 | 94 | Some caveats: 95 | 96 | - Do not use `"""` for anything other than docstrings. 97 | - First line of any docstring has to contain the first short title 98 | ([example](https://github.com/cmry/markdoc/blob/master/markdoc.py#L162)). 99 | - Classes have to be structured new-style: `class SomeClass(object):`. 100 | - Codeblocks in examples have to be vanilla Python (`>>>` and `...`). 101 | - Please do not use `class` as a parameter name! 102 | 103 | ## Notes 104 | 105 | Script has only been tested with Python 3.4 on Ubuntu. 106 | -------------------------------------------------------------------------------- /markdoc.md: -------------------------------------------------------------------------------- 1 | # Docstring to Markdown 2 | 3 | 4 | This piece of code is used within Shed (https://www.github.com/cmry/shed) to 5 | convert sklearn-styled python docstrings to markdown. In this way, only the 6 | code documentation needs to be editted, and the readthedocs is automatically 7 | updated thereafter. To use it, simply type `python3 markdoc.py 8 | somefile.py somedoc.md`. Check class help for python usage. Don't forget to 9 | also check the class notes (they are important for docstring format 10 | guidelines). 11 | 12 | 13 | # MarkDoc 14 | 15 | ``` python 16 | class MarkDoc(file_in, file_out, cold=False) 17 | ``` 18 | 19 | Documentation to Markdown. 20 | 21 | This class converts a python file with properly formed sklearn-style 22 | docstrings to a markdown file that can be used in for example Readthedocs 23 | or jekyll. 24 | 25 | | Parameters | Type | Doc | 26 | |:-------|:-----------------|:----------------| 27 | | file_in | str | Python (.py) file from where the docstrings should be taken. | 28 | | file_out | str | Markdown (.md) file where the documentation should be written. | 29 | | cold | boolean, optional, default False | If you want to start the class without starting the reader, set True. | 30 | 31 | 32 | | Attributes | Type | Doc | 33 | |:-------|:-----------------|:----------------| 34 | | markdown | str | The entire markdown file that will be appended to and written out. | 35 | 36 | 37 | ------- 38 | 39 | ##Examples 40 | 41 | Use of the package should just be from commandline with `python3 markdoc.py 42 | in.py out.md`. However, if for some reason it is required to load it as a 43 | class, you can just do: 44 | 45 | 46 | ``` python 47 | >>> from markdoc import MarkDoc 48 | >>> d2m = MarkDoc('some/dir/file.py', '/some/otherdir/doc.md') 49 | 50 | ``` 51 | 52 | 53 | The string of the markdown can be accessed by: 54 | 55 | 56 | ``` python 57 | >>> print(d2m.markdown) 58 | 59 | ``` 60 | 61 | 62 | 63 | ------- 64 | 65 | ##Notes 66 | 67 | List of common errors and styleguide conflicts: 68 | 69 | #### list index out of range? 70 | 71 | - """ used for non-docstring strings. 72 | 73 | 74 | - First Class or method Documentation line has newline after """. 75 | 76 | 77 | Fix with: """This is the first line. 78 | 79 | 80 | Here comes the rest. 81 | 82 | 83 | """ 84 | #### Class parameters not showing in code blocks? 85 | 86 | - Class is not structed new-style (object after class). 87 | 88 | 89 | #### Code blocks not showing? 90 | 91 | - Code is ipython style rather than vanilla (i.e. In [1]: instead of >>>). 92 | 93 | 94 | #### tuple index out of range? 95 | 96 | - Mistake in var : type lists (for example var: type). 97 | 98 | 99 | - Description also contains a :. 100 | 101 | 102 | #### some table parts are not showing? 103 | 104 | - Probably included some seperators in there such as : or -----. 105 | 106 | 107 | #### methods are garbled? 108 | 109 | - Did you name a parameter 'class' or any other matches? 110 | 111 | 112 | 113 | --------- 114 | 115 | ## Methods 116 | 117 | 118 | | method | Doc | 119 | |:-------|:----------------| 120 | | read | Open python code file, spit out markdown file. | 121 | | handle_classes | Given class docs only, write out their docs and methods. | 122 | | find_declaration | Find `class Bla(object):` or `def method(params):` across lines. | 123 | | structure_doc | Produce a structured dictionary from docstring. | 124 | | md_title | Handle title and text parts for main and class-specific. | 125 | | md_table | Place parameters and attributes in a table overview. | 126 | | md_code_text | Section text and/or code in the example part of the doc. | 127 | | md_class_method | Replace `class ...(object)` with `__init__` arguments. | 128 | | md_methods_doc | Merge all method elements in one string. | 129 | | md_class_doc | Merge all class section elements in one string. | 130 | 131 | 132 | 133 | ### read 134 | 135 | ``` python 136 | read(file_in, file_out) 137 | ``` 138 | 139 | 140 | Open python code file, spit out markdown file. 141 | 142 | ### handle_classes 143 | 144 | ``` python 145 | handle_classes(classes) 146 | ``` 147 | 148 | 149 | Given class docs only, write out their docs and methods. 150 | 151 | ### find_declaration 152 | 153 | ``` python 154 | find_declaration(lines) 155 | ``` 156 | 157 | 158 | Find `class Bla(object):` or `def method(params):` across lines. 159 | 160 | ### structure_doc 161 | 162 | ``` python 163 | structure_doc(doc) 164 | ``` 165 | 166 | 167 | Produce a structured dictionary from docstring. 168 | 169 | The key is the part of the documentation (name, title, params, 170 | attributes, etc.), and value is a list of lines that make up this part. 171 | Is used to handle the order and different parses of different sections. 172 | 173 | | Parameters | Type | Doc | 174 | |:-------|:-----------------|:----------------| 175 | | doc | str | Flat string structure of a docstring block. | 176 | 177 | 178 | | Returns | Type | Doc | 179 | |:-------|:-----------------|:----------------| 180 | | X_doc | dict | Class or method doc, a structure dict with section; lines. | 181 | 182 | 183 | ### md_title 184 | 185 | ``` python 186 | md_title(title_part, class_line=False) 187 | ``` 188 | 189 | 190 | Handle title and text parts for main and class-specific. 191 | 192 | Pretty convoluted but delicate method that seperates the top part 193 | that should not be part of the title docstring off the document, 194 | handles correct punctuation of titles, for both classes and methods. 195 | Also detects if there's a description under the first line. 196 | 197 | | Parameters | Type | Doc | 198 | |:-------|:-----------------|:----------------| 199 | | title_part | str | Part of the document that starts with triple qutoes and is below any line seperators. | 200 | | class_line | bool, optional, default False | The first line of the docstring by default servers as a title header. However, for classes it should be formed into a normal description. Thefefore, the # marker is removed and a . is added. | 201 | 202 | 203 | | Returns | Type | Doc | 204 | |:-------|:-----------------|:----------------| 205 | | line_buffer | str | Correctly formated markdown from the buffer. | 206 | 207 | 208 | ### md_table 209 | 210 | ``` python 211 | md_table(doc, name) 212 | ``` 213 | 214 | 215 | Place parameters and attributes in a table overview. 216 | 217 | The documentation parts that are typed by 'var : type \n description' 218 | can be splitted into a table. The same holds true for a list of class 219 | methods. These are handled by this method. Table structures are on 220 | the top of this python file. 221 | 222 | | Parameters | Type | Doc | 223 | |:-------|:-----------------|:----------------| 224 | | doc | str | Flat string structure of a docstring block. | 225 | | name | str | Indicator for the table, can be for example Parameters or Attributes. | 226 | 227 | 228 | | Returns | Type | Doc | 229 | |:-------|:-----------------|:----------------| 230 | | table | str | Markdown-type table with filled rows and formatted name. | 231 | 232 | 233 | ### md_code_text 234 | 235 | ``` python 236 | md_code_text(doc, name, flat=False) 237 | ``` 238 | 239 | 240 | Section text and/or code in the example part of the doc. 241 | 242 | If some code related beginnings (>>>, ...) is founds buffer to code, 243 | else we regard it as text. Record order so that multiple consecutive 244 | blocks of text and code are possible. 245 | 246 | | Parameters | Type | Doc | 247 | |:-------|:-----------------|:----------------| 248 | | doc | str | Flat string structure of a docstring block. | 249 | | name | str | Name of the section (Examples, Notes). | 250 | | flat | bool, optional, default False | If there are no code blocks, flat can be used to stop parsing them. | 251 | 252 | 253 | | Returns | Type | Doc | 254 | |:-------|:-----------------|:----------------| 255 | | code | str | Formatted block of fenced markdown code-snippets and flat text. | 256 | 257 | 258 | ------- 259 | 260 | #### Notes 261 | 262 | Please note that this method assumes markdown is uses so-called 263 | fenced codeblocks that are commonly accepted by Readthedocs, Github, 264 | Bitbucket, and Jekyll (for example with Redcarpet). 265 | 266 | 267 | This current implementation does NOT allow for ipython styled code 268 | examples. (In [1]: Out[1]: etc.) 269 | 270 | 271 | ### md_class_method 272 | 273 | ``` python 274 | md_class_method(doc, class_parts) 275 | ``` 276 | 277 | 278 | Replace `class ...(object)` with `__init__` arguments. 279 | 280 | In sklearn-style, each class is represented as it's name, along with 281 | the parameters accepted in `__init__` as its parameters, as you would 282 | call the method in python. The markdown therefore needs to fill the 283 | (object) tag that is assigned to classes with the `__init__` params. 284 | 285 | | Parameters | Type | Doc | 286 | |:-------|:-----------------|:----------------| 287 | | doc | str | Flat string structure of a docstring block. | 288 | | class_parts | list | List with parts that the docstring constituted of. First line should be the class name and can therefore be replaced. | 289 | 290 | 291 | | Returns | Type | Doc | 292 | |:-------|:-----------------|:----------------| 293 | | doc | str | Same as doc, only now (object) has been replaced with `__init__` parameters. | 294 | 295 | 296 | ------- 297 | 298 | #### Notes 299 | 300 | Please note that this implementation assumes you're using new-style 301 | class declarations where for example `class SomeClass:` should be 302 | written as `class SomeClass(object):`. 303 | 304 | 305 | 306 | 307 | ### md_methods_doc 308 | 309 | ``` python 310 | md_methods_doc(method_doc) 311 | ``` 312 | 313 | 314 | Merge all method elements in one string. 315 | 316 | This method will scan each method docstring and extract parameters, 317 | returns and descriptions. It will append them to the method table 318 | and link them in the doc (TODO). 319 | 320 | | Parameters | Type | Doc | 321 | |:-------|:-----------------|:----------------| 322 | | method_doc | list | Lower part of the class docstrings that holds a method docstring per entry. | 323 | 324 | 325 | | Returns | Type | Doc | 326 | |:-------|:-----------------|:----------------| 327 | | methods | str | Formatted markdown string with all method information. | 328 | 329 | 330 | ------- 331 | 332 | #### Notes 333 | 334 | On the bottom, dict calls are being done to order several parts of 335 | the docstring. If you use more than Parameters and Returns, please make 336 | sure they are added in the code there. 337 | 338 | 339 | 340 | 341 | ### md_class_doc 342 | 343 | ``` python 344 | md_class_doc(class_doc) 345 | ``` 346 | 347 | 348 | Merge all class section elements in one string. 349 | 350 | This will piece together the descriptions contained in the docstring 351 | of the class. Currently, they are written *below* the top of the 352 | python file, and not put into any subfiles per class. 353 | 354 | | Parameters | Type | Doc | 355 | |:-------|:-----------------|:----------------| 356 | | class_doc | str | Flat text containing the class docstring part. | 357 | 358 | 359 | | Returns | Type | Doc | 360 | |:-------|:-----------------|:----------------| 361 | | mark_doc | str | Markdown formatted class docstring, including all subsections. | 362 | 363 | 364 | ------- 365 | 366 | #### Notes 367 | 368 | On the bottom, dict calls are being done to order several parts of 369 | the docstring. If you use more than Parameters, Attributes, Examples, 370 | and Notes, please make sure they are added in the code there. 371 | 372 | 373 | -------------------------------------------------------------------------------- /markdoc.py: -------------------------------------------------------------------------------- 1 | """Docstring to Markdown. 2 | 3 | This piece of code is used within [Shed](https://www.github.com/cmry/shed) to 4 | convert NumPy-styled Python docstrings to Markdown. In this way, only the 5 | code documentation needs to be editted, and the Readthedocs is automatically 6 | updated thereafter. To use it, simply type `python3 markdoc.py 7 | somefile.py somedoc.md`. Check class help for Python usage. Don't forget to 8 | also check the class notes (they are important for docstring format 9 | guidelines). 10 | """ 11 | 12 | from sys import argv 13 | 14 | # Author: Chris Emmery 15 | # License: MIT 16 | # pylint: disable=W0142,R0201 17 | 18 | ERR = ''' 19 | * list index out of range 20 | - \"\"\" used for non-docstring strings. 21 | - First Class or method Documentation line has newline after \"\"\". 22 | Fix with: \"\"\"This is the first line. 23 | 24 | Here comes the rest. 25 | \"\"\" 26 | 27 | * Class parameters not showing in code blocks 28 | - Class is not structed new-style (class SomeClass(object):) 29 | 30 | * Code blocks not showing 31 | - Code is in IPython style rather than vanilla (i.e. In [1]: instead of >>>) 32 | 33 | * tuple index out of range 34 | - Mistake in var : type lists (for example var: type). 35 | - Description also contains a :. 36 | 37 | * some table parts are not showing 38 | - Probably included some seperators in there such as : or -----. 39 | 40 | * methods are garbled 41 | - Did you name a parameter 'class' or any other matches? 42 | ''' 43 | 44 | MD_TABLE = ''' 45 | 46 | | {0} | Type | Doc | 47 | |:-------|:-----------------|:----------------| 48 | ''' 49 | MD_TABLE_ROW = '| {0} | {1} | {2} | \n' 50 | 51 | MD_TABLE_ALT = ''' 52 | | {0} | Doc | 53 | |:-------|:----------------| 54 | ''' 55 | MD_TABLE_ROW_ALT = '| {0} | {1} | \n' 56 | 57 | MD_CODE = '``` python \n {0} \n```\n\n' 58 | 59 | 60 | class MarkDoc(object): 61 | 62 | r"""Documentation to Markdown. 63 | 64 | This class converts a Python file with properly formed NumPy-styled 65 | docstrings to a markdown file that can be used in for example Readthedocs 66 | or Jekyll. 67 | 68 | Parameters 69 | ---------- 70 | file_in : str 71 | Python (.py) file from where the docstrings should be taken. 72 | 73 | file_out : str 74 | Markdown (.md) file where the documentation should be written. 75 | 76 | cold : boolean, optional, default False 77 | If you want to start the class without starting the reader, set True. 78 | 79 | Attributes 80 | ---------- 81 | markdown : str 82 | The entire Markdown file that will be appended to and written out. 83 | 84 | Examples 85 | -------- 86 | 87 | Use of the package should just be from commandline with `python3 markdoc.py 88 | in.py out.md`. However, if for some reason it is required to load it as a 89 | class, you can just do: 90 | 91 | >>> from markdoc import MarkDoc 92 | >>> md = MarkDoc('some/dir/file.py', '/some/otherdir/doc.md') 93 | 94 | The string of the markdown can be accessed by: 95 | 96 | >>> print(md.markdown) 97 | 98 | Notes 99 | ----- 100 | List of common errors and styleguide conflicts: 101 | 102 | #### list index out of range? 103 | - \"\"\" used for non-docstring strings. 104 | - First Class or method Documentation line has newline after \"\"\". 105 | Fix with: \"\"\"This is the first line. 106 | 107 | Here comes the rest. 108 | \"\"\" 109 | 110 | #### Class parameters not showing in code blocks? 111 | - Class is not structed new-style (object after class). 112 | 113 | #### Code blocks not showing? 114 | - Code is IPython style rather than vanilla (i.e. In [1]: instead of >>>). 115 | 116 | #### tuple index out of range? 117 | - Mistake in var : type lists (for example var: type). 118 | - Description also contains a :. 119 | 120 | #### some table parts are not showing? 121 | - Probably included some seperators in there such as : or \-\-\-\-\-. 122 | 123 | #### methods are garbled? 124 | - Did you name a parameter 'class' or any other matches? 125 | """ 126 | 127 | def __init__(self, file_in, file_out, cold=False): 128 | """Handle in/out to readers and parsers.""" 129 | self.markdown = '' 130 | if not cold: 131 | self.read(file_in, file_out) 132 | # FIXME: code does not handle staticmethods yet? 133 | 134 | def read(self, file_in, file_out): 135 | """Open Python code file, spit out markdown file.""" 136 | with open(file_in, 'r') as file_in: 137 | # get rid of special docstring parts 138 | classes = file_in.read().replace('r"""', '"""').split('\nclass ') 139 | self.markdown = [self.md_title(classes.pop(0))] # top of the file 140 | self.handle_classes(classes) 141 | with open(file_out, 'w') as file_out: 142 | file_out.write('\n'.join(self.markdown)) 143 | 144 | def handle_classes(self, classes): 145 | """Given class docs only, write out their docs and methods.""" 146 | for class_code in classes: 147 | class_code = " class " + class_code # got cut-off in split 148 | class_parts = class_code.split('"""\n')[:-1] 149 | # process all the class information 150 | class_doc = class_parts.pop(0) # class docstring, rest is methods 151 | self.markdown.append( 152 | self.md_class_doc(self.structure_doc(class_doc))) 153 | # replace object part with __init__ params 154 | self.markdown[1] = \ 155 | self.md_class_method(self.markdown[1], class_parts) 156 | # start handling methods 157 | self.markdown.append( 158 | self.md_methods_doc([self.structure_doc(method) 159 | for method in class_parts])) 160 | 161 | def find_declaration(self, lines): 162 | """Find `class Bla(object):` or `def method(params):` across lines.""" 163 | declaration, found_start = '', False 164 | # skip lines until class/def, end if : 165 | while ':' not in declaration: 166 | if lines[0].startswith(' class ') or \ 167 | lines[0].startswith(' def '): 168 | found_start = True 169 | line = lines.pop(0) 170 | if found_start: 171 | declaration += line 172 | return declaration 173 | 174 | def structure_doc(self, doc): 175 | """Produce a structured dictionary from docstring. 176 | 177 | The key is the part of the documentation (name, title, params, 178 | attributes, etc.), and value is a list of lines that make up this part. 179 | Is used to handle the order and different parses of different sections. 180 | 181 | Parameters 182 | ---------- 183 | doc : str 184 | Flat string structure of a docstring block. 185 | 186 | Returns 187 | ------- 188 | X_doc : dict 189 | Class or method doc, a structure dict with section; lines. 190 | """ 191 | lines = [x for x in doc.split('\n') if x] 192 | parts = {'name': self.find_declaration(lines)} 193 | buffer_to = 'title' 194 | 195 | # if ----- is found, set title to above, else buffer to name 196 | for i in range(0, len(lines)): 197 | line = lines[i] 198 | if '---' in line: # get tile 199 | buffer_to = lines[i-1] 200 | continue 201 | if not parts.get(buffer_to): # buffer to 'title' by default 202 | parts[buffer_to] = [buffer_to] 203 | if parts.get(buffer_to) and line: 204 | parts[buffer_to].append(line) 205 | 206 | # if line is shorter than 3, likely that there is no previous section 207 | return {k.replace(' ', ''): 208 | (v[:-1] if len(v) > 2 else v)[1:] for k, v in parts.items()} 209 | 210 | def md_title(self, title_part, class_line=False): 211 | """Handle title and text parts for main and class-specific. 212 | 213 | Pretty convoluted but delicate method that seperates the top part 214 | that should not be part of the title docstring off the document, 215 | handles correct punctuation of titles, for both classes and methods. 216 | Also detects if there's a description under the first line. 217 | 218 | Parameters 219 | ---------- 220 | title_part : str 221 | Part of the document that starts with triple quotes and is below 222 | any line seperators. 223 | 224 | class_line : bool, optional, default False 225 | The first line of the docstring by default servers as a title 226 | header. However, for classes it should be formed into a normal 227 | description. Thefefore, the # marker is removed and a . is added. 228 | 229 | Returns 230 | ------- 231 | line_buffer : str 232 | Correctly formated markdown from the buffer. 233 | """ 234 | line_buffer, start_buffer, description = [], False, False 235 | for line in title_part.split('\n'): 236 | line = line.replace(' ', '') 237 | if line.count('"""') == 2: # one liner 238 | return '# ' + line.replace('"""', '')[:-1] # title 239 | elif line.startswith('"""') and not start_buffer: 240 | title = '# ' + line.replace('"""', '')[:-1] # title 241 | if class_line: 242 | title = title[2:] + '.' # normal description 243 | line_buffer.append(title) 244 | start_buffer = True 245 | elif line.startswith('"""') and start_buffer: # handle top of file 246 | return '\n'.join(line_buffer) 247 | elif start_buffer: 248 | if not description and not line.startswith('\n'): 249 | line = "\n" + line 250 | description = True 251 | line_buffer.append(line) 252 | 253 | return '\n'.join(line_buffer) 254 | 255 | def md_table(self, doc, name): 256 | r"""Place parameters and attributes in a table overview. 257 | 258 | The documentation parts that are typed by 'var : type \n description' 259 | can be splitted into a table. The same holds true for a list of class 260 | methods. These are handled by this method. Table structures are on 261 | the top of this Python file. 262 | 263 | Parameters 264 | ---------- 265 | doc : str 266 | Flat string structure of a docstring block. 267 | 268 | name : str 269 | Indicator for the table, can be for example Parameters or 270 | Attributes. 271 | 272 | Returns 273 | ------- 274 | table : str 275 | Markdown-type table with filled rows and formatted name. 276 | """ 277 | var_type, line_buffer, lines = '', (), [] 278 | table = MD_TABLE.format(name) 279 | 280 | # given var : type \n description, splits these up into 3 cellss 281 | if not doc: 282 | return '' 283 | 284 | for row in doc: 285 | if ':' in row and line_buffer: # if we hit the next, store table 286 | line_buffer += (''.join(lines), ) 287 | table += MD_TABLE_ROW.format(*line_buffer) 288 | line_buffer = () 289 | lines = [] 290 | if ':' in row and not line_buffer: # it's var : type 291 | var_type = row.split(' : ') 292 | for part in var_type: 293 | line_buffer += (part, ) 294 | elif line_buffer: # if no next, keep appending rows 295 | lines.append(row.replace('\n', ' ')) 296 | 297 | if line_buffer: # empty the buffer 298 | line_buffer += (''.join(lines), ) 299 | table += MD_TABLE_ROW.format(*line_buffer) 300 | 301 | return table 302 | 303 | def md_code_text(self, doc, name, flat=False): 304 | """Section text and/or code in the example part of the doc. 305 | 306 | If some code related beginnings (>>>, ...) is founds buffer to code, 307 | else we regard it as text. Record order so that multiple consecutive 308 | blocks of text and code are possible. 309 | 310 | Parameters 311 | ---------- 312 | doc : str 313 | Flat string structure of a docstring block. 314 | name : str 315 | Name of the section (Examples, Notes). 316 | flat : bool, optional, default False 317 | If there are no code blocks, flat can be used to stop parsing them. 318 | 319 | Returns 320 | ------- 321 | code : str 322 | Formatted block of fenced markdown code-snippets and flat text. 323 | 324 | Notes 325 | ----- 326 | Please note that this method assumes markdown is uses so-called 327 | fenced codeblocks that are commonly accepted by Readthedocs, GitHub, 328 | Bitbucket, and Jekyll (for example with Redcarpet). 329 | 330 | This current implementation does NOT allow for IPython styled code 331 | examples. (In [1]: Out[1]: etc.) 332 | """ 333 | head = '\n\n------- \n\n##{1}\n\n{0}' 334 | order, text, code = [], '', '' 335 | 336 | if not doc: 337 | return '' 338 | 339 | # if code marker, store to code, else store to text 340 | for row in doc: 341 | if not flat and ('>>>' in row or '...' in row): 342 | if text: 343 | order.append(text) 344 | text = '' 345 | row = row.replace(' >>>', '>>>') 346 | row = row.replace(' ...', '...') 347 | code += row + '\n' 348 | else: 349 | if code: 350 | # if we found code, encapsulate in ``` python ``` blocks. 351 | order.append(MD_CODE.format(code)) 352 | code = '' 353 | row = row.replace(' ', '') 354 | text += row + '\n' + ('\n' if any([row.endswith(x) for x 355 | in ('!', '?', '.', ':')]) else '') 356 | 357 | if text: # empty the buffer 358 | order.append(text.replace('\\', '')) 359 | if code: 360 | order.append(MD_CODE.format(code)) 361 | 362 | return head.format('\n'.join(order).replace('.\n', '.\n\n'), name) 363 | 364 | def md_class_method(self, doc, class_parts): 365 | """Replace `class ...(object)` with `__init__` arguments. 366 | 367 | In NumPy-style, each class is represented as it's name, along with 368 | the parameters accepted in `__init__` as its parameters, as you would 369 | call the method in Python. The markdown therefore needs to fill the 370 | (object) tag that is assigned to classes with the `__init__` params. 371 | 372 | Parameters 373 | ---------- 374 | doc : str 375 | Flat string structure of a docstring block. 376 | 377 | class_parts : list 378 | List with parts that the docstring constituted of. First line 379 | should be the class name and can therefore be replaced. 380 | 381 | Returns 382 | ------- 383 | doc : str 384 | Same as doc, only now (object) has been replaced with `__init__` 385 | parameters. 386 | 387 | Notes 388 | ----- 389 | Please note that this implementation assumes you're using new-style 390 | class declarations where for example `class SomeClass:` should be 391 | written as `class SomeClass(object):`. 392 | """ 393 | # FIXME: does not seem to work for consecutive classes! 394 | 395 | init_doc = self.structure_doc(class_parts.pop(0))['name'] 396 | init_doc = init_doc.replace(' ', '') 397 | init_doc = init_doc.replace(' def __init__(self, ', '(') 398 | 399 | # FIXME: (object) screws up other class inherrits, make general regex 400 | return doc.replace('(object)', init_doc) 401 | 402 | def md_methods_doc(self, method_doc): 403 | """Merge all method elements in one string. 404 | 405 | This method will scan each method docstring and extract parameters, 406 | returns and descriptions. It will append them to the method table 407 | and link (TODO) them in the doc. 408 | 409 | Parameters 410 | ---------- 411 | method_doc : list 412 | Lower part of the class docstrings that holds a method docstring 413 | per entry. 414 | 415 | Returns 416 | ------- 417 | methods : str 418 | Formatted markdown string with all method information. 419 | 420 | Notes 421 | ----- 422 | On the bottom, dict calls are being done to order several parts of 423 | the docstring. If you use more than Parameters and Returns, please make 424 | sure they are added in the code there. 425 | """ 426 | mark_doc, mark_head = '', '\n--------- \n\n## Methods \n\n {0} \n {1}' 427 | method_table = MD_TABLE_ALT.format('method') 428 | 429 | for method in method_doc: 430 | # isolate only the name of the function (without def and params) 431 | name = method['name'].replace('(', ' (').split()[1] 432 | title = self.md_title('\n'.join(method['title']), class_line=True) 433 | 434 | # name and first line of description are added to method_table 435 | method_table += MD_TABLE_ROW_ALT.format(name, title.split('\n')[0]) 436 | mark_doc += '\n\n### ' + name + '\n\n' 437 | 438 | # show the method call with parameters 439 | code = MD_CODE.format(method['name'].replace('def ', '')) 440 | code = code.replace('self', '').replace('(, ', '(') 441 | 442 | mark_doc += code 443 | mark_doc += '\n' + title 444 | 445 | # add to sections here if more blocks are possible! 446 | mark_doc += self.md_table(method.get('Parameters'), 'Parameters') 447 | mark_doc += self.md_table(method.get('Returns'), 'Returns') 448 | mark_doc += self.md_code_text(method.get('Notes'), '## Notes') 449 | 450 | return mark_head.format(method_table, mark_doc) 451 | 452 | def md_class_doc(self, class_doc): 453 | """Merge all class section elements in one string. 454 | 455 | This will piece together the descriptions contained in the docstring 456 | of the class. Currently, they are written *below* the top of the 457 | Python file, and not put into any subfiles per class. 458 | 459 | Parameters 460 | ---------- 461 | class_doc : str 462 | Flat text containing the class docstring part. 463 | 464 | Returns 465 | ------- 466 | mark_doc : str 467 | Markdown formatted class docstring, including all subsections. 468 | 469 | Notes 470 | ----- 471 | On the bottom, dict calls are being done to order several parts of 472 | the docstring. If you use more than Parameters, Attributes, Examples, 473 | and Notes, please make sure they are added in the code there. 474 | """ 475 | mark_doc = '' 476 | 477 | # class name is being used as title 478 | mark_doc += '\n\n# {0} \n\n'.format( 479 | class_doc['name'].replace('(', ' (').split()[1]) 480 | mark_doc += MD_CODE.format(class_doc['name']) 481 | 482 | # frist docstring line for a class is just written as a sentence 483 | mark_doc += self.md_title('\n'.join(class_doc['title']), 484 | class_line=True) 485 | 486 | # add to sections here if more blocks are possible! 487 | mark_doc += self.md_table(class_doc.get('Parameters'), 'Parameters') 488 | mark_doc += self.md_table(class_doc.get('Attributes'), 'Attributes') 489 | mark_doc += self.md_code_text(class_doc.get('Examples'), 'Examples') 490 | mark_doc += self.md_code_text(class_doc.get('Notes'), 'Notes', 491 | flat=True) 492 | 493 | return mark_doc 494 | 495 | if __name__ == '__main__': 496 | try: 497 | D2M = MarkDoc(argv[1], argv[2]) 498 | print("Succesfully converted!") 499 | except Exception as err: 500 | exit("-"*10 + "\nERROR! Malformed code \n\nTraceback: \n\n{1} \n\n\n\ 501 | Some common errors: \n{0}".format(ERR, err)) 502 | --------------------------------------------------------------------------------