├── images
├── infoMacro.png
├── noteMacro.png
└── warningMacro.png
├── .gitignore
├── requirements.txt
├── markdownlint.json
├── LICENSE
├── README.md
├── archive
└── md_to_conf.py
├── .pylintrc
└── md2conf.py
/images/infoMacro.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/RittmanMead/md_to_conf/HEAD/images/infoMacro.png
--------------------------------------------------------------------------------
/images/noteMacro.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/RittmanMead/md_to_conf/HEAD/images/noteMacro.png
--------------------------------------------------------------------------------
/images/warningMacro.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/RittmanMead/md_to_conf/HEAD/images/warningMacro.png
--------------------------------------------------------------------------------
/.gitignore:
--------------------------------------------------------------------------------
1 | .idea
2 | venv
3 | *.vsidx
4 | *.lock
5 | .vs/md_to_conf/v17/.wsuo
6 | .vs/
7 | .vs/launch
8 | env/
9 | .vscode/
10 |
--------------------------------------------------------------------------------
/requirements.txt:
--------------------------------------------------------------------------------
1 | certifi==2023.7.22
2 | chardet==5.0.0
3 | idna==3.3
4 | Markdown==3.4.1
5 | requests==2.31.0
6 | urllib3==1.26.18
7 |
--------------------------------------------------------------------------------
/markdownlint.json:
--------------------------------------------------------------------------------
1 | {
2 | "default": true,
3 | "MD007": { "indent": 4 },
4 | "MD033": { "allowed_elements": ["br"] },
5 | "MD013": false,
6 | "MD024": false,
7 | "MD046": { "style": "fenced" }
8 | }
9 |
--------------------------------------------------------------------------------
/LICENSE:
--------------------------------------------------------------------------------
1 | MIT License
2 |
3 | Copyright (c) 2018 Rittman Mead
4 |
5 | Permission is hereby granted, free of charge, to any person obtaining a copy
6 | of this software and associated documentation files (the "Software"), to deal
7 | in the Software without restriction, including without limitation the rights
8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9 | copies of the Software, and to permit persons to whom the Software is
10 | furnished to do so, subject to the following conditions:
11 |
12 | The above copyright notice and this permission notice shall be included in all
13 | copies or substantial portions of the Software.
14 |
15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21 | SOFTWARE.
22 |
--------------------------------------------------------------------------------
/README.md:
--------------------------------------------------------------------------------
1 | # Markdown to Confluence Converter
2 |
3 | A script to import a named markdown document into Confluence.
4 |
5 | It handles inline images as well as code blocks.
6 |
7 | Also there is support for some custom markdown tags for use with commonly used Confluence macros.
8 |
9 | The file will be converted into HTML or Confluence storage markup when required.
10 |
11 | Then a page will be created in the space or if it already exists, the page will be uploaded.
12 |
13 | ## Configuration
14 |
15 | [Download](https://github.com/rittmanmead/md_to_conf)
16 |
17 | ## Requirements
18 |
19 | Python 3.6+
20 |
21 | ### Python venv
22 |
23 | The project code and dependencies can be used based on python virtualenv.
24 |
25 | Create a new python virtualenv:
26 |
27 | ```less
28 | > python3 -m venv venv
29 | ```
30 |
31 | Or in **Anaconda**
32 |
33 | conda create --name md_to_conf python=3.7 --yes
34 |
35 | Make the virtualenv active:
36 |
37 | ```less
38 | > source venv/bin/activate
39 | ```
40 |
41 | ### Dependencies
42 |
43 | Required python dependencies can be installed using:
44 |
45 | ```less
46 | pip3 install -r requirements.txt
47 | ```
48 |
49 | ### Environment Variables
50 |
51 | To use it, you will need your Confluence username, API key and organisation name.
52 | To generate an API key go to [https://id.atlassian.com/manage/api-tokens](https://id.atlassian.com/manage/api-tokens).
53 |
54 | You will also need the organization name that is used in the subdomain.
55 | For example the URL: `https://fawltytowers.atlassian.net/wiki/` would indicate an organization name of **fawltytowers**.
56 |
57 | If the organization name contains a dot, it will be considered as a Fully Qualified Domain Name.
58 | For example the URL: `https://fawltytowers.mydomain.com/` would indicate an organization name of **fawltytowers.mydomain.com**.
59 |
60 | These can be specified at runtime or set as Confluence environment variables
61 | (e.g. add to your `~/.profile` or `~/.bash_profile` on Mac OS):
62 |
63 | ``` bash
64 | export CONFLUENCE_USERNAME='basil'
65 | export CONFLUENCE_API_KEY='abc123'
66 | export CONFLUENCE_ORGNAME='fawltytowers'
67 | ```
68 |
69 | On Windows, this can be set via system properties.
70 |
71 | ## Use
72 |
73 | ### Basic
74 |
75 | The minimum accepted parameters are the markdown file to upload as well as the Confluence space key you wish to upload to. For the following examples assume 'Test Space' with key: `TST`.
76 |
77 | ```less
78 | python3 md2conf.py readme.md TST
79 | ```
80 |
81 | Mandatory Confluence parameters can also be set here if not already set as environment variables:
82 |
83 | * **-u** **--username**: Confluence User
84 | * **-p** **--apikey**: Confluence API Key
85 | * **-o** **--orgname**: Confluence Organisation
86 |
87 | ```less
88 | python3 md2conf.py readme.md TST -u basil -p abc123 -o fawltytowers
89 | ```
90 |
91 | Use **-h** to view a list of all available options.
92 |
93 | ### Other Uses
94 |
95 | Use **-a** or **--ancestor** to designate the name of a page which the page should be created under.
96 |
97 | ```less
98 | python md2conf.py readme.md TST -a "Parent Page Name"
99 | ```
100 |
101 | Use **-d** or **--delete** to delete the page instead of create it. Obviously this won't work if it doesn't already exist.
102 |
103 | Use **-n** or **--nossl** to specify a non-SSL url, i.e. **** instead of ****.
104 |
105 | Use **-l** or **--loglevel** to specify a different logging level, i.e **DEBUG**.
106 |
107 | Use **-s** or **--simulate** to stop processing before interacting with confluence API, i.e. only
108 | converting the markdown document to confluence format.
109 |
110 | Use **--title** to set the title for the page, otherwise the title is going to be the first line in the markdown file
111 |
112 | Use **--remove-emojies** to emove emojies if there are any. This may be need if the database doesn't support emojies
113 |
114 | ## Markdown
115 |
116 | The original markdown to HTML conversion is performed by the Python **markdown** library.
117 | Additionally, the page name is taken from the first line of the markdown file, usually assumed to be the title.
118 | In the case of this document, the page would be called: **Markdown to Confluence Converter**.
119 |
120 | Standard markdown syntax for images and code blocks will be automatically converted.
121 | The images are uploaded as attachments and the references updated in the HTML.
122 | The code blocks will be converted to the Confluence Code Block macro and also supports syntax highlighting.
123 |
124 | ### Doctoc
125 |
126 | If present, what is between the [doctoc](https://github.com/thlorenz/doctoc) anchor format:
127 |
128 | ```less
129 |
132 | ```
133 |
134 | will be replaced by confluence "toc" macro leading to something like:
135 |
136 | ```html
137 | Table of Content
138 |
139 |
140 | true
141 | disc
142 | 7
143 | 1
144 | list
145 | clear
146 | .*
147 |
148 |
149 | ```
150 |
151 | ### Information, Note and Warning Macros
152 |
153 | > **Warning:** Any blockquotes used will implement an information macro. This could potentially harm your formatting.
154 |
155 | Block quotes in Markdown are rendered as information macros.
156 |
157 | ```less
158 | > This is an info
159 | ```
160 |
161 | 
162 |
163 | ```less
164 | > Note: This is a note
165 | ```
166 |
167 | 
168 |
169 | ```less
170 | > Warning: This is a warning
171 | ```
172 |
173 | 
174 |
175 | Alternatively, using a custom Markdown syntax also works:
176 |
177 | ```less
178 | ~?This is an info.?~
179 |
180 | ~!This is a note.!~
181 |
182 | ~%This is a warning.%~
183 | ```
184 |
185 | ## Miscellaneous
186 |
187 | ```less
188 | ╚⊙ ⊙╝
189 | ╚═(███)═╝
190 | ╚═(███)═╝
191 | ╚═(███)═╝
192 | ╚═(███)═╝
193 | ╚═(███)═╝
194 | ╚═(███)═╝
195 | ```
196 |
--------------------------------------------------------------------------------
/archive/md_to_conf.py:
--------------------------------------------------------------------------------
1 | #!/usr/bin/python
2 | #
3 | # [ DEPRECATED ]
4 | # Use md2conf.py which has been written using the REST API instead of XMLRPC
5 | #
6 | #
7 | # @rmoff 20140330
8 | #
9 | # "A Hack But It Works"
10 | #----------------------------------------
11 | #
12 | # This will import specified Markdown file into Confluence
13 | #
14 | # It uses [depreciated] XMLRPC API for Confluence. Supports inline images and code blocks.
15 | #
16 | #-----------------------------------------
17 | #
18 | # TODO:
19 | # Put some proper structure in, not a dirty mess of indented if statements.
20 | # Convert to use REST API?
21 | #
22 | #
23 | import sys,os,markdown,mimetypes,codecs,re
24 | from xmlrpclib import Server,Binary
25 |
26 | def convertCodeBlock(html):
27 | codeBlocks = re.findall('.*?<\/code><\/pre>', html, re.DOTALL)
28 | if codeBlocks:
29 | for tag in codeBlocks:
30 |
31 | confML = ''
32 | confML = confML + 'Midnight'
33 | confML = confML + 'true'
34 |
35 | lang = re.search('code class="(.*)"', tag)
36 | if lang:
37 | lang = lang.group(1)
38 | else:
39 | lang = 'none'
40 |
41 | confML = confML + '' + lang + ''
42 | content = re.search('(.*?)<\/code><\/pre>', tag, re.DOTALL).group(1)
43 | content = ''
44 | content = content.replace('<', '<').replace('>', '>')
45 | content = content.replace('"', '"').replace('&', '&')
46 | confML = confML + content + '
'
47 |
48 | html = html.replace(tag, confML)
49 |
50 | return html
51 |
52 | # username=os.getenv('CONFLUENCE_USERNAME', 'UNSET')
53 | # password=os.getenv('CONFLUENCE_PASSWORD', 'UNSET')
54 | # orgname=os.getenv('CONFLUENCE_ORGNAME', 'UNSET')
55 |
56 | username='minesh.patel'
57 | password='Password01'
58 | orgname='rittmanmead'
59 |
60 | if (username=='UNSET' or password=='UNSET'):
61 | print '\nConfluence username/password not found.\n\n\t==> Please set CONFLUENCE_USERNAME and CONFLUENCE_PASSWORD environment variables and try again.'
62 | sys.exit(2)
63 | if orgname=='UNSET':
64 | print '\nConfluence orgname not set. (https://xxxx.atlassian.net/wiki/)\n\n\t==> Please set CONFLUENCE_ORGNAME environment variable and try again.'
65 | sys.exit(2)
66 | if len(sys.argv)<2:
67 | print '\n\t\n\tError: Filename missing. Program aborts. Specify the full path of the file to import as the first commandline argument.\n\n'
68 | else:
69 | markdown_file_to_import=sys.argv[1]
70 | if os.path.exists(markdown_file_to_import):
71 | if len(sys.argv)>2:
72 | spacekey=sys.argv[2]
73 | else:
74 | spacekey='~%s' % (username)
75 | headers = {'Accept':'application/json'}
76 | s=Server("https://%s.atlassian.net/wiki/rpc/xmlrpc"%(orgname))
77 | conf_token = s.confluence2.login(username,password)
78 |
79 | try:
80 | space=s.confluence2.getSpace(conf_token,spacekey)
81 | print '\n\tAtlas Space: %s' % space['name']
82 | except:
83 | print 'Space %s not found\n\tEither create a Personal Space, or specify a spacekey (eg INF) as the second commandline argument.' % (spacekey)
84 | sys.exit(1)
85 |
86 | source_folder=os.path.dirname(markdown_file_to_import)
87 |
88 | # markdown_basename=os.path.basename(markdown_file_to_import)
89 | with open(markdown_file_to_import, 'r') as f:
90 | markdown_basename = f.readline().strip()
91 |
92 | # Necessary to handle unicode
93 | with codecs.open(markdown_file_to_import,'r','utf-8') as f:
94 | html=markdown.markdown(f.read(), extensions = ['markdown.extensions.tables', 'markdown.extensions.fenced_code'])
95 |
96 | html = '\n'.join(html.split('\n')[1:])
97 |
98 | # Custom Info, Note and Warning tags
99 | html=html.replace('~?','
').replace('?~
', '
')
100 | html=html.replace('~!','
').replace('!~
', '
')
101 | html=html.replace('~%','
').replace('%~
', '
')
102 |
103 | html = convertCodeBlock(html)
104 |
105 | try:
106 | # Check if page exists and build one if it doesn't
107 | try:
108 | conf_page_data = s.confluence2.getPage(conf_token, spacekey, markdown_basename)
109 | action = 'updated'
110 | except:
111 | conf_page_data = {}
112 | conf_page_data['space'] = spacekey
113 | conf_page_data['title'] = markdown_basename
114 | action = 'created'
115 |
116 | conf_page_data['content'] = html
117 |
118 | conf_page = s.confluence2.storePage(conf_token, conf_page_data)
119 | print '\nAtlas page %s:\n\t%s\n\thttps://%s.atlassian.net/wiki/pages/viewpage.action?pageId=%s' % (action, conf_page['title'],orgname,conf_page['id'])
120 |
121 | # Process images
122 | if (conf_page_data['content'].find('0):
123 | for img in conf_page_data['content'].split('?$
128 |
129 | # Allow the body of an if to be on the same line as the test if there is no
130 | # else.
131 | single-line-if-stmt=no
132 |
133 | # List of optional constructs for which whitespace checking is disabled. `dict-
134 | # separator` is used to allow tabulation in dicts, etc.: {1 : 1,\n222: 2}.
135 | # `trailing-comma` allows a space between comma and closing bracket: (a, ).
136 | # `empty-line` allows space-only lines.
137 | no-space-check=trailing-comma,dict-separator
138 |
139 | # Maximum number of lines in a module
140 | max-module-lines=1000
141 |
142 | # String used as indentation unit. This is usually " " (4 spaces) or "\t" (1
143 | # tab).
144 | indent-string=' '
145 |
146 | # Number of spaces of indent required inside a hanging or continued line.
147 | indent-after-paren=4
148 |
149 | # Expected format of line ending, e.g. empty (any line ending), LF or CRLF.
150 | expected-line-ending-format=
151 |
152 |
153 | [SPELLING]
154 |
155 | # Spelling dictionary name. Available dictionaries: none. To make it working
156 | # install python-enchant package.
157 | spelling-dict=
158 |
159 | # List of comma separated words that should not be checked.
160 | spelling-ignore-words=
161 |
162 | # A path to a file that contains private dictionary; one word per line.
163 | spelling-private-dict-file=
164 |
165 | # Tells whether to store unknown words to indicated private dictionary in
166 | # --spelling-private-dict-file option instead of raising a message.
167 | spelling-store-unknown-words=no
168 |
169 |
170 | [VARIABLES]
171 |
172 | # Tells whether we should check for unused import in __init__ files.
173 | init-import=no
174 |
175 | # A regular expression matching the name of dummy variables (i.e. expectedly
176 | # not used).
177 | dummy-variables-rgx=(_+[a-zA-Z0-9]*?$)|dummy
178 |
179 | # List of additional names supposed to be defined in builtins. Remember that
180 | # you should avoid to define new builtins when possible.
181 | additional-builtins=
182 |
183 | # List of strings which can identify a callback function by name. A callback
184 | # name must start or end with one of those strings.
185 | callbacks=cb_,_cb
186 |
187 | # List of qualified module names which can have objects that can redefine
188 | # builtins.
189 | redefining-builtins-modules=six.moves,future.builtins
190 |
191 |
192 | [TYPECHECK]
193 |
194 | # Tells whether missing members accessed in mixin class should be ignored. A
195 | # mixin class is detected if its name ends with "mixin" (case insensitive).
196 | ignore-mixin-members=yes
197 |
198 | # List of module names for which member attributes should not be checked
199 | # (useful for modules/projects where namespaces are manipulated during runtime
200 | # and thus existing member attributes cannot be deduced by static analysis. It
201 | # supports qualified module names, as well as Unix pattern matching.
202 | ignored-modules=flask_sqlalchemy,app.extensions.flask_sqlalchemy
203 |
204 | # List of class names for which member attributes should not be checked (useful
205 | # for classes with dynamically set attributes). This supports the use of
206 | # qualified names.
207 | ignored-classes=optparse.Values,thread._local,_thread._local
208 |
209 | # List of members which are set dynamically and missed by pylint inference
210 | # system, and so shouldn't trigger E1101 when accessed. Python regular
211 | # expressions are accepted.
212 | generated-members=fget,query,begin,add,merge,delete,commit,rollback
213 |
214 | # List of decorators that produce context managers, such as
215 | # contextlib.contextmanager. Add to this list to register other decorators that
216 | # produce valid context managers.
217 | contextmanager-decorators=contextlib.contextmanager
218 |
219 |
220 | [MISCELLANEOUS]
221 |
222 | # List of note tags to take in consideration, separated by a comma.
223 | notes=FIXME,XXX,TODO
224 |
225 |
226 | [BASIC]
227 |
228 | # Good variable names which should always be accepted, separated by a comma
229 | good-names=i,j,k,ex,Run,_,log,api
230 |
231 | # Bad variable names which should always be refused, separated by a comma
232 | bad-names=foo,bar,baz,toto,tutu,tata
233 |
234 | # Colon-delimited sets of names that determine each other's naming style when
235 | # the name regexes allow several styles.
236 | name-group=
237 |
238 | # Include a hint for the correct naming format with invalid-name
239 | include-naming-hint=no
240 |
241 | # List of decorators that produce properties, such as abc.abstractproperty. Add
242 | # to this list to register other decorators that produce valid properties.
243 | property-classes=abc.abstractproperty
244 |
245 | # Regular expression matching correct class names
246 | class-rgx=[A-Z_][a-zA-Z0-9]+$
247 |
248 | # Naming hint for class names
249 | class-name-hint=[A-Z_][a-zA-Z0-9]+$
250 |
251 | # Regular expression matching correct constant names
252 | const-rgx=(([A-Z_][A-Z0-9_]*)|(__.*__))$
253 |
254 | # Naming hint for constant names
255 | const-name-hint=(([A-Z_][A-Z0-9_]*)|(__.*__))$
256 |
257 | # Regular expression matching correct argument names
258 | argument-rgx=[a-z_][a-z0-9_]{2,40}$
259 |
260 | # Naming hint for argument names
261 | argument-name-hint=[a-z_][a-z0-9_]{2,40}$
262 |
263 | # Regular expression matching correct inline iteration names
264 | inlinevar-rgx=[A-Za-z_][A-Za-z0-9_]*$
265 |
266 | # Naming hint for inline iteration names
267 | inlinevar-name-hint=[A-Za-z_][A-Za-z0-9_]*$
268 |
269 | # Regular expression matching correct method names
270 | method-rgx=(([a-z_][a-z0-9_]{2,40})|(setUp)|(tearDown))$
271 |
272 | # Naming hint for method names
273 | method-name-hint=[a-z_][a-z0-9_]{2,40}$
274 |
275 | # Regular expression matching correct function names
276 | function-rgx=[a-z_][a-z0-9_]{2,40}$
277 |
278 | # Naming hint for function names
279 | function-name-hint=[a-z_][a-z0-9_]{2,40}$
280 |
281 | # Regular expression matching correct class attribute names
282 | class-attribute-rgx=([A-Za-z_][A-Za-z0-9_]{2,40}|(__.*__))$
283 |
284 | # Naming hint for class attribute names
285 | class-attribute-name-hint=([A-Za-z_][A-Za-z0-9_]{2,40}|(__.*__))$
286 |
287 | # Regular expression matching correct attribute names
288 | attr-rgx=[a-z_][a-z0-9_]{2,40}$
289 |
290 | # Naming hint for attribute names
291 | attr-name-hint=[a-z_][a-z0-9_]{2,40}$
292 |
293 | # Regular expression matching correct variable names
294 | variable-rgx=[a-z_][a-z0-9_]{2,40}$
295 |
296 | # Naming hint for variable names
297 | variable-name-hint=[a-z_][a-z0-9_]{2,40}$
298 |
299 | # Regular expression matching correct module names
300 | module-rgx=(([a-z_][a-z0-9_]*)|([A-Z][a-zA-Z0-9]+))$
301 |
302 | # Naming hint for module names
303 | module-name-hint=(([a-z_][a-z0-9_]*)|([A-Z][a-zA-Z0-9]+))$
304 |
305 | # Regular expression which should only match function or class names that do
306 | # not require a docstring.
307 | no-docstring-rgx=^_
308 |
309 | # Minimum line length for functions/classes that require docstrings, shorter
310 | # ones are exempt.
311 | docstring-min-length=5
312 |
313 |
314 | [ELIF]
315 |
316 | # Maximum number of nested blocks for function / method body
317 | max-nested-blocks=5
318 |
319 |
320 | [DESIGN]
321 |
322 | # Maximum number of arguments for function / method
323 | max-args=5
324 |
325 | # Argument names that match this expression will be ignored. Default to name
326 | # with leading underscore
327 | ignored-argument-names=_.*
328 |
329 | # Maximum number of locals for function / method body
330 | max-locals=15
331 |
332 | # Maximum number of return / yield for function / method body
333 | max-returns=6
334 |
335 | # Maximum number of branch for function / method body
336 | max-branches=12
337 |
338 | # Maximum number of statements in function / method body
339 | max-statements=50
340 |
341 | # Maximum number of parents for a class (see R0901).
342 | max-parents=10
343 |
344 | # Maximum number of attributes for a class (see R0902).
345 | max-attributes=7
346 |
347 | # Minimum number of public methods for a class (see R0903).
348 | min-public-methods=2
349 |
350 | # Maximum number of public methods for a class (see R0904).
351 | max-public-methods=20
352 |
353 | # Maximum number of boolean expressions in a if statement
354 | max-bool-expr=5
355 |
356 |
357 | [IMPORTS]
358 |
359 | # Deprecated modules which should not be used, separated by a comma
360 | deprecated-modules=optparse
361 |
362 | # Create a graph of every (i.e. internal and external) dependencies in the
363 | # given file (report RP0402 must not be disabled)
364 | import-graph=
365 |
366 | # Create a graph of external dependencies in the given file (report RP0402 must
367 | # not be disabled)
368 | ext-import-graph=
369 |
370 | # Create a graph of internal dependencies in the given file (report RP0402 must
371 | # not be disabled)
372 | int-import-graph=
373 |
374 | # Force import order to recognize a module as part of the standard
375 | # compatibility libraries.
376 | known-standard-library=
377 |
378 | # Force import order to recognize a module as part of a third party library.
379 | known-third-party=flask_restplus_patched
380 |
381 | # Analyse import fallback blocks. This can be used to support both Python 2 and
382 | # 3 compatible code, which means that the block might have code that exists
383 | # only in one or another interpreter, leading to false positives when analysed.
384 | analyse-fallback-blocks=no
385 |
386 |
387 | [CLASSES]
388 |
389 | # List of method names used to declare (i.e. assign) instance attributes.
390 | defining-attr-methods=__init__,__new__,setUp
391 |
392 | # List of valid names for the first argument in a class method.
393 | valid-classmethod-first-arg=cls
394 |
395 | # List of valid names for the first argument in a metaclass class method.
396 | valid-metaclass-classmethod-first-arg=mcs
397 |
398 | # List of member names, which should be excluded from the protected access
399 | # warning.
400 | exclude-protected=_asdict,_fields,_replace,_source,_make
401 |
402 |
403 | [EXCEPTIONS]
404 |
405 | # Exceptions that will emit a warning when being caught. Defaults to
406 | # "Exception"
407 | overgeneral-exceptions=Exception
408 |
409 |
--------------------------------------------------------------------------------
/md2conf.py:
--------------------------------------------------------------------------------
1 | #!/usr/bin/env python3
2 | """
3 | # --------------------------------------------------------------------------------------------------
4 | # Rittman Mead Markdown to Confluence Tool
5 | # --------------------------------------------------------------------------------------------------
6 | # Create or Update Atlas pages remotely using markdown files.
7 | #
8 | # --------------------------------------------------------------------------------------------------
9 | # Usage: rest_md2conf.py markdown spacekey
10 | # --------------------------------------------------------------------------------------------------
11 | """
12 |
13 | import logging
14 | import sys
15 | import os
16 | import re
17 | import json
18 | import collections
19 | import mimetypes
20 | import codecs
21 | import argparse
22 | import urllib
23 | import webbrowser
24 | import requests
25 | import markdown
26 |
27 | logging.basicConfig(level=logging.INFO, format='%(asctime)s - \
28 | %(levelname)s - %(funcName)s [%(lineno)d] - \
29 | %(message)s')
30 | LOGGER = logging.getLogger(__name__)
31 |
32 | # ArgumentParser to parse arguments and options
33 | PARSER = argparse.ArgumentParser()
34 | PARSER.add_argument("markdownFile", help="Full path of the markdown file to convert and upload.")
35 | PARSER.add_argument('spacekey',
36 | help="Confluence Space key for the page. If omitted, will use user space.")
37 | PARSER.add_argument('-u', '--username', help='Confluence username if $CONFLUENCE_USERNAME not set.')
38 | PARSER.add_argument('-p', '--apikey', help='Confluence API key if $CONFLUENCE_API_KEY not set.')
39 | PARSER.add_argument('--pat', help='Confluence Personal Access Token if $CONFLUENCE_PERSONAL_ACCESS_TOKEN not set.')
40 | PARSER.add_argument('-o', '--orgname',
41 | help='Confluence organisation if $CONFLUENCE_ORGNAME not set. '
42 | 'e.g. https://XXX.atlassian.net/wiki'
43 | 'If orgname contains a dot, considered as the fully qualified domain name.'
44 | 'e.g. https://XXX')
45 | PARSER.add_argument('-a', '--ancestor',
46 | help='Parent page under which page will be created or moved.')
47 | PARSER.add_argument('-t', '--attachment', nargs='+',
48 | help='Attachment(s) to upload to page. Paths relative to the markdown file.')
49 | PARSER.add_argument('-c', '--contents', action='store_true', default=False,
50 | help='Use this option to generate a contents page.')
51 | PARSER.add_argument('-g', '--nogo', action='store_true', default=False,
52 | help='Use this option to skip navigation after upload.')
53 | PARSER.add_argument('-n', '--nossl', action='store_true', default=False,
54 | help='Use this option if NOT using SSL. Will use HTTP instead of HTTPS.')
55 | PARSER.add_argument('-d', '--delete', action='store_true', default=False,
56 | help='Use this option to delete the page instead of create it.')
57 | PARSER.add_argument('-l', '--loglevel', default='INFO',
58 | help='Use this option to set the log verbosity.')
59 | PARSER.add_argument('-s', '--simulate', action='store_true', default=False,
60 | help='Use this option to only show conversion result.')
61 | PARSER.add_argument('-v', '--version', type=int, action='store', default=1,
62 | help='Version of confluence page (default is 1).')
63 | PARSER.add_argument('-mds', '--markdownsrc', action='store', default='default',
64 | choices=['default', 'bitbucket'],
65 | help='Use this option to specify a markdown source (i.e. what processor this markdown was targeting). '
66 | 'Possible values: bitbucket.')
67 | PARSER.add_argument('--label', action='append', dest='labels', default=[],
68 | help='A list of labels to set on the page.')
69 | PARSER.add_argument('--property', action='append', dest='properties', default=[],
70 | type=lambda kv: kv.split("="),
71 | help='A list of content properties to set on the page.')
72 | PARSER.add_argument('--detail', action='append', dest='details', default=[],
73 | type=lambda kv: kv.split("="),
74 | help='A list of page details to set on the page')
75 | PARSER.add_argument('--hide-details', action='store_true', dest='details_visibility', default=False,
76 | help='Use this option to make page details table hidden.')
77 | PARSER.add_argument('--pages-map', action='append', dest='pages_map', default=[],
78 | type=lambda kv: kv.split("="),
79 | help='Use this option to specify a mapping between a base URL (for links) '
80 | 'and base local directory (for .md files) to resolve page to page links')
81 | PARSER.add_argument('--title', action='store', dest='title', default=None,
82 | help='Set the title for the page, otherwise the title is going to be the first line in the markdown file')
83 | PARSER.add_argument('--remove-emojies', action='store_true', dest='remove_emojies', default=False,
84 | help='Remove emojies if there are any. This may be need if the database doesn\'t support emojies')
85 |
86 | ARGS = PARSER.parse_args()
87 |
88 | # Assign global variables
89 | try:
90 | # Set log level
91 | LOGGER.setLevel(getattr(logging, ARGS.loglevel.upper(), None))
92 |
93 | MARKDOWN_FILE = ARGS.markdownFile
94 | SPACE_KEY = ARGS.spacekey
95 | USERNAME = os.getenv('CONFLUENCE_USERNAME', ARGS.username)
96 | API_KEY = os.getenv('CONFLUENCE_API_KEY', ARGS.apikey)
97 | PA_TOKEN = os.getenv('CONFLUENCE_PERSONAL_ACCESS_TOKEN', ARGS.pat)
98 | ORGNAME = os.getenv('CONFLUENCE_ORGNAME', ARGS.orgname)
99 | ANCESTOR = ARGS.ancestor
100 | NOSSL = ARGS.nossl
101 | DELETE = ARGS.delete
102 | SIMULATE = ARGS.simulate
103 | VERSION = ARGS.version
104 | MARKDOWN_SOURCE = ARGS.markdownsrc
105 | LABELS = ARGS.labels
106 | PROPERTIES = dict(ARGS.properties)
107 | DETAILS = dict(ARGS.details)
108 | DETAILS_VISIBILITY = ARGS.details_visibility
109 | PAGES_MAP = dict(ARGS.pages_map)
110 | ATTACHMENTS = ARGS.attachment
111 | GO_TO_PAGE = not ARGS.nogo
112 | CONTENTS = ARGS.contents
113 | TITLE = ARGS.title
114 | REMOVE_EMOJIES = ARGS.remove_emojies
115 |
116 | if USERNAME is None and PA_TOKEN is None:
117 | LOGGER.error('Error: Username/PAT Token not specified by environment variable or option.')
118 | sys.exit(1)
119 |
120 | if API_KEY is None and PA_TOKEN is None:
121 | LOGGER.error('Error: API key or Personal Access Token not specified by environment variable or option.')
122 | sys.exit(1)
123 |
124 | if not os.path.exists(MARKDOWN_FILE):
125 | LOGGER.error('Error: Markdown file: %s does not exist.', MARKDOWN_FILE)
126 | sys.exit(1)
127 |
128 | if SPACE_KEY is None:
129 | SPACE_KEY = '~%s' % (USERNAME)
130 |
131 | if ORGNAME is not None:
132 | if ORGNAME.find('.') != -1:
133 | CONFLUENCE_API_URL_TMP = 'https://%s' % ORGNAME
134 | else:
135 | CONFLUENCE_API_URL_TMP = 'https://%s.atlassian.net/wiki' % ORGNAME
136 | else:
137 | LOGGER.error('Error: Org Name not specified by environment variable or option.')
138 | sys.exit(1)
139 | CONFLUENCE_API_URL = CONFLUENCE_API_URL_TMP
140 | if NOSSL:
141 | CONFLUENCE_API_URL = CONFLUENCE_API_URL_TMP.replace('https://', 'http://')
142 | else:
143 | CONFLUENCE_API_URL = CONFLUENCE_API_URL_TMP
144 |
145 | except Exception as err:
146 | LOGGER.error('\n\nException caught:\n%s ', err)
147 | LOGGER.error('\nFailed to process command line arguments. Exiting.')
148 | sys.exit(1)
149 |
150 | def convert_comment_block(html):
151 | """
152 | Convert markdown code bloc to Confluence hidden comment
153 |
154 | :param html: string
155 | :return: modified html string
156 | """
157 | open_tag = ''
158 | close_tag = ''
159 |
160 | html = html.replace('', close_tag)
161 |
162 | return html
163 |
164 | def create_table_of_content(html):
165 | """
166 | Check for the string '[TOC]' and replaces it the Confluence "Table of Content" macro
167 |
168 | :param html: string
169 | :return: modified html string
170 | """
171 | html = re.sub(
172 | r'\[TOC\]
',
173 | '
',
174 | html,
175 | 1)
176 |
177 | return html
178 |
179 |
180 | def convert_code_block(html):
181 | """
182 | Convert html code blocks to Confluence macros
183 |
184 | :param html: string
185 | :return: modified html string
186 | """
187 | code_blocks = re.findall(r'.*?
', html, re.DOTALL)
188 | if code_blocks:
189 | for tag in code_blocks:
190 |
191 | conf_ml = ''
192 | conf_ml = conf_ml + 'Midnight'
193 | conf_ml = conf_ml + 'true'
194 |
195 | lang = re.search('code class="(.*)"', tag)
196 | if lang:
197 | lang = lang.group(1)
198 | else:
199 | lang = 'none'
200 |
201 | conf_ml = conf_ml + '' + lang + ''
202 | content = re.search(r'(.*?)
', tag, re.DOTALL).group(1)
203 | content = content.replace("]]", "]]]]>'
205 | conf_ml = conf_ml + content + ''
206 | conf_ml = conf_ml.replace('<', '<').replace('>', '>')
207 | conf_ml = conf_ml.replace('"', '"').replace('&', '&')
208 |
209 | html = html.replace(tag, conf_ml)
210 |
211 | return html
212 |
213 | def convert_iframe_macros(html):
214 | """[summary]
215 | Converts to Confluence iframe macro
216 |
217 | :param html: html string
218 | :return: modified html string
219 |
220 | """
221 |
222 | html_tag = '
'
224 |
225 | iframes = re.findall('', html, re.DOTALL)
226 | if iframes:
227 | for iframe in iframes:
228 | src = ''
229 | dst = html_tag + src + close_tag
230 | html = html.replace(src, dst)
231 |
232 | return html
233 |
234 | def remove_emojies(html):
235 | """
236 | Remove emojies if there are any
237 |
238 | :param html: string
239 | :return: modified html string
240 | """
241 | regrex_pattern = re.compile(pattern = "["
242 | u"\U0001F600-\U0001F64F" # emoticons
243 | u"\U0001F300-\U0001F5FF" # symbols & pictographs
244 | u"\U0001F680-\U0001F6FF" # transport & map symbols
245 | u"\U0001F1E0-\U0001F1FF" # flags (iOS)
246 | "]+", flags = re.UNICODE)
247 | return regrex_pattern.sub(r'',html)
248 |
249 |
250 | def convert_info_macros(html):
251 | """
252 | Converts html for info, note or warning macros
253 |
254 | :param html: html string
255 | :return: modified html string
256 | """
257 | info_tag = ''
258 | note_tag = info_tag.replace('info', 'note')
259 | warning_tag = info_tag.replace('info', 'warning')
260 | close_tag = '
'
261 |
262 | # Custom tags converted into macros
263 | html = html.replace('~?', info_tag).replace('?~
', close_tag)
264 | html = html.replace('~!', note_tag).replace('!~
', close_tag)
265 | html = html.replace('~%', warning_tag).replace('%~
', close_tag)
266 |
267 | # Convert block quotes into macros
268 | quotes = re.findall('(.*?)
', html, re.DOTALL)
269 | if quotes:
270 | for quote in quotes:
271 | note = re.search('^<.*>Note', quote.strip(), re.IGNORECASE)
272 | warning = re.search('^<.*>Warning', quote.strip(), re.IGNORECASE)
273 |
274 | if note:
275 | clean_tag = strip_type(quote, 'Note')
276 | macro_tag = clean_tag.replace('', note_tag).replace('
', close_tag).strip()
277 | elif warning:
278 | clean_tag = strip_type(quote, 'Warning')
279 | macro_tag = clean_tag.replace('', warning_tag).replace('
', close_tag).strip()
280 | else:
281 | macro_tag = quote.replace('', info_tag).replace('
', close_tag).strip()
282 |
283 | html = html.replace('%s
' % quote, macro_tag)
284 |
285 | # Convert doctoc to toc confluence macro
286 | html = convert_doctoc(html)
287 |
288 | return html
289 |
290 |
291 | def convert_doctoc(html):
292 | """
293 | Convert doctoc to confluence macro
294 |
295 | :param html: html string
296 | :return: modified html string
297 | """
298 |
299 | toc_tag = '''
300 |
301 | true
302 | disc
303 | 7
304 | 1
305 | list
306 | clear
307 | .*
308 |
309 |
'''
310 |
311 | html = re.sub('\<\!\-\- START doctoc.*END doctoc \-\-\>', toc_tag, html, flags=re.DOTALL)
312 |
313 | return html
314 |
315 |
316 | def strip_type(tag, tagtype):
317 | """
318 | Strips Note or Warning tags from html in various formats
319 |
320 | :param tag: tag name
321 | :param tagtype: tag type
322 | :return: modified tag
323 | """
324 | tag = re.sub('%s:\s' % tagtype, '', tag.strip(), re.IGNORECASE)
325 | tag = re.sub('%s\s:\s' % tagtype, '', tag.strip(), re.IGNORECASE)
326 | tag = re.sub('<.*?>%s:\s<.*?>' % tagtype, '', tag, re.IGNORECASE)
327 | tag = re.sub('<.*?>%s\s:\s<.*?>' % tagtype, '', tag, re.IGNORECASE)
328 | tag = re.sub('<(em|strong)>%s:<.*?>\s' % tagtype, '', tag, re.IGNORECASE)
329 | tag = re.sub('<(em|strong)>%s\s:<.*?>\s' % tagtype, '', tag, re.IGNORECASE)
330 | tag = re.sub('<(em|strong)>%s<.*?>:\s' % tagtype, '', tag, re.IGNORECASE)
331 | tag = re.sub('<(em|strong)>%s\s<.*?>:\s' % tagtype, '', tag, re.IGNORECASE)
332 | string_start = re.search('<.*?>', tag)
333 | tag = upper_chars(tag, [string_start.end()])
334 | return tag
335 |
336 |
337 | def upper_chars(string, indices):
338 | """
339 | Make characters uppercase in string
340 |
341 | :param string: string to modify
342 | :param indices: character indice to change to uppercase
343 | :return: uppercased string
344 | """
345 | upper_string = "".join(c.upper() if i in indices else c for i, c in enumerate(string))
346 | return upper_string
347 |
348 |
349 | def slug(string, lowercase):
350 | """
351 | Creates a slug string
352 |
353 | :param string: string to modify
354 | :param lowercase: bool indicating whether string has to be lowercased
355 | :return: slug string
356 | """
357 |
358 | slug_string = string
359 | if lowercase:
360 | slug_string = string.lower()
361 |
362 |
363 | # Remove all html code tags
364 | slug_string = re.sub(r'<[^>]+>', '', slug_string)
365 |
366 | # Remove html code like '&'
367 | slug_string = re.sub(r'&[a-z]+;', '', slug_string)
368 |
369 | # Replace all spaces ( ) with dash (-)
370 | slug_string = re.sub(r'[ ]', '-', slug_string)
371 |
372 | # Remove all special chars, except for dash (-)
373 | slug_string = re.sub(r'[^a-zA-Z0-9-]', '', slug_string)
374 | return slug_string
375 |
376 |
377 | def process_refs(html):
378 | """
379 | Process references
380 |
381 | :param html: html string
382 | :return: modified html string
383 | """
384 | refs = re.findall('\n(\[\^(\d)\].*)|(\[\^(\d)\].*)', html)
385 |
386 | if refs:
387 |
388 | for ref in refs:
389 | if ref[0]:
390 | full_ref = ref[0].replace('
', '').replace('', '')
391 | ref_id = ref[1]
392 | else:
393 | full_ref = ref[2]
394 | ref_id = ref[3]
395 |
396 | full_ref = full_ref.replace('
', '').replace('', '')
397 | html = html.replace(full_ref, '')
398 | href = re.search('href="(.*?)"', full_ref).group(1)
399 |
400 | superscript = '%s' % (href, ref_id)
401 | html = html.replace('[^%s]' % ref_id, superscript)
402 |
403 | return html
404 |
405 |
406 | def get_page(title):
407 | """
408 | Retrieve page details by title
409 |
410 | :param title: page tile
411 | :return: Confluence page info
412 | """
413 | LOGGER.info('\tRetrieving page information: %s', title)
414 | url = '%s/rest/api/content?title=%s&spaceKey=%s&expand=version,ancestors' % (
415 | CONFLUENCE_API_URL, urllib.parse.quote_plus(title), SPACE_KEY)
416 |
417 | # We retrieve content property values as part of page content
418 | # to make sure we are able to update them later
419 | if PROPERTIES:
420 | url = '%s,%s' % (url, ','.join("metadata.properties.%s" % v for v in PROPERTIES.keys()))
421 |
422 | session = requests.Session()
423 |
424 | if PA_TOKEN:
425 | session.headers.update({'Authorization': 'Bearer ' + PA_TOKEN})
426 | else:
427 | session.auth = (USERNAME, API_KEY)
428 |
429 | retry_max_requests=5
430 | retry_backoff_factor=0.1
431 | retry_status_forcelist=(404, 500, 501, 502, 503, 504)
432 | retry = requests.adapters.Retry(
433 | total=retry_max_requests,
434 | connect=retry_max_requests,
435 | read=retry_max_requests,
436 | backoff_factor=retry_backoff_factor,
437 | status_forcelist=retry_status_forcelist,
438 | )
439 | adapter = requests.adapters.HTTPAdapter(max_retries=retry)
440 | session.mount('http://', adapter)
441 | session.mount('https://', adapter)
442 |
443 | response = session.get(url)
444 |
445 | # Check for errors
446 | try:
447 | response.raise_for_status()
448 | except requests.RequestException as err:
449 | LOGGER.error('err.response: %s', err)
450 | if response.status_code == 404:
451 | LOGGER.error('Error: Page not found. Check the following are correct:')
452 | LOGGER.error('\tSpace Key : %s', SPACE_KEY)
453 | LOGGER.error('\tOrganisation Name: %s', ORGNAME)
454 | else:
455 | LOGGER.error('Error: %d - %s', response.status_code, response.content)
456 | sys.exit(1)
457 |
458 | data = response.json()
459 |
460 | LOGGER.debug("data: %s", str(data))
461 |
462 | if len(data[u'results']) >= 1:
463 | page_id = data[u'results'][0][u'id']
464 | version_num = data[u'results'][0][u'version'][u'number']
465 | link = '%s%s' % (CONFLUENCE_API_URL, data[u'results'][0][u'_links'][u'webui'])
466 |
467 | try:
468 | properties = data[u'results'][0][u'metadata'][u'properties']
469 | except KeyError:
470 | # In case when page has no content properties we can simply ignore them
471 | properties = {}
472 | pass
473 |
474 | page_info = collections.namedtuple('PageInfo', ['id', 'version', 'link', 'properties'])
475 | page = page_info(page_id, version_num, link, properties)
476 | return page
477 |
478 | return False
479 |
480 |
481 | # Scan for images and upload as attachments if found
482 | def add_images(page_id, html):
483 | """
484 | Scan for images and upload as attachments if found
485 |
486 | :param page_id: Confluence page id
487 | :param html: html string
488 | :return: html with modified image reference
489 | """
490 | source_folder = os.path.dirname(os.path.abspath(MARKDOWN_FILE))
491 |
492 | for tag in re.findall('', html):
493 | rel_path = re.search('src="(.*?)"', tag).group(1)
494 | alt_text = re.search('alt="(.*?)"', tag).group(1)
495 | abs_path = os.path.join(source_folder, rel_path)
496 | basename = os.path.basename(rel_path)
497 | upload_attachment(page_id, abs_path, alt_text)
498 | if re.search('http.*', rel_path) is None:
499 | if CONFLUENCE_API_URL.endswith('/wiki'):
500 | html = html.replace('%s' % (rel_path),
501 | '/wiki/download/attachments/%s/%s' % (page_id, basename))
502 | else:
503 | html = html.replace('%s' % (rel_path),
504 | '/download/attachments/%s/%s' % (page_id, basename))
505 | return html
506 |
507 |
508 | def add_contents(html):
509 | """
510 | Add contents page
511 |
512 | :param html: html string
513 | :return: modified html string
514 | """
515 | contents_markup = '\n' \
516 | 'true\ndisc'
517 | contents_markup = contents_markup + '5\n' \
518 | '1'
519 | contents_markup = contents_markup + 'rm-contents\n' \
520 | '\n' \
521 | 'list'
522 | contents_markup = contents_markup + 'false\n' \
523 | '\n' \
524 | ''
525 |
526 | html = contents_markup + '\n' + html
527 | return html
528 |
529 |
530 | def add_attachments(page_id, files):
531 | """
532 | Add attachments for an array of files
533 |
534 | :param page_id: Confluence page id
535 | :param files: list of files to attach to the given Confluence page
536 | :return: None
537 | """
538 | source_folder = os.path.dirname(os.path.abspath(MARKDOWN_FILE))
539 |
540 | if files:
541 | for file in files:
542 | upload_attachment(page_id, os.path.join(source_folder, file), '')
543 |
544 |
545 | def add_local_refs(page_id, title, html):
546 | """
547 | Convert local links to correct confluence local links
548 |
549 | :param page_title: string
550 | :param page_id: integer
551 | :param html: string
552 | :return: modified html string
553 | """
554 |
555 | ref_prefixes = {
556 | "default": "#",
557 | "bitbucket": "#markdown-header-"
558 | }
559 | ref_postfixes = {
560 | "default": "_%d",
561 | "bitbucket": "_%d"
562 | }
563 |
564 | # We ignore local references in case of unknown or unspecified markdown source
565 | if not MARKDOWN_SOURCE in ref_prefixes or \
566 | not MARKDOWN_SOURCE in ref_postfixes:
567 | LOGGER.warning('Local references weren\'t processed because '
568 | '--markdownsrc wasn\'t set or specified source isn\'t supported')
569 | return html
570 |
571 | ref_prefix = ref_prefixes[MARKDOWN_SOURCE]
572 | ref_postfix = ref_postfixes[MARKDOWN_SOURCE]
573 |
574 | LOGGER.info('Converting confluence local links...')
575 |
576 | headers = re.findall(r'(.*?)', html, re.DOTALL)
577 | if headers:
578 | headers_map = {}
579 | headers_count = {}
580 |
581 | for header in headers:
582 | key = ref_prefix + slug(header, True)
583 |
584 | if VERSION == 1:
585 | value = re.sub(r'(<.+?>|[ ])', '', header)
586 | if VERSION == 2:
587 | value = slug(header, False)
588 |
589 | if key in headers_map:
590 | alt_count = headers_count[key]
591 |
592 | alt_key = key + (ref_postfix % alt_count)
593 | alt_value = value + ('.%s' % alt_count)
594 |
595 | headers_map[alt_key] = alt_value
596 | headers_count[key] = alt_count + 1
597 | else:
598 | headers_map[key] = value
599 | headers_count[key] = 1
600 |
601 | links = re.findall(r'.+?', html)
602 | if links:
603 | for link in links:
604 | matches = re.search(r'(.+?)', link)
605 | ref = matches.group(1)
606 | alt = matches.group(2)
607 |
608 | LOGGER.debug('--- Found local link: %s', ref)
609 |
610 | if ref not in headers_map:
611 | LOGGER.error("Invalid '%s' local link detected: '%s'. Please update the source file or change the markdown source (-mds) parameter.", MARKDOWN_SOURCE, ref)
612 | sys.exit(1)
613 |
614 | result_ref = headers_map.get(ref)
615 |
616 | LOGGER.debug('--- Found local header: %s', result_ref)
617 |
618 | if result_ref:
619 | base_uri = '%s/spaces/%s/pages/%s/%s' % (CONFLUENCE_API_URL, SPACE_KEY, page_id, '+'.join(title.split()))
620 | if VERSION == 1:
621 | replacement_uri = '%s#%s-%s' % (base_uri, ''.join(title.split()), result_ref)
622 | replacement = '' % (result_ref, re.sub(r'( *<.+?> *)', ' ', alt))
623 | if VERSION == 2:
624 | replacement_uri = '%s#%s' % (base_uri, result_ref)
625 | replacement = '%s' % (replacement_uri, alt, alt)
626 |
627 | html = html.replace(link, replacement)
628 |
629 | LOGGER.info('\tTransformed "%s" to "%s"', ref, replacement_uri)
630 |
631 | return html
632 |
633 |
634 | def add_pages_refs(html):
635 | """
636 | Convert markdown page to page links to correct confluence page to page links
637 |
638 | :param html: string
639 | :return: modified html string
640 | """
641 |
642 | # We ignore page to page references if no maps are specified
643 | if not PAGES_MAP:
644 | LOGGER.warning('Page to page references weren\'t processed because '
645 | '--pages_map weren\'t specified')
646 | return html
647 |
648 | LOGGER.info('Converting confluence page to page links...')
649 |
650 | links = re.findall(r'.+?', html)
651 | if links:
652 | for link in links:
653 | matches = re.search(r'(.+?)', link)
654 | ref = matches.group(1)
655 | alt = matches.group(2)
656 |
657 | LOGGER.debug('--- Found page to page link: %s', ref)
658 | for key in PAGES_MAP:
659 | if ref.startswith(key):
660 | path = os.path.join(PAGES_MAP[key], urllib.parse.unquote(ref[len(key):]))
661 |
662 | LOGGER.debug('--- Possible page local path: %s', path)
663 | try:
664 | with open(path, 'r') as mdfile:
665 | title = mdfile.readline().lstrip('#').strip()
666 |
667 | LOGGER.debug('--- Found local page: %s', title)
668 |
669 | page = get_page(title)
670 | if not page:
671 | LOGGER.error('Cannot find confluence page "%s"', title)
672 | sys.exit(1)
673 |
674 | LOGGER.debug('--- Found confluence page: %s', page.link)
675 |
676 | replacement = '%s' % (page.link, alt, alt)
677 | html = html.replace(link, replacement)
678 |
679 | LOGGER.info('\tTransformed "%s" to "%s"', ref, page.link)
680 |
681 | except IOError:
682 | LOGGER.error('Cannot find local file "%s" when resolving page to page link "%s"', path, ref)
683 |
684 | return html
685 |
686 |
687 | def create_page(title, body, ancestors):
688 | """
689 | Create a new page
690 |
691 | :param title: confluence page title
692 | :param body: confluence page content
693 | :param ancestors: confluence page ancestor
694 | :return:
695 | """
696 | LOGGER.info('Creating page...')
697 |
698 | url = '%s/rest/api/content/' % CONFLUENCE_API_URL
699 |
700 | session = requests.Session()
701 | if PA_TOKEN:
702 | session.headers.update({'Authorization': 'Bearer ' + PA_TOKEN})
703 | else:
704 | session.auth = (USERNAME, API_KEY)
705 | session.headers.update({'Content-Type': 'application/json'})
706 |
707 | new_page = {
708 | 'type': 'page',
709 | 'title': title,
710 | 'space': {'key': SPACE_KEY},
711 | 'body': {
712 | 'storage': {
713 | 'value': body,
714 | 'representation': 'storage'
715 | }
716 | },
717 | 'ancestors': ancestors,
718 | 'metadata': {
719 | 'properties': {
720 | 'editor': {
721 | 'value': 'v%d' % VERSION
722 | }
723 | }
724 | }
725 | }
726 |
727 | LOGGER.debug("data: %s", json.dumps(new_page))
728 |
729 | response = session.post(url, data=json.dumps(new_page))
730 | try:
731 | response.raise_for_status()
732 | except requests.exceptions.HTTPError as excpt:
733 | LOGGER.error("error: %s - %s", excpt, response.content)
734 | exit(1)
735 |
736 | if response.status_code == 200:
737 | data = response.json()
738 | space_name = data[u'space'][u'name']
739 | page_id = data[u'id']
740 | version = data[u'version'][u'number']
741 | link = '%s%s' % (CONFLUENCE_API_URL, data[u'_links'][u'webui'])
742 |
743 | LOGGER.info('Page created in %s with ID: %s.', space_name, page_id)
744 | LOGGER.info('URL: %s', link)
745 |
746 | # Populate properties dictionary with initial property values
747 | properties = {}
748 | if PROPERTIES:
749 | for key in PROPERTIES:
750 | properties[key] = {"key": key, "version": 1, "value": PROPERTIES[key]}
751 |
752 | img_check = re.search('', body)
753 | local_ref_check = re.search('(.+?)', body)
754 | if img_check or local_ref_check or properties or ATTACHMENTS or LABELS:
755 | LOGGER.info('\tAttachments, local references, content properties or labels found, update procedure called.')
756 | update_page(page_id, title, body, version, ancestors, properties, ATTACHMENTS)
757 | else:
758 | if GO_TO_PAGE:
759 | webbrowser.open(link)
760 | else:
761 | LOGGER.error('Could not create page.')
762 | sys.exit(1)
763 |
764 |
765 | def delete_page(page_id):
766 | """
767 | Delete a page
768 |
769 | :param page_id: confluence page id
770 | :return: None
771 | """
772 | LOGGER.info('Deleting page...')
773 | url = '%s/rest/api/content/%s' % (CONFLUENCE_API_URL, page_id)
774 |
775 | session = requests.Session()
776 | if PA_TOKEN:
777 | session.headers.update({'Authorization': 'Bearer ' + PA_TOKEN})
778 | else:
779 | session.auth = (USERNAME, API_KEY)
780 | session.headers.update({'Content-Type': 'application/json'})
781 |
782 | response = session.delete(url)
783 | response.raise_for_status()
784 |
785 | if response.status_code == 204:
786 | LOGGER.info('Page %s deleted successfully.', page_id)
787 | else:
788 | LOGGER.error('Page %s could not be deleted.', page_id)
789 |
790 |
791 | def update_page(page_id, title, body, version, ancestors, properties, attachments):
792 | """
793 | Update a page
794 |
795 | :param page_id: confluence page id
796 | :param title: confluence page title
797 | :param body: confluence page content
798 | :param version: confluence page version
799 | :param ancestors: confluence page ancestor
800 | :param attachments: confluence page attachments
801 | :return: None
802 | """
803 | LOGGER.info('Updating page...')
804 |
805 | # Add images and attachments
806 | body = add_images(page_id, body)
807 | add_attachments(page_id, attachments)
808 |
809 | # Add local references
810 | body = add_local_refs(page_id, title, body)
811 |
812 | # Add page to page references
813 | body = add_pages_refs(body)
814 |
815 | url = '%s/rest/api/content/%s' % (CONFLUENCE_API_URL, page_id)
816 |
817 | session = requests.Session()
818 | if PA_TOKEN:
819 | session.headers.update({'Authorization': 'Bearer ' + PA_TOKEN})
820 | else:
821 | session.auth = (USERNAME, API_KEY)
822 | session.headers.update({'Content-Type': 'application/json'})
823 |
824 | page_json = {
825 | "id": page_id,
826 | "type": "page",
827 | "title": title,
828 | "space": {"key": SPACE_KEY},
829 | "body": {
830 | "storage": {
831 | "value": body,
832 | "representation": "storage"
833 | }
834 | },
835 | "version": {
836 | "number": version + 1,
837 | "minorEdit" : True
838 | },
839 | 'ancestors': ancestors
840 | }
841 |
842 | if LABELS:
843 | if 'metadata' not in page_json:
844 | page_json['metadata'] = {}
845 |
846 | labels = []
847 | for value in LABELS:
848 | labels.append({"name": value})
849 |
850 | page_json['metadata']['labels'] = labels
851 |
852 | response = session.put(url, data=json.dumps(page_json))
853 |
854 | # Check for errors
855 | try:
856 | response.raise_for_status()
857 | except requests.RequestException as err:
858 | LOGGER.error('err.response: %s', err)
859 | if response.status_code == 404:
860 | LOGGER.error('Error: Page not found. Check the following are correct:')
861 | LOGGER.error('\tSpace Key : %s', SPACE_KEY)
862 | LOGGER.error('\tOrganisation Name: %s', ORGNAME)
863 | else:
864 | LOGGER.error('Error: %d - %s', response.status_code, response.content)
865 | sys.exit(1)
866 |
867 | if response.status_code == 200:
868 | data = response.json()
869 | link = '%s%s' % (CONFLUENCE_API_URL, data[u'_links'][u'webui'])
870 |
871 | LOGGER.info("Page updated successfully.")
872 | LOGGER.info('URL: %s', link)
873 |
874 | if properties:
875 | LOGGER.info("Updating page content properties...")
876 |
877 | for key in properties:
878 | prop_url = '%s/property/%s' % (url, key)
879 | prop_json = {"key": key, "version": {"number": properties[key][u"version"]}, "value": properties[key][u"value"]}
880 |
881 | response = session.put(prop_url, data=json.dumps(prop_json))
882 | response.raise_for_status()
883 |
884 | if response.status_code == 200:
885 | LOGGER.info("\tUpdated property %s", key)
886 |
887 | if GO_TO_PAGE:
888 | webbrowser.open(link)
889 | else:
890 | LOGGER.error("Page could not be updated.")
891 |
892 |
893 | def get_attachment(page_id, filename):
894 | """
895 | Get page attachment
896 |
897 | :param page_id: confluence page id
898 | :param filename: attachment filename
899 | :return: attachment info in case of success, False otherwise
900 | """
901 | url = '%s/rest/api/content/%s/child/attachment?filename=%s' % (CONFLUENCE_API_URL, page_id, filename)
902 |
903 | session = requests.Session()
904 | if PA_TOKEN:
905 | session.headers.update({'Authorization': 'Bearer ' + PA_TOKEN})
906 | else:
907 | session.auth = (USERNAME, API_KEY)
908 |
909 | response = session.get(url)
910 | response.raise_for_status()
911 | data = response.json()
912 |
913 | if len(data[u'results']) >= 1:
914 | att_id = data[u'results'][0]['id']
915 | att_info = collections.namedtuple('AttachmentInfo', ['id'])
916 | attr_info = att_info(att_id)
917 | return attr_info
918 |
919 | return False
920 |
921 |
922 | def upload_attachment(page_id, file, comment):
923 | """
924 | Upload an attachement
925 |
926 | :param page_id: confluence page id
927 | :param file: attachment file
928 | :param comment: attachment comment
929 | :return: boolean
930 | """
931 | if re.search('http.*', file):
932 | return False
933 |
934 | content_type = mimetypes.guess_type(file)[0]
935 | filename = os.path.basename(file)
936 |
937 | if not os.path.isfile(file):
938 | LOGGER.error('File %s cannot be found --> skip ', file)
939 | return False
940 |
941 | file_to_upload = {
942 | 'comment': comment,
943 | 'file': (filename, open(file, 'rb'), content_type, {'Expires': '0'})
944 | }
945 |
946 | attachment = get_attachment(page_id, filename)
947 | if attachment:
948 | url = '%s/rest/api/content/%s/child/attachment/%s/data' % (CONFLUENCE_API_URL, page_id, attachment.id)
949 | else:
950 | url = '%s/rest/api/content/%s/child/attachment/' % (CONFLUENCE_API_URL, page_id)
951 |
952 | session = requests.Session()
953 | if PA_TOKEN:
954 | session.headers.update({'Authorization': 'Bearer ' + PA_TOKEN})
955 | else:
956 | session.auth = (USERNAME, API_KEY)
957 | session.headers.update({'X-Atlassian-Token': 'no-check'})
958 |
959 | LOGGER.info('\tUploading attachment %s...', filename)
960 |
961 | response = session.post(url, files=file_to_upload)
962 | response.raise_for_status()
963 |
964 | return True
965 |
966 |
967 | def main():
968 | """
969 | Main program
970 |
971 | :return:
972 | """
973 | LOGGER.info('\t\t----------------------------------')
974 | LOGGER.info('\t\tMarkdown to Confluence Upload Tool')
975 | LOGGER.info('\t\t----------------------------------\n\n')
976 |
977 | LOGGER.info('Markdown file:\t%s', MARKDOWN_FILE)
978 | LOGGER.info('Space Key:\t%s', SPACE_KEY)
979 |
980 | if TITLE:
981 | title = TITLE
982 | else:
983 | with open(MARKDOWN_FILE, 'r') as mdfile:
984 | title = mdfile.readline().lstrip('#').strip()
985 | mdfile.seek(0)
986 |
987 | LOGGER.info('Title:\t\t%s', title)
988 |
989 | with codecs.open(MARKDOWN_FILE, 'r', 'utf-8') as mdfile:
990 | html = mdfile.read()
991 | html = markdown.markdown(html, extensions=['tables', 'fenced_code', 'footnotes'])
992 |
993 | if not TITLE:
994 | html = '\n'.join(html.split('\n')[1:])
995 |
996 | if DETAILS:
997 | LOGGER.info('Generating page properties macro...')
998 |
999 | # Print 'page properties' macro on a page
1000 | details = '''
1001 |
1002 | ''' + str(DETAILS_VISIBILITY) + '''
1003 |
1004 |
1005 | '''
1006 |
1007 | for key in DETAILS:
1008 | details += '| ' + key + ' | ' + DETAILS[key] + ' |
|---|
'
1009 |
1010 | details += '''
1011 |
1012 |
1013 |
1014 | '''
1015 |
1016 | html = details + html
1017 |
1018 |
1019 | html = create_table_of_content(html)
1020 |
1021 | html = convert_info_macros(html)
1022 | html = convert_comment_block(html)
1023 | html = convert_code_block(html)
1024 | html = convert_iframe_macros(html)
1025 |
1026 | if REMOVE_EMOJIES:
1027 | html = remove_emojies(html)
1028 |
1029 | if CONTENTS:
1030 | html = add_contents(html)
1031 |
1032 | html = process_refs(html)
1033 |
1034 | LOGGER.debug('html: %s', html)
1035 |
1036 | if SIMULATE:
1037 | LOGGER.info("Simulate mode is active - stop processing here.")
1038 | sys.exit(0)
1039 |
1040 | LOGGER.info('Checking if Atlas page exists...')
1041 | page = get_page(title)
1042 |
1043 | if DELETE and page:
1044 | delete_page(page.id)
1045 | sys.exit(1)
1046 |
1047 | if ANCESTOR:
1048 | parent_page = get_page(ANCESTOR)
1049 | if parent_page:
1050 | ancestors = [{'type': 'page', 'id': parent_page.id}]
1051 | else:
1052 | LOGGER.error('Error: Parent page does not exist: %s', ANCESTOR)
1053 | sys.exit(1)
1054 | else:
1055 | ancestors = []
1056 |
1057 | if page:
1058 | # Populate properties dictionary with updated property values
1059 | properties = {}
1060 | if PROPERTIES:
1061 | for key in PROPERTIES:
1062 | if key in page.properties:
1063 | properties[key] = {"key": key, "version": page.properties[key][u'version'][u'number'] + 1, "value": PROPERTIES[key]}
1064 | else:
1065 | properties[key] = {"key": key, "version": 1, "value": PROPERTIES[key]}
1066 |
1067 | update_page(page.id, title, html, page.version, ancestors, properties, ATTACHMENTS)
1068 | else:
1069 | create_page(title, html, ancestors)
1070 |
1071 | LOGGER.info('Markdown Converter completed successfully.')
1072 |
1073 |
1074 | if __name__ == "__main__":
1075 | main()
1076 |
--------------------------------------------------------------------------------