4 | 
5 |
6 | **pedantmark** is (not only) a python binding for the GitHub's fork of CommonMark (cmark).
7 | It has also been enhanced to support custom renderers.
8 |
9 | > Only two maybes I've thought of: Strict Markdown or Pedantic Markdown. "Strict" still doesn't seem right.
10 | >
11 | > -- [John Gruber](https://twitter.com/gruber/status/507615356295200770)
12 |
13 | Ok, let's call it **pedantmark**.
14 |
15 |
16 | ## Install
17 |
18 | **pedantmark** is available in Python 2.7 and 3.5+ for Linux and Mac,
19 | Python 3.5+ for Windows. Wheels are built by [multibuild][].
20 |
21 | Install wheels by pip:
22 |
23 | $ pip install pedantmark
24 |
25 | [multibuild]: https://github.com/matthew-brett/multibuild
26 |
27 |
28 | ## Standard Usage
29 |
30 | The C source code has serval built-in renderers. The simplest interface is
31 | `pedantmark.html(text, options)`, which will render text into HTML.
32 |
33 | ```python
34 | import pedantmark
35 |
36 | text = '...'
37 | html = pedantmark.html(text, options=[pedantmark.OPT_VALIDATE_UTF8])
38 | ```
39 |
40 | The function `pedantmark.html()` accepts no extensions, but you can add
41 | extensions via `pedantmark.markdown()`:
42 |
43 | ```python
44 | import pedantmark
45 |
46 | text = '...'
47 | html = pedantmark.markdown(
48 | text,
49 | options=[pedantmark.OPT_VALIDATE_UTF8],
50 | extensions=['strikethrough', 'autolink', 'table'],
51 | renderer='html',
52 | )
53 | ```
54 |
55 | Available extensions: `table`, `autolink`, `tagfilter`, `strikethrough`.
56 | You can enable them all with a shortcut:
57 |
58 | pedantmark.markdown(..., extensions=pedantmark.EXTENSIONS)
59 |
60 | Available renderers: `html`, `xml`, `man`, `commonmark`, `plaintext`,
61 | and `latex`.
62 |
63 | ## Options
64 |
65 | Here is a full list of options:
66 |
67 | ```
68 | #: Include a `data-sourcepos` attribute on all block elements.
69 | OPT_SOURCEPOS
70 |
71 | #: Render `softbreak` elements as hard line breaks.
72 | OPT_HARDBREAKS
73 |
74 | #: Render `softbreak` elements as spaces.
75 | OPT_NOBREAKS
76 |
77 | #: Validate UTF-8 in the input before parsing, replacing illegal
78 | #: sequences with the replacement character U+FFFD.
79 | OPT_VALIDATE_UTF8
80 |
81 | #: Convert straight quotes to curly, --- to em dashes, -- to en dashes.
82 | OPT_SMART
83 |
84 | #: Use GitHub-style tags for code blocks instead of
85 | #: .
86 | OPT_PRE_LANG
87 |
88 | #: Be liberal in interpreting inline HTML tags.
89 | OPT_LIBERAL_HTML_TAG
90 |
91 | #: Parse footnotes.
92 | OPT_FOOTNOTES
93 |
94 | #: Only parse strikethroughs if surrounded by exactly 2 tildes.
95 | OPT_STRIKETHROUGH_DOUBLE_TILDE
96 |
97 | #: Use style attributes to align table cells instead of align attributes.
98 | OPT_TABLE_PREFER_STYLE_ATTRIBUTES
99 |
100 | #: Include the remainder of the info string in code blocks in
101 | #: a separate attribute.
102 | OPT_FULL_INFO_STRING
103 |
104 | #: Allow raw HTML and unsafe links, `javascript:`, `vbscript:`, `file:`,
105 | #: and all `data:` URLs -- by default, only `image/png`, `image/gif`,
106 | #: `image/jpeg`, or `image/webp` mime types are allowed. Without this
107 | #: option, raw HTML is replaced by a placeholder HTML comment, and unsafe
108 | #: links are replaced by empty strings.
109 | OPT_UNSAFE
110 | ```
111 |
112 | ## Custom Renderer
113 |
114 | Besides the native renderers, **pedantmark** has provided you a custom renderer,
115 | which you can customize the output yourself. Here is an example of pygments code
116 | highlighting integration:
117 |
118 | ```python
119 | from pedantmark import HTMLRenderer, markdown
120 | from pygments import highlight
121 | from pygments.lexers import get_lexer_by_name
122 | from pygments.formatters import html
123 |
124 | class MyRenderer(HTMLRenderer):
125 | def code_block(self, code, lang):
126 | if lang:
127 | # everything is in bytes
128 | lang = lang.decode('utf-8')
129 | code = code.decode('utf-8')
130 | lexer = get_lexer_by_name(lang, stripall=True)
131 | formatter = html.HtmlFormatter()
132 | output = highlight(code, lexer, formatter)
133 | # return bytes
134 | return output.encode('utf-8')
135 | return super(MyRenderer, self).code_block(code, lang)
136 |
137 | text = '...'
138 | markdown(text, renderer=MyRenderer())
139 | ```
140 |
141 | The default `HTMLRenderer` has a built-in hook for code highlight, you don't need
142 | to subclass at all:
143 |
144 | ```python
145 | def add_code_highlight(code, lang):
146 | lang = lang.decode('utf-8')
147 | code = code.decode('utf-8')
148 | lexer = get_lexer_by_name(lang, stripall=True)
149 | formatter = html.HtmlFormatter()
150 | output = highlight(code, lexer, formatter)
151 | return output.encode('utf-8')
152 |
153 | text = '...'
154 | markdown(text, renderer=HTMLRenderer(highlight=add_code_highlight))
155 | ```
156 |
157 | Here is a full list of renderers:
158 |
159 | ```
160 | thematic_break(self)
161 | html_block(self, text)
162 | block_quote(self, text)
163 | code_block(self, text, lang)
164 | heading(self, text, level, index=None)
165 | paragraph(self, text)
166 | list_item(self, text)
167 | list(self, text, ordered, start=None)
168 | table_cell(self, text, tag, align=None)
169 | table_row(self, text)
170 | table_header(self, text)
171 | table(self, text)
172 | softbreak(self)
173 | linebreak(self)
174 | html_inline(self, text)
175 | text(self, text)
176 | emph(self, text)
177 | strong(self, text)
178 | strikethrough(self, text)
179 | code(self, text)
180 | link(self, url, text, title)
181 | image(self, src, alt, title)
182 | footnote_ref(self, key)
183 | footnote_item(self, text, index):
184 | footnotes(self, text)
185 | ```
186 |
187 | ## TOC
188 |
189 | To render "Table of Contents", you need to use a `MarkdownState` to record TOC
190 | contents:
191 |
192 | ```python
193 | from pedantmark import MarkdownState
194 | from pedantmark.toc import render_toc
195 |
196 | # only record level 1 ~ 3 headings
197 | state = MarkdownState(toc_level=3)
198 |
199 | markdown(text, renderer=HTMLRenderer(), state)
200 | render_toc(state.toc)
201 | ```
202 |
203 | ## Author & License
204 |
205 | This library is created by Hsiaming Yang, licensed under BSD.
206 |
--------------------------------------------------------------------------------
/README.rst:
--------------------------------------------------------------------------------
1 | Pedantic Markdown
2 | =================
3 |
4 | **pedantmark** is (not only) a python binding for the GitHub's fork of CommonMark (cmark).
5 | It has also been enhanced to support custom renderers.
6 |
7 | Install
8 | -------
9 |
10 | **pedantmark** is available in Python 2.7 and 3.5+ for Linux and Mac,
11 | Python 3.5+ for Windows.
12 |
13 | Install wheels by pip::
14 |
15 | $ pip install pedantmark
16 |
17 | Usage
18 | -----
19 |
20 | A simple overview of how to use pedantmark:
21 |
22 | .. code-block:: python
23 |
24 | import pedantmark
25 |
26 | text = '...'
27 | html = pedantmark.markdown(
28 | text,
29 | options=[pedantmark.OPT_VALIDATE_UTF8],
30 | extensions=['strikethrough', 'autolink', 'table'],
31 | renderer='html',
32 | )
33 |
34 | Available extensions: ``table``, ``autolink``, ``tagfilter``, ``strikethrough``.
35 | You can enable them all with a shortcut::
36 |
37 | pedantmark.markdown(..., extensions=pedantmark.EXTENSIONS)
38 |
39 | Available renderers: ``html``, ``xml``, ``man``, ``commonmark``, ``plaintext``,
40 | ``latex`` and custom renderers.
41 |
42 | Author & License
43 | ----------------
44 |
45 | This library is created by Hsiaming Yang, licensed under BSD.
46 |
--------------------------------------------------------------------------------
/build_ffi.py:
--------------------------------------------------------------------------------
1 | import sys
2 | import glob
3 | import cffi
4 |
5 |
6 | INCLUDE_DIRS = [
7 | 'cmark-gfm/src',
8 | 'cmark-gfm/extensions',
9 | 'pedantmark',
10 | ]
11 | SOURCES_FILES = [
12 | f for f in glob.iglob('cmark-gfm/src/*.c') if not f.endswith('main.c')
13 | ]
14 | SOURCES_FILES.extend(list(glob.iglob('cmark-gfm/extensions/*.c')))
15 | SOURCES_FILES.append('pedantmark/extern.c')
16 |
17 |
18 | def _compiler_type():
19 | import distutils.dist
20 | import distutils.ccompiler
21 | dist = distutils.dist.Distribution()
22 | dist.parse_config_files()
23 | cmd = dist.get_command_obj('build')
24 | cmd.ensure_finalized()
25 | compiler = distutils.ccompiler.new_compiler(compiler=cmd.compiler)
26 | return compiler.compiler_type
27 |
28 |
29 | COMPILER_TYPE = _compiler_type()
30 | if COMPILER_TYPE == 'unix':
31 | EXTRA_COMPILE_ARGS = ['-std=c99']
32 | INCLUDE_DIRS.append('vendors/unix/src')
33 | INCLUDE_DIRS.append('vendors/unix/extensions')
34 | elif COMPILER_TYPE == 'msvc':
35 | EXTRA_COMPILE_ARGS = ['/TP']
36 | INCLUDE_DIRS.append('vendors/windows/src')
37 | INCLUDE_DIRS.append('vendors/windows/extensions')
38 |
39 |
40 | ffi = cffi.FFI()
41 |
42 | with open('pedantmark/cmark_cdef.h', 'r') as f:
43 | CDEF_H = f.read()
44 |
45 | with open('pedantmark/extern_cdef.h', 'r') as f:
46 | CDEF_H += f.read()
47 |
48 | MODULE_H = '''
49 | #ifndef CMARK_MODULE_H
50 | #define CMARK_MODULE_H
51 |
52 | #ifdef __cplusplus
53 | extern "C" {
54 | #endif
55 |
56 | #define CMARKEXTENSIONS_STATIC_DEFINE
57 |
58 | #include "cmark-gfm.h"
59 | #include "cmark-gfm-extension_api.h"
60 | #include "cmark-gfm-core-extensions.h"
61 | #include "extern.h"
62 |
63 | #ifdef __cplusplus
64 | }
65 | #endif
66 |
67 | #endif
68 | '''
69 |
70 | ffi.cdef(CDEF_H)
71 | ffi.set_source(
72 | 'pedantmark._cmark',
73 | MODULE_H,
74 | sources=SOURCES_FILES,
75 | include_dirs=INCLUDE_DIRS,
76 | extra_compile_args=EXTRA_COMPILE_ARGS,
77 | )
78 |
79 | if __name__ == '__main__':
80 | ffi.compile(verbose=True)
81 |
--------------------------------------------------------------------------------
/pedantmark/__init__.py:
--------------------------------------------------------------------------------
1 | from .api import (
2 | markdown, html, EXTENSIONS,
3 | OPT_SOURCEPOS,
4 | OPT_HARDBREAKS,
5 | OPT_NOBREAKS,
6 | OPT_VALIDATE_UTF8,
7 | OPT_SMART,
8 | OPT_PRE_LANG,
9 | OPT_LIBERAL_HTML_TAG,
10 | OPT_FOOTNOTES,
11 | OPT_STRIKETHROUGH_DOUBLE_TILDE,
12 | OPT_TABLE_PREFER_STYLE_ATTRIBUTES,
13 | OPT_FULL_INFO_STRING,
14 | OPT_UNSAFE,
15 | )
16 | from .extern import (
17 | escape_html, escape_href,
18 | MarkdownState, HTMLRenderer,
19 | )
20 |
21 | __all__ = [
22 | 'markdown', 'html', 'EXTENSIONS',
23 | 'OPT_SOURCEPOS',
24 | 'OPT_HARDBREAKS',
25 | 'OPT_NOBREAKS',
26 | 'OPT_VALIDATE_UTF8',
27 | 'OPT_SMART',
28 | 'OPT_PRE_LANG',
29 | 'OPT_LIBERAL_HTML_TAG',
30 | 'OPT_FOOTNOTES',
31 | 'OPT_STRIKETHROUGH_DOUBLE_TILDE',
32 | 'OPT_TABLE_PREFER_STYLE_ATTRIBUTES',
33 | 'OPT_FULL_INFO_STRING',
34 | 'OPT_UNSAFE',
35 | 'escape_html', 'escape_href',
36 | 'MarkdownState', 'HTMLRenderer',
37 | ]
38 |
39 | __version__ = '0.1'
40 | __author__ = 'Hsiaoming Yang '
41 |
--------------------------------------------------------------------------------
/pedantmark/api.py:
--------------------------------------------------------------------------------
1 | from ._cmark import lib, ffi
2 |
3 | #: Include a `data-sourcepos` attribute on all block elements.
4 | OPT_SOURCEPOS = lib.CMARK_OPT_SOURCEPOS
5 |
6 | #: Render `softbreak` elements as hard line breaks.
7 | OPT_HARDBREAKS = lib.CMARK_OPT_HARDBREAKS
8 |
9 | #: Render `softbreak` elements as spaces.
10 | OPT_NOBREAKS = lib.CMARK_OPT_NOBREAKS
11 |
12 | #: Validate UTF-8 in the input before parsing, replacing illegal
13 | #: sequences with the replacement character U+FFFD.
14 | OPT_VALIDATE_UTF8 = lib.CMARK_OPT_VALIDATE_UTF8
15 |
16 | #: Convert straight quotes to curly, --- to em dashes, -- to en dashes.
17 | OPT_SMART = lib.CMARK_OPT_SMART
18 |
19 | #: Use GitHub-style tags for code blocks instead of
20 | #: .
21 | OPT_PRE_LANG = lib.CMARK_OPT_GITHUB_PRE_LANG
22 |
23 | #: Be liberal in interpreting inline HTML tags.
24 | OPT_LIBERAL_HTML_TAG = lib.CMARK_OPT_LIBERAL_HTML_TAG
25 |
26 | #: Parse footnotes.
27 | OPT_FOOTNOTES = lib.CMARK_OPT_FOOTNOTES
28 |
29 | #: Only parse strikethroughs if surrounded by exactly 2 tildes.
30 | OPT_STRIKETHROUGH_DOUBLE_TILDE = lib.CMARK_OPT_STRIKETHROUGH_DOUBLE_TILDE
31 |
32 | #: Use style attributes to align table cells instead of align attributes.
33 | OPT_TABLE_PREFER_STYLE_ATTRIBUTES = lib.CMARK_OPT_TABLE_PREFER_STYLE_ATTRIBUTES
34 |
35 | #: Include the remainder of the info string in code blocks in
36 | #: a separate attribute.
37 | OPT_FULL_INFO_STRING = lib.CMARK_OPT_FULL_INFO_STRING
38 |
39 | #: Allow raw HTML and unsafe links, `javascript:`, `vbscript:`, `file:`,
40 | #: and all `data:` URLs -- by default, only `image/png`, `image/gif`,
41 | #: `image/jpeg`, or `image/webp` mime types are allowed. Without this
42 | #: option, raw HTML is replaced by a placeholder HTML comment, and unsafe
43 | #: links are replaced by empty strings.
44 | OPT_UNSAFE = lib.CMARK_OPT_UNSAFE
45 |
46 | #: Available extensions
47 | EXTENSIONS = ('table', 'autolink', 'tagfilter', 'strikethrough')
48 |
49 |
50 | def markdown(text, options=None, extensions=None, renderer='html',
51 | width=0, state=None):
52 | """Parse text as markdown and render it into other format of string.
53 |
54 | :text: string of text.
55 | :options: list of options for cmark.
56 | :extensions: list of extensions.
57 | :renderer: renderer type or HTMLRenderer instance.
58 | :width: width option for renderers like man/latex.
59 | :state: MarkdownState instance for HTMLRenderer.
60 | :return: string
61 | """
62 | if options is None:
63 | options = []
64 |
65 | if extensions is None:
66 | extensions = []
67 |
68 | _options = get_options(*options)
69 | parser = lib.cmark_parser_new(_options)
70 | try:
71 | _extensions = attach_extensions(parser, *extensions)
72 |
73 | bytes_text = text.encode('utf-8')
74 | lib.cmark_parser_feed(parser, bytes_text, len(bytes_text))
75 | root = lib.cmark_parser_finish(parser)
76 |
77 | output = _render_root(
78 | renderer, root, _options, _extensions, width, state)
79 | finally:
80 | lib.cmark_parser_free(parser)
81 | return output
82 |
83 |
84 | def html(text, options=None):
85 | """Parse text as markdown and render it into HTML string.
86 |
87 | :text: string of text.
88 | :options: list of options for cmark.
89 | :return: HTML string.
90 | """
91 | if options is None:
92 | options = []
93 |
94 | _options = get_options(*options)
95 | bytes_text = text.encode('utf-8')
96 | rv = lib.cmark_markdown_to_html(bytes_text, len(bytes_text), _options)
97 | return ffi.string(rv).decode('utf-8')
98 |
99 |
100 | def attach_extensions(parser, *names):
101 | if not names:
102 | return ffi.NULL
103 |
104 | lib.cmark_gfm_core_extensions_ensure_registered()
105 | for name in names:
106 | extension = lib.cmark_find_syntax_extension(name.encode('utf-8'))
107 | if extension == ffi.NULL:
108 | raise ValueError('Invalid extensions: {!r}'.format(name))
109 | lib.cmark_parser_attach_syntax_extension(parser, extension)
110 |
111 | return lib.cmark_parser_get_syntax_extensions(parser)
112 |
113 |
114 | def get_options(*options):
115 | value = lib.CMARK_OPT_DEFAULT
116 | for opt in options:
117 | value = value | opt
118 | return value
119 |
120 |
121 | def _render_root(renderer, root, options, extensions, width, state):
122 | if callable(renderer):
123 | return renderer(root, options, state)
124 | elif renderer == 'html':
125 | output = lib.cmark_render_html(root, options, extensions)
126 | elif renderer == 'xml':
127 | output = lib.cmark_render_xml(root, options)
128 | elif renderer == 'man':
129 | output = lib.cmark_render_man(root, options, width)
130 | elif renderer == 'commonmark':
131 | output = lib.cmark_render_commonmark(root, options, width)
132 | elif renderer == 'plaintext':
133 | output = lib.cmark_render_plaintext(root, options, width)
134 | elif renderer == 'latex':
135 | output = lib.cmark_render_latex(root, options, width)
136 | else:
137 | raise ValueError('Invalid "renderer" value')
138 |
139 | return ffi.string(output).decode('utf-8')
140 |
--------------------------------------------------------------------------------
/pedantmark/cmark_cdef.h:
--------------------------------------------------------------------------------
1 | // cmark-gfm.h
2 | typedef enum {
3 | CMARK_NODE_NONE = ...
4 | } cmark_node_type;
5 |
6 | typedef struct cmark_node cmark_node;
7 | typedef struct cmark_parser cmark_parser;
8 |
9 | typedef struct cmark_mem {
10 | void *(*calloc)(size_t, size_t);
11 | void *(*realloc)(void *, size_t);
12 | void (*free)(void *);
13 | } cmark_mem;
14 |
15 | typedef void (*cmark_free_func) (cmark_mem *mem, void *user_data);
16 |
17 | typedef struct _cmark_llist {
18 | struct _cmark_llist *next;
19 | void *data;
20 | } cmark_llist;
21 |
22 | cmark_llist * cmark_llist_append (cmark_mem * mem,
23 | cmark_llist * head,
24 | void * data);
25 |
26 | void cmark_llist_free_full (cmark_mem * mem,
27 | cmark_llist * head,
28 | cmark_free_func free_func);
29 |
30 | void cmark_llist_free (cmark_mem * mem,
31 | cmark_llist * head);
32 |
33 | const char *cmark_version_string();
34 |
35 | char *cmark_markdown_to_html(const char *text, size_t len, int options);
36 | cmark_node *cmark_parse_document(const char *buffer, size_t len, int options);
37 |
38 | char *cmark_render_xml(cmark_node *root, int options);
39 | char *cmark_render_html(cmark_node *root, int options, cmark_llist *extensions);
40 | char *cmark_render_man(cmark_node *root, int options, int width);
41 | char *cmark_render_commonmark(cmark_node *root, int options, int width);
42 | char *cmark_render_plaintext(cmark_node *root, int options, int width);
43 | char *cmark_render_latex(cmark_node *root, int options, int width);
44 |
45 | cmark_parser *cmark_parser_new(int options);
46 | void cmark_parser_free(cmark_parser *parser);
47 | void cmark_parser_feed(cmark_parser *parser, const char *buffer, size_t len);
48 | cmark_node *cmark_parser_finish(cmark_parser *parser);
49 |
50 | #define CMARK_OPT_DEFAULT 0
51 | #define CMARK_OPT_SOURCEPOS ...
52 | #define CMARK_OPT_HARDBREAKS ...
53 | #define CMARK_OPT_NOBREAKS ...
54 | #define CMARK_OPT_VALIDATE_UTF8 ...
55 | #define CMARK_OPT_SMART ...
56 | #define CMARK_OPT_GITHUB_PRE_LANG ...
57 | #define CMARK_OPT_LIBERAL_HTML_TAG ...
58 | #define CMARK_OPT_FOOTNOTES ...
59 | #define CMARK_OPT_STRIKETHROUGH_DOUBLE_TILDE ...
60 | #define CMARK_OPT_TABLE_PREFER_STYLE_ATTRIBUTES ...
61 | #define CMARK_OPT_FULL_INFO_STRING ...
62 | #define CMARK_OPT_UNSAFE ...
63 |
64 | typedef struct cmark_syntax_extension cmark_syntax_extension;
65 |
66 | // buffer.h
67 | typedef int32_t bufsize_t;
68 | typedef struct {
69 | cmark_mem *mem;
70 | unsigned char *ptr;
71 | bufsize_t asize, size;
72 | } cmark_strbuf;
73 | void cmark_strbuf_puts(cmark_strbuf *buf, const char *string);
74 |
75 | // cmark-gfm-extension_api.h
76 | cmark_syntax_extension *cmark_find_syntax_extension(const char *name);
77 | int cmark_parser_attach_syntax_extension(cmark_parser *parser, cmark_syntax_extension *extension);
78 | cmark_llist *cmark_parser_get_syntax_extensions(cmark_parser *parser);
79 |
80 |
81 | // extensions/cmark-gfm-core-extensions.h
82 | void cmark_gfm_core_extensions_ensure_registered(void);
83 |
--------------------------------------------------------------------------------
/pedantmark/extern.c:
--------------------------------------------------------------------------------
1 | #include
2 | #include
3 | #include
4 |
5 | #include "cmark-gfm.h"
6 | #include "node.h"
7 | #include "buffer.h"
8 | #include "houdini.h"
9 | #include "syntax_extension.h"
10 | #include "scanners.h"
11 | #include "cmark-gfm-core-extensions.h"
12 | #include "extern.h"
13 |
14 | unsigned char *escape_html(const unsigned char *source, size_t len, int secure) {
15 | cmark_mem *mem = cmark_get_default_mem_allocator();
16 | cmark_strbuf buf = CMARK_BUF_INIT(mem);
17 | houdini_escape_html0(&buf, source, len, secure);
18 | unsigned char *data = buf.ptr;
19 | cmark_strbuf_free(&buf);
20 | return data;
21 | }
22 |
23 | unsigned char *escape_href(const unsigned char *source, size_t len) {
24 | cmark_mem *mem = cmark_get_default_mem_allocator();
25 | cmark_strbuf buf = CMARK_BUF_INIT(mem);
26 | houdini_escape_href(&buf, source, len);
27 | unsigned char *data = buf.ptr;
28 | cmark_strbuf_free(&buf);
29 | return data;
30 | }
31 |
32 | const char *pedant_get_node_type(cmark_node *node) {
33 | if (node == NULL) {
34 | return "NONE";
35 | }
36 |
37 | if (node->extension && node->extension->get_type_string_func) {
38 | return node->extension->get_type_string_func(node->extension, node);
39 | }
40 |
41 | switch (node->type) {
42 | case CMARK_NODE_NONE:
43 | return "none";
44 | case CMARK_NODE_DOCUMENT:
45 | return "document";
46 | case CMARK_NODE_BLOCK_QUOTE:
47 | return "block_quote";
48 | case CMARK_NODE_LIST:
49 | return "list";
50 | case CMARK_NODE_ITEM:
51 | return "list_item";
52 | case CMARK_NODE_CODE_BLOCK:
53 | return "code_block";
54 | case CMARK_NODE_HTML_BLOCK:
55 | return "html_block";
56 | case CMARK_NODE_CUSTOM_BLOCK:
57 | return "custom_block";
58 | case CMARK_NODE_PARAGRAPH:
59 | return "paragraph";
60 | case CMARK_NODE_HEADING:
61 | return "heading";
62 | case CMARK_NODE_THEMATIC_BREAK:
63 | return "thematic_break";
64 | case CMARK_NODE_TEXT:
65 | return "text";
66 | case CMARK_NODE_SOFTBREAK:
67 | return "softbreak";
68 | case CMARK_NODE_LINEBREAK:
69 | return "linebreak";
70 | case CMARK_NODE_CODE:
71 | return "code";
72 | case CMARK_NODE_HTML_INLINE:
73 | return "html_inline";
74 | case CMARK_NODE_CUSTOM_INLINE:
75 | return "custom_inline";
76 | case CMARK_NODE_EMPH:
77 | return "emph";
78 | case CMARK_NODE_STRONG:
79 | return "strong";
80 | case CMARK_NODE_LINK:
81 | return "link";
82 | case CMARK_NODE_IMAGE:
83 | return "image";
84 | case CMARK_NODE_FOOTNOTE_REFERENCE:
85 | return "footnote_ref";
86 | case CMARK_NODE_FOOTNOTE_DEFINITION:
87 | return "footnote_def";
88 | }
89 |
90 | return "_unknown";
91 | }
92 |
93 | const char *pedant_get_node_code_info(cmark_node *node) {
94 | return (const char *)node->as.code.info.data;
95 | }
96 | const int pedant_get_node_heading_level(cmark_node *node) {
97 | return node->as.heading.level;
98 | }
99 | const bool pedant_get_node_list_bullet(cmark_node *node) {
100 | cmark_list_type list_type = node->as.list.list_type;
101 | return list_type == CMARK_BULLET_LIST;
102 | }
103 | const int pedant_get_node_list_start(cmark_node *node) {
104 | return node->as.list.start;
105 | }
106 | const char *pedant_get_node_link_url(cmark_node *node) {
107 | return (const char *)node->as.link.url.data;
108 | }
109 | const char *pedant_get_node_link_title(cmark_node *node) {
110 | return (const char *)escape_html(node->as.link.title.data, node->as.link.title.len, 0);
111 | }
112 | const bool pedant_get_node_paragraph_tight(cmark_node *node) {
113 | cmark_node *parent = cmark_node_parent(node);
114 | cmark_node *grandparent = cmark_node_parent(parent);
115 | if (grandparent != NULL && grandparent->type == CMARK_NODE_LIST) {
116 | return grandparent->as.list.tight;
117 | } else {
118 | return false;
119 | }
120 | }
121 | const char *pedant_get_node_table_cell_info(cmark_node *node) {
122 | static char buffer[3];
123 | bool is_head = cmark_gfm_extensions_get_table_row_is_header(node->parent);
124 | uint8_t *alignments = cmark_gfm_extensions_get_table_alignments(node->parent->parent);
125 |
126 | if (is_head) {
127 | buffer[0] = 'h';
128 | } else {
129 | buffer[0] = 'd';
130 | }
131 |
132 | cmark_node *n;
133 | int i = 0;
134 | for (n = node->parent->first_child; n; n = n->next, ++i)
135 | if (n == node)
136 | break;
137 |
138 | buffer[1] = alignments[i];
139 | buffer[2] = '\0';
140 | return buffer;
141 | }
142 |
143 | static int S_plain_node(cmark_strbuf *buf, cmark_node *node) {
144 | if (node->first_child) {
145 | cmark_node *next = node->first_child;
146 | while (next) {
147 | S_plain_node(buf, next);
148 | next = next->next;
149 | }
150 | } else {
151 | switch (node->type) {
152 | case CMARK_NODE_TEXT:
153 | case CMARK_NODE_CODE:
154 | case CMARK_NODE_HTML_INLINE:
155 | houdini_escape_html0(buf, node->as.literal.data, node->as.literal.len, 0);
156 | break;
157 | case CMARK_NODE_LINEBREAK:
158 | case CMARK_NODE_SOFTBREAK:
159 | cmark_strbuf_putc(buf, ' ');
160 | break;
161 | }
162 | return 1;
163 | }
164 | return 1;
165 | }
166 |
167 | static char *S_flat_node(pedant_render_node_t cb, cmark_node *node, int options,
168 | bool render_plain, void *userdata) {
169 |
170 | cmark_strbuf buf = CMARK_BUF_INIT(cmark_node_mem(node));
171 |
172 | if (render_plain) {
173 | S_plain_node(&buf, node);
174 | return (char *)cmark_strbuf_detach(&buf);
175 | }
176 |
177 | char *flat_s;
178 |
179 | switch (node->type) {
180 | case CMARK_NODE_CODE:
181 | case CMARK_NODE_TEXT:
182 | cb(&buf, node, escape_html(node->as.literal.data, node->as.literal.len, 0), userdata);
183 | break;
184 | case CMARK_NODE_CODE_BLOCK:
185 | cb(&buf, node, escape_html(node->as.code.literal.data, node->as.code.literal.len, 0), userdata);
186 | break;
187 | case CMARK_NODE_HTML_BLOCK:
188 | case CMARK_NODE_HTML_INLINE:
189 | if (options & CMARK_OPT_UNSAFE) {
190 | cb(&buf, node, node->as.literal.data, userdata);
191 | } else {
192 | cb(&buf, node, escape_html(node->as.literal.data, node->as.literal.len, 0), userdata);
193 | }
194 | break;
195 | case CMARK_NODE_THEMATIC_BREAK:
196 | case CMARK_NODE_LINEBREAK:
197 | case CMARK_NODE_SOFTBREAK:
198 | cb(&buf, node, (const unsigned char *)"", userdata);
199 | break;
200 | case CMARK_NODE_FOOTNOTE_REFERENCE:
201 | cb(&buf, node, node->as.literal.data, userdata);
202 | break;
203 | default:
204 | if (node->first_child) {
205 | bool next_plain = node->type == CMARK_NODE_IMAGE;
206 | cmark_node *next = node->first_child;
207 | while (next) {
208 | flat_s = S_flat_node(cb, next, options, next_plain, userdata);
209 | cmark_strbuf_puts(&buf, flat_s);
210 | free(flat_s);
211 | next = next->next;
212 | }
213 | const unsigned char *text = (const unsigned char *)cmark_strbuf_detach(&buf);
214 | cb(&buf, node, text, userdata);
215 | }
216 | }
217 |
218 | return (char *)cmark_strbuf_detach(&buf);
219 | }
220 |
221 | char *cmark_render_pedant(pedant_render_node_t cb, cmark_node *root,
222 | int options, void *userdata) {
223 | return S_flat_node(cb, root, options, 0, userdata);
224 | }
225 |
--------------------------------------------------------------------------------
/pedantmark/extern.h:
--------------------------------------------------------------------------------
1 | #include "buffer.h"
2 |
3 | unsigned char *escape_html(const unsigned char *source, size_t length, int secure);
4 | unsigned char *escape_href(const unsigned char *source, size_t len);
5 |
6 | typedef void (*pedant_render_node_t)(cmark_strbuf *buf, cmark_node *node, const unsigned char *text, void *userdata);
7 | char *cmark_render_pedant(pedant_render_node_t cb, cmark_node *root, int options, void *userdata);
8 |
9 | const char *pedant_get_node_type(cmark_node *node);
10 | const char *pedant_get_node_code_info(cmark_node *node);
11 | const int pedant_get_node_heading_level(cmark_node *node);
12 | const bool pedant_get_node_list_bullet(cmark_node *node);
13 | const int pedant_get_node_list_start(cmark_node *node);
14 | const char *pedant_get_node_link_url(cmark_node *node);
15 | const char *pedant_get_node_link_title(cmark_node *node);
16 | const bool pedant_get_node_paragraph_tight(cmark_node *node);
17 | const char *pedant_get_node_table_cell_info(cmark_node *node);
18 |
--------------------------------------------------------------------------------
/pedantmark/extern.py:
--------------------------------------------------------------------------------
1 | from ._cmark import lib, ffi
2 |
3 |
4 | def escape_html(text, secure=False):
5 | source = text.encode('utf-8')
6 | rv = lib.escape_html(source, len(source), int(secure))
7 | return _to_s(rv)
8 |
9 |
10 | def escape_href(url):
11 | source = url.encode('utf-8')
12 | rv = lib.escape_href(source, len(source))
13 | return _to_s(rv)
14 |
15 |
16 | class MarkdownState(object):
17 | def __init__(self, toc_level=0):
18 | self.footnotes = []
19 | self.toc = []
20 | self.toc_level = toc_level
21 |
22 |
23 | class HTMLRenderer(object):
24 | BREAK_TYPES = {'thematic_break', 'linebreak', 'softbreak'}
25 | TEXT_TYPES = {
26 | 'document', 'html_block', 'block_quote', 'list_item',
27 | 'table_row', 'table_header', 'table',
28 | 'text', 'emph', 'strong', 'strikethrough',
29 | 'code', 'html_inline', 'footnote_ref',
30 | }
31 | DANGEROUS_SCHEMES = (b'javascript:', b'vbscript:')
32 |
33 | def __init__(self, highlight=None, hardbreaks=False, nobreaks=False):
34 | self._highlight = highlight
35 | self._hardbreaks = hardbreaks
36 | self._nobreaks = nobreaks
37 |
38 | def _unknown(self):
39 | return b''
40 |
41 | def _is_dangerous_url(self, url):
42 | return url.startswith(self.DANGEROUS_SCHEMES)
43 |
44 | def __call__(self, root, options, state=None):
45 | if state is None:
46 | state = MarkdownState()
47 | userdata = ffi.new_handle((self, state))
48 | output = _to_b(lib.cmark_render_pedant(
49 | lib.pedant_render_node, root, options, userdata))
50 |
51 | if state.footnotes:
52 | text = b''.join(
53 | self.footnote_item(s, i+1)
54 | for i, s in enumerate(state.footnotes)
55 | )
56 | output += self.footnotes(text)
57 | return output.decode('utf-8')
58 |
59 | def none(self):
60 | return b''
61 |
62 | def custom_block(self, text):
63 | return b'TODO'
64 |
65 | def custom_inline(self, text):
66 | return b'TODO'
67 |
68 | def document(self, text):
69 | return text
70 |
71 | def thematic_break(self):
72 | """Rendering thematic break tag like ``
``."""
73 | return b'
\n'
74 |
75 | def html_block(self, text):
76 | """Rendering block level pure html content.
77 |
78 | :param text: text content of the html snippet.
79 | """
80 | return text
81 |
82 | def block_quote(self, text):
83 | """Rendering with the given text.
84 |
85 | :param text: text content of the blockquote.
86 | """
87 | return b'' + text + b'
\n'
88 |
89 | def code_block(self, text, lang):
90 | """Rendering block level code. ``pre > code``.
91 |
92 | :param code: text content of the code block.
93 | :param lang: language of the given code.
94 | """
95 | if self._highlight and lang:
96 | return self._highlight(text, lang)
97 | out = b'' + text + b'
\n'
101 |
102 | def heading(self, text, level, index=None):
103 | """Rendering heading tags like ```` ````.
104 |
105 | :param text: rendered text content for the header.
106 | :param level: a number for the header level, for example: 1.
107 | :param index: index of this heading in TOC (if enabled).
108 | """
109 | tag = b'h' + str(level).encode('utf-8')
110 | out = b'<' + tag
111 | if index:
112 | out += b' id="toc-' + str(index).encode('utf-8') + b'"'
113 | return out + b'>' + text + b'\n'
114 |
115 | def paragraph(self, text):
116 | """Rendering paragraph tags. Like ``
``."""
117 | return b'
' + text + b'
\n'
118 |
119 | def list_item(self, text):
120 | """Rendering list item snippet. Like ``
``."""
121 | return b' ' + text + b' \n'
122 |
123 | def list(self, text, ordered, start=None):
124 | """Rendering list tags like ```` and ````.
125 |
126 | :param body: body contents of the list.
127 | :param ordered: whether this list is ordered or not.
128 | :param start: start property for ordered list ````.
129 | """
130 | if not ordered:
131 | return b'\n' + text + b'
\n'
132 | out = b'\n' + text + b'
\n'
136 |
137 | def table_cell(self, text, tag, align=None):
138 | """Rendering a table cell. Like ```` `` ``.
139 |
140 | :param text: content of current table cell.
141 | :param tag: tag of ``th`` or ``td``.
142 | :param align: align of current table cell.
143 | """
144 | out = b'<' + tag
145 | if align:
146 | out += b' style="text-align:' + align + b'"'
147 | return out + b'>' + text + b'\n'
148 |
149 | def table_row(self, text):
150 | """Render a row of a table.
151 |
152 | :param text: content of the table row.
153 | """
154 | return b' \n' + text + b' \n'
155 |
156 | def table_header(self, text):
157 | """Render thead row of a table.
158 |
159 | :param text: content of the table row.
160 | """
161 | return b'\n' + text + b' \n'
162 |
163 | def table(self, text):
164 | """Wrapper of the table content."""
165 | return b'\n' + text + b'
\n'
166 |
167 | def softbreak(self):
168 | """Rendering soft linebreaks depending on options."""
169 | if self._hardbreaks:
170 | return self.linebreak()
171 | if self._nobreaks:
172 | return b' '
173 | return b'\n'
174 |
175 | def linebreak(self):
176 | """Rendering line break like ``
``."""
177 | return b'
\n'
178 |
179 | def html_inline(self, text):
180 | """Rendering inline level pure html content.
181 |
182 | :param text: text content of the html snippet.
183 | """
184 | return text
185 |
186 | def text(self, text):
187 | """Rendering plain text.
188 |
189 | :param text: text content.
190 | """
191 | return text
192 |
193 | def emph(self, text):
194 | """Rendering *emphasis* text.
195 |
196 | :param text: text content for emphasis.
197 | """
198 | return b'' + text + b''
199 |
200 | def strong(self, text):
201 | """Rendering **strong** text.
202 |
203 | :param text: text content for emphasis.
204 | """
205 | return b'' + text + b''
206 |
207 | def strikethrough(self, text):
208 | """Rendering ~~strikethrough~~ text.
209 |
210 | :param text: text content for strikethrough.
211 | """
212 | return b'' + text + b''
213 |
214 | def code(self, text):
215 | """Rendering inline `code` text.
216 |
217 | :param text: text content for inline code.
218 | """
219 | return b'' + text + b''
220 |
221 | def link(self, url, text, title):
222 | """Rendering a given link with content and title.
223 |
224 | :param link: href link for ```` tag.
225 | :param text: text content for description.
226 | :param title: title content for `title` attribute.
227 | """
228 | if self._is_dangerous_url(url):
229 | url = b''
230 | else:
231 | # autolink url ended with \n
232 | url = url.strip()
233 | out = b'' + text + b''
237 |
238 | def image(self, src, alt, title):
239 | """Rendering a image with title and text.
240 |
241 | :param src: source link of the image.
242 | :param alt: alt text of the image.
243 | :param title: title text of the image.
244 | """
245 | if self._is_dangerous_url(src):
246 | return b''
247 | out = b'
'
253 |
254 | def footnote_ref(self, key):
255 | """Rendering the ref anchor of a footnote.
256 |
257 | :param key: identity key for the footnote.
258 | """
259 | out = b''
261 | return out + key + b''
262 |
263 | def footnote_item(self, text, index):
264 | """Rendering a footnote item.
265 |
266 | :param text: text content of the footnote.
267 | :param index: index of the footnote item.
268 | """
269 | ref_ix = str(index).encode('utf-8')
270 | out = b'- \n'
271 | backref = b' ↩'
273 | i = text.rfind(b'\n'
276 | return out + text[:i] + backref + text[i:] + b'
- \n'
277 |
278 | def footnotes(self, text):
279 | """Wrapper for all footnotes.
280 |
281 | :param text: contents of all footnotes.
282 | """
283 | out = b'
\n'
284 | return out + text + b'
\n- ' + _render_item(text, 1)
7 | parents = [start_level]
8 |
9 | index = 2
10 | for text, level in toc[1:]:
11 | if level > parents[-1]:
12 | out += '\n
\n- '
13 | parents.append(level)
14 | elif level == parents[-1]:
15 | out += '
\n- '
16 | else:
17 | out = _outdent_item(out, parents, level)
18 | out += _render_item(text, index)
19 | index += 1
20 |
21 | return out + '
\n
\n'
22 |
23 |
24 | def _render_item(text, index):
25 | return '{}'.format(index, text.decode('utf-8'))
26 |
27 |
28 | def _outdent_item(out, parents, level):
29 | parents.pop()
30 | if not parents or level > parents[-1]:
31 | parents.append(level)
32 | return out + ' \n- '
33 | elif level == parents[-1]:
34 | return out + '
\n
\n \n- '
35 | else:
36 | return _outdent_item(out + '
\n
\n', parents, level)
37 |
--------------------------------------------------------------------------------
/setup.cfg:
--------------------------------------------------------------------------------
1 | [bdist_wheel]
2 | universal=0
3 |
--------------------------------------------------------------------------------
/setup.py:
--------------------------------------------------------------------------------
1 | #!/usr/bin/env python
2 | import re
3 | from setuptools import setup
4 |
5 | _version_re = re.compile(r"__version__ = '(.*)'")
6 |
7 |
8 | with open('pedantmark/__init__.py', 'r') as f:
9 | version = _version_re.search(f.read()).group(1)
10 |
11 |
12 | with open('README.rst') as f:
13 | long_description = f.read()
14 |
15 |
16 | setup(
17 | name='pedantmark',
18 | version=version,
19 | description='Python binding of GitHub cmark with extensions and renderers',
20 | long_description=long_description,
21 | url='https://github.com/lepture/pedantmark',
22 | zip_safe=False,
23 | license='BSD',
24 | packages=['pedantmark'],
25 | include_package_data=True,
26 | install_requires=["cffi>=1.11.0"],
27 | setup_requires=["cffi>=1.11.0"],
28 | cffi_modules=["build_ffi.py:ffi"],
29 | tests_require=['nose'],
30 | test_suite='nose.collector',
31 | classifiers=[
32 | 'Development Status :: 4 - Beta',
33 | 'Environment :: Web Environment',
34 | 'Intended Audience :: Developers',
35 | 'License :: OSI Approved :: BSD License',
36 | 'Operating System :: OS Independent',
37 | 'Programming Language :: Python :: 2.7',
38 | 'Programming Language :: Python :: 3',
39 | 'Programming Language :: Python :: 3.5',
40 | 'Programming Language :: Python :: 3.6',
41 | 'Programming Language :: Python :: 3.7',
42 | 'Programming Language :: Python :: Implementation',
43 | 'Programming Language :: Python :: Implementation :: CPython',
44 | 'Topic :: Text Processing :: Markup',
45 | 'Topic :: Software Development :: Libraries :: Python Modules',
46 | ]
47 | )
48 |
--------------------------------------------------------------------------------
/tests/__init__.py:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/lepture/pedantmark/8643c9c9a95a911266faa3c0246e6ad9d6fc51f3/tests/__init__.py
--------------------------------------------------------------------------------
/tests/bench.py:
--------------------------------------------------------------------------------
1 | # coding: utf-8
2 |
3 | import os
4 | import time
5 | import functools
6 |
7 |
8 | class benchmark(object):
9 | suites = []
10 |
11 | def __init__(self, name):
12 | self._name = name
13 |
14 | def __call__(self, func):
15 | @functools.wraps(func)
16 | def wrapper(text, loops=1000):
17 | start = time.clock()
18 | while loops:
19 | func(text)
20 | loops -= 1
21 | end = time.clock()
22 | return end - start
23 | # register
24 | benchmark.suites.append((self._name, wrapper))
25 | return wrapper
26 |
27 | @classmethod
28 | def bench(cls, text, loops=100):
29 | print('Parsing the CommonMark spec.txt %d times...' % loops)
30 | for name, func in cls.suites:
31 | try:
32 | total = func(text, loops=loops)
33 | print('{0}: {1}'.format(name, total))
34 | except ImportError:
35 | print('{0} is not available'.format(name))
36 |
37 |
38 | @benchmark('pedantmark.html')
39 | def benchmark_pedant_html(text):
40 | import pedantmark
41 | pedantmark.html(text)
42 |
43 |
44 | @benchmark('pedantmark.markdown')
45 | def benchmark_pedant_markdown(text):
46 | import pedantmark
47 | pedantmark.markdown(
48 | text,
49 | options=[pedantmark.OPT_FOOTNOTES],
50 | extensions=['autolink', 'strikethrough', 'table'],
51 | renderer='html',
52 | )
53 |
54 |
55 | @benchmark('pedantmark.custom')
56 | def benchmark_pedant_custom(text):
57 | import pedantmark
58 | pedantmark.markdown(
59 | text,
60 | options=[pedantmark.OPT_FOOTNOTES],
61 | extensions=['autolink', 'strikethrough', 'table'],
62 | renderer=pedantmark.HTMLRenderer(),
63 | )
64 |
65 |
66 | @benchmark('misaka')
67 | def benchmark_misaka(text):
68 | import misaka as m
69 | extensions = (
70 | m.EXT_NO_INTRA_EMPHASIS | m.EXT_FENCED_CODE | m.EXT_AUTOLINK |
71 | m.EXT_TABLES | m.EXT_STRIKETHROUGH
72 | )
73 | md = m.Markdown(m.HtmlRenderer(), extensions=extensions)
74 | md(text)
75 |
76 |
77 | @benchmark('mistune')
78 | def benchmark_mistune(text):
79 | import mistune
80 | mistune.markdown(text)
81 |
82 |
83 | @benchmark('mistletoe')
84 | def benchmark_mistletoe(text):
85 | import mistletoe
86 | mistletoe.markdown(text)
87 |
88 |
89 | @benchmark('commonmark')
90 | def benchmark_commonmark(text):
91 | import commonmark
92 | commonmark.commonmark(text)
93 |
94 |
95 | @benchmark('markdown')
96 | def benchmark_markdown(text):
97 | import markdown
98 | markdown.markdown(text, extensions=['extra'])
99 |
100 |
101 | if __name__ == '__main__':
102 | root = os.path.dirname(__file__)
103 | filepath = os.path.join(root, 'spec.txt')
104 | with open(filepath, 'r') as f:
105 | text = f.read()
106 |
107 | benchmark.bench(text)
108 |
--------------------------------------------------------------------------------
/tests/test_cmark.py:
--------------------------------------------------------------------------------
1 | import pedantmark
2 |
3 |
4 | TEXT = '''
5 | # h
6 |
7 | a `b` ~c~
8 |
9 |
10 | '''
11 |
12 |
13 | def test_default_html():
14 | html = pedantmark.html(TEXT)
15 | assert 'h
' in html
16 | assert 'b' in html
17 | assert '~c~' in html
18 | assert 'omitted' in html
19 |
20 | html = pedantmark.html(TEXT, [pedantmark.OPT_UNSAFE])
21 | assert '' in html
22 |
23 |
24 | def test_markdown_html():
25 | html = pedantmark.markdown(TEXT, renderer='html')
26 | assert html == pedantmark.html(TEXT)
27 |
28 | html = pedantmark.markdown(
29 | TEXT, renderer='html',
30 | extensions=['strikethrough']
31 | )
32 | assert 'c' in html
33 |
34 | html = pedantmark.markdown(
35 | TEXT, renderer='html',
36 | options=[pedantmark.OPT_STRIKETHROUGH_DOUBLE_TILDE],
37 | extensions=['strikethrough']
38 | )
39 | assert '~c~' in html
40 |
41 |
42 | def test_markdown_man():
43 | rv = pedantmark.markdown(TEXT, renderer='man')
44 | assert r'\f[C]b\f[]' in rv
45 |
46 |
47 | def test_markdown_latex():
48 | rv = pedantmark.markdown(TEXT, renderer='latex')
49 | assert r'\section{h}' in rv
50 |
51 |
52 | def test_markdown_plaintext():
53 | rv = pedantmark.markdown(TEXT, renderer='plaintext')
54 | assert r'a b ~c~' in rv
55 |
56 |
57 | def test_markdown_commonmark():
58 | rv = pedantmark.markdown(TEXT, renderer='commonmark')
59 | assert r'# h' in rv
60 |
61 |
62 | def test_markdown_xml():
63 | rv = pedantmark.markdown(TEXT, renderer='xml')
64 | assert '' in rv
65 |
66 |
67 | def test_markdown_invalid():
68 | try:
69 | pedantmark.markdown(TEXT, renderer='invalid')
70 | success = True
71 | except ValueError:
72 | success = False
73 | assert success is False
74 |
75 |
76 | def test_invalid_extension():
77 | try:
78 | pedantmark.markdown(TEXT, extensions=['invalid'])
79 | success = True
80 | except ValueError:
81 | success = False
82 | assert success is False
83 |
--------------------------------------------------------------------------------
/tests/test_escape.py:
--------------------------------------------------------------------------------
1 | from pedantmark import escape_html, escape_href
2 |
3 |
4 | def test_escape_html():
5 | assert escape_html('') == '<a>'
6 |
7 |
8 | def test_escape_href():
9 | assert escape_href('http://a b') == 'http://a%20b'
10 |
--------------------------------------------------------------------------------
/tests/test_renderer.py:
--------------------------------------------------------------------------------
1 | import pedantmark
2 | from pedantmark import markdown, HTMLRenderer
3 |
4 |
5 | def test_none():
6 | rv = markdown('\n', renderer=HTMLRenderer())
7 | assert rv == ''
8 |
9 |
10 | def test_strong():
11 | rv = markdown('**a**', renderer=HTMLRenderer())
12 | assert rv.strip() == 'a
'
13 |
14 |
15 | def test_emph():
16 | rv = markdown('_a_', renderer=HTMLRenderer())
17 | assert rv.strip() == 'a
'
18 |
19 |
20 | def test_strikethrough():
21 | rv = markdown('~a~', renderer=HTMLRenderer(), extensions=['strikethrough'])
22 | assert rv.strip() == 'a
'
23 |
24 |
25 | def test_code():
26 | rv = markdown('`a`', renderer=HTMLRenderer())
27 | assert rv.strip() == 'a
'
28 |
29 |
30 | def test_link():
31 | rv = markdown('[a]()', renderer=HTMLRenderer())
32 | assert rv.strip() == ''
33 |
34 | rv = markdown('[a]( "b")', renderer=HTMLRenderer())
35 | assert rv.strip() == ''
36 |
37 | rv = markdown('[a]( "b")', renderer=HTMLRenderer())
38 | assert rv.strip() == ''
39 |
40 |
41 | def test_image():
42 | rv = markdown('![a]()', renderer=HTMLRenderer())
43 | assert rv.strip() == '
'
44 |
45 | rv = markdown('![**a**]( "b")', renderer=HTMLRenderer())
46 | assert rv.strip() == '
'
47 |
48 | rv = markdown('![a]( "b")', renderer=HTMLRenderer())
49 | assert rv.strip() == ''
50 |
51 |
52 | def test_html_inline():
53 | rv = markdown('a i', renderer=HTMLRenderer())
54 | assert '<i>' in rv
55 |
56 | rv = markdown(
57 | 'a i',
58 | options=[pedantmark.OPT_UNSAFE],
59 | renderer=HTMLRenderer()
60 | )
61 | assert 'i' in rv
62 |
63 |
64 | def test_html_block():
65 | rv = markdown('\na\n', renderer=HTMLRenderer())
66 | assert '<div>' in rv
67 |
68 | rv = markdown(
69 | '\na\n',
70 | options=[pedantmark.OPT_UNSAFE],
71 | renderer=HTMLRenderer()
72 | )
73 | assert '' in rv
74 |
75 |
76 | def test_thematic_break():
77 | rv = markdown('***', renderer=HTMLRenderer())
78 | assert rv.strip() == '
'
79 |
80 |
81 | def test_linebreak():
82 | rv = markdown('a \nb\n', renderer=HTMLRenderer())
83 | assert rv.strip() == 'a
\nb
'
84 |
85 |
86 | def test_softbreak():
87 | rv = markdown('a\nb\n', renderer=HTMLRenderer())
88 | assert 'a\nb' in rv
89 |
90 | rv = markdown('a\nb\n', renderer=HTMLRenderer(hardbreaks=True))
91 | assert '
' in rv
92 |
93 | rv = markdown('a\nb\n', renderer=HTMLRenderer(nobreaks=True))
94 | assert 'a b' in rv
95 |
96 |
97 | def test_block_quote():
98 | rv = markdown('> a', renderer=HTMLRenderer())
99 | assert rv.strip() == 'a
\n
'
100 |
101 |
102 | def test_code_block():
103 | rv = markdown(' a\n', renderer=HTMLRenderer())
104 | assert rv.strip() == 'a\n
'
105 |
106 | rv = markdown('```c\na\n```', renderer=HTMLRenderer())
107 | assert rv.strip() == 'a\n
'
108 |
109 | def highlight(code, lang):
110 | return b'test'
111 |
112 | rv = markdown('```c\na\n```', renderer=HTMLRenderer(highlight))
113 | assert rv == 'test'
114 |
115 |
116 | def test_ordered_list():
117 | rv = markdown('1. a', renderer=HTMLRenderer())
118 | assert rv.strip() == '\n- a
\n
'
119 |
120 | rv = markdown('5. a', renderer=HTMLRenderer())
121 | assert rv.strip() == '\n- a
\n
'
122 |
123 |
124 | def test_unordered_list():
125 | rv = markdown('- a', renderer=HTMLRenderer())
126 | assert rv.strip() == '\n- a
\n
'
127 |
128 |
129 | def test_footnote():
130 | s = '[^1]\n\n[^1]: a'
131 | rv = markdown(
132 | s, renderer=HTMLRenderer(),
133 | options=[pedantmark.OPT_FOOTNOTES]
134 | )
135 | assert 'id="fn' in rv
136 | assert 'class="footnotes"' in rv
137 |
138 |
139 | TABLE_TEXT_1 = '''
140 | a1 | a2
141 | -- | --
142 | b1 | b2
143 | '''
144 |
145 | TABLE_TEXT_2 = '''
146 | a1 | a2
147 | :-- | :--:
148 | b1 | b2
149 | '''
150 |
151 |
152 | def test_table():
153 | rv = markdown(
154 | TABLE_TEXT_1, renderer=HTMLRenderer(),
155 | extensions=['table'],
156 | )
157 | assert '' in rv
158 |
159 | rv = markdown(
160 | TABLE_TEXT_2, renderer=HTMLRenderer(),
161 | extensions=['table'],
162 | )
163 | assert 'style=' in rv
164 |
--------------------------------------------------------------------------------
/tests/test_toc.py:
--------------------------------------------------------------------------------
1 | from pedantmark import MarkdownState, HTMLRenderer, markdown
2 | from pedantmark.toc import render_toc
3 |
4 | NORMAL_INDENT = '''
5 | # a
6 |
7 | ## b
8 |
9 | ### c
10 |
11 | #### d
12 |
13 | # e
14 | '''
15 |
16 | NORMAL_HTML = '''
17 |
29 | '''
30 |
31 | MESS_INDENT = '''
32 | ## a
33 |
34 | ###### b
35 |
36 | ### c
37 |
38 | #### d
39 |
40 | #### e
41 |
42 | # f
43 | '''
44 |
45 | MESS_HTML = '''
46 |
60 | '''
61 |
62 |
63 | def test_normal_indent():
64 | state = MarkdownState(toc_level=3)
65 | html = markdown(NORMAL_INDENT, renderer=HTMLRenderer(), state=state)
66 | # h4 has no id
67 | assert '' in html
68 | toc = render_toc(state.toc)
69 | assert toc.strip() == NORMAL_HTML.strip()
70 |
71 |
72 | def test_mess_indent():
73 | state = MarkdownState(toc_level=6)
74 | markdown(MESS_INDENT, renderer=HTMLRenderer(), state=state)
75 | toc = render_toc(state.toc)
76 | assert toc.strip() == MESS_HTML.strip()
77 |
78 |
79 | def test_none():
80 | assert render_toc([]) == ''
81 |
--------------------------------------------------------------------------------
/vendors/.gitkeep:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/lepture/pedantmark/8643c9c9a95a911266faa3c0246e6ad9d6fc51f3/vendors/.gitkeep
--------------------------------------------------------------------------------
/vendors/unix/extensions/cmark-gfm-extensions_export.h:
--------------------------------------------------------------------------------
1 |
2 | #ifndef CMARK_GFM_EXTENSIONS_EXPORT_H
3 | #define CMARK_GFM_EXTENSIONS_EXPORT_H
4 |
5 | #ifdef CMARK_GFM_EXTENSIONS_STATIC_DEFINE
6 | # define CMARK_GFM_EXTENSIONS_EXPORT
7 | # define CMARK_GFM_EXTENSIONS_NO_EXPORT
8 | #else
9 | # ifndef CMARK_GFM_EXTENSIONS_EXPORT
10 | # ifdef libcmark_gfm_extensions_EXPORTS
11 | /* We are building this library */
12 | # define CMARK_GFM_EXTENSIONS_EXPORT __attribute__((visibility("default")))
13 | # else
14 | /* We are using this library */
15 | # define CMARK_GFM_EXTENSIONS_EXPORT __attribute__((visibility("default")))
16 | # endif
17 | # endif
18 |
19 | # ifndef CMARK_GFM_EXTENSIONS_NO_EXPORT
20 | # define CMARK_GFM_EXTENSIONS_NO_EXPORT __attribute__((visibility("hidden")))
21 | # endif
22 | #endif
23 |
24 | #ifndef CMARK_GFM_EXTENSIONS_DEPRECATED
25 | # define CMARK_GFM_EXTENSIONS_DEPRECATED __attribute__ ((__deprecated__))
26 | #endif
27 |
28 | #ifndef CMARK_GFM_EXTENSIONS_DEPRECATED_EXPORT
29 | # define CMARK_GFM_EXTENSIONS_DEPRECATED_EXPORT CMARK_GFM_EXTENSIONS_EXPORT CMARK_GFM_EXTENSIONS_DEPRECATED
30 | #endif
31 |
32 | #ifndef CMARK_GFM_EXTENSIONS_DEPRECATED_NO_EXPORT
33 | # define CMARK_GFM_EXTENSIONS_DEPRECATED_NO_EXPORT CMARK_GFM_EXTENSIONS_NO_EXPORT CMARK_GFM_EXTENSIONS_DEPRECATED
34 | #endif
35 |
36 | #if 0 /* DEFINE_NO_DEPRECATED */
37 | # ifndef CMARK_GFM_EXTENSIONS_NO_DEPRECATED
38 | # define CMARK_GFM_EXTENSIONS_NO_DEPRECATED
39 | # endif
40 | #endif
41 |
42 | #endif /* CMARK_GFM_EXTENSIONS_EXPORT_H */
43 |
--------------------------------------------------------------------------------
/vendors/unix/src/cmark-gfm_export.h:
--------------------------------------------------------------------------------
1 |
2 | #ifndef CMARK_GFM_EXPORT_H
3 | #define CMARK_GFM_EXPORT_H
4 |
5 | #ifdef CMARK_GFM_STATIC_DEFINE
6 | # define CMARK_GFM_EXPORT
7 | # define CMARK_GFM_NO_EXPORT
8 | #else
9 | # ifndef CMARK_GFM_EXPORT
10 | # ifdef libcmark_gfm_EXPORTS
11 | /* We are building this library */
12 | # define CMARK_GFM_EXPORT __attribute__((visibility("default")))
13 | # else
14 | /* We are using this library */
15 | # define CMARK_GFM_EXPORT __attribute__((visibility("default")))
16 | # endif
17 | # endif
18 |
19 | # ifndef CMARK_GFM_NO_EXPORT
20 | # define CMARK_GFM_NO_EXPORT __attribute__((visibility("hidden")))
21 | # endif
22 | #endif
23 |
24 | #ifndef CMARK_GFM_DEPRECATED
25 | # define CMARK_GFM_DEPRECATED __attribute__ ((__deprecated__))
26 | #endif
27 |
28 | #ifndef CMARK_GFM_DEPRECATED_EXPORT
29 | # define CMARK_GFM_DEPRECATED_EXPORT CMARK_GFM_EXPORT CMARK_GFM_DEPRECATED
30 | #endif
31 |
32 | #ifndef CMARK_GFM_DEPRECATED_NO_EXPORT
33 | # define CMARK_GFM_DEPRECATED_NO_EXPORT CMARK_GFM_NO_EXPORT CMARK_GFM_DEPRECATED
34 | #endif
35 |
36 | #if 0 /* DEFINE_NO_DEPRECATED */
37 | # ifndef CMARK_GFM_NO_DEPRECATED
38 | # define CMARK_GFM_NO_DEPRECATED
39 | # endif
40 | #endif
41 |
42 | #endif /* CMARK_GFM_EXPORT_H */
43 |
--------------------------------------------------------------------------------
/vendors/unix/src/cmark-gfm_version.h:
--------------------------------------------------------------------------------
1 | #ifndef CMARK_GFM_VERSION_H
2 | #define CMARK_GFM_VERSION_H
3 |
4 | #define CMARK_GFM_VERSION ((0 << 24) | (28 << 16) | (3 << 8) | 19)
5 | #define CMARK_GFM_VERSION_STRING "0.28.3.gfm.19"
6 |
7 | #endif
8 |
--------------------------------------------------------------------------------
/vendors/unix/src/config.h:
--------------------------------------------------------------------------------
1 | #ifndef CMARK_CONFIG_H
2 | #define CMARK_CONFIG_H
3 |
4 | #ifdef __cplusplus
5 | extern "C" {
6 | #endif
7 |
8 | #define HAVE_STDBOOL_H
9 |
10 | #ifdef HAVE_STDBOOL_H
11 | #include
12 | #elif !defined(__cplusplus)
13 | typedef char bool;
14 | #endif
15 |
16 | #define HAVE___BUILTIN_EXPECT
17 |
18 | #define HAVE___ATTRIBUTE__
19 |
20 | #ifdef HAVE___ATTRIBUTE__
21 | #define CMARK_ATTRIBUTE(list) __attribute__ (list)
22 | #else
23 | #define CMARK_ATTRIBUTE(list)
24 | #endif
25 |
26 | #ifndef CMARK_INLINE
27 | #if defined(_MSC_VER) && !defined(__cplusplus)
28 | #define CMARK_INLINE __inline
29 | #else
30 | #define CMARK_INLINE inline
31 | #endif
32 | #endif
33 |
34 | /* snprintf and vsnprintf fallbacks for MSVC before 2015,
35 | due to Valentin Milea http://stackoverflow.com/questions/2915672/
36 | */
37 |
38 | #if defined(_MSC_VER) && _MSC_VER < 1900
39 |
40 | #include
41 | #include
42 |
43 | #define snprintf c99_snprintf
44 | #define vsnprintf c99_vsnprintf
45 |
46 | CMARK_INLINE int c99_vsnprintf(char *outBuf, size_t size, const char *format, va_list ap)
47 | {
48 | int count = -1;
49 |
50 | if (size != 0)
51 | count = _vsnprintf_s(outBuf, size, _TRUNCATE, format, ap);
52 | if (count == -1)
53 | count = _vscprintf(format, ap);
54 |
55 | return count;
56 | }
57 |
58 | CMARK_INLINE int c99_snprintf(char *outBuf, size_t size, const char *format, ...)
59 | {
60 | int count;
61 | va_list ap;
62 |
63 | va_start(ap, format);
64 | count = c99_vsnprintf(outBuf, size, format, ap);
65 | va_end(ap);
66 |
67 | return count;
68 | }
69 |
70 | #endif
71 |
72 | #ifdef __cplusplus
73 | }
74 | #endif
75 |
76 | #endif
77 |
--------------------------------------------------------------------------------