├── .github
    └── FUNDING.yml
├── .gitignore
├── CHANGELOG.md
├── LICENSE
├── README.md
├── docs
    ├── CNAME
    ├── _config.yml
    ├── index.md
    └── pypi.md
├── hypertag
├── pyproject.toml
├── setup.cfg
├── src
    └── hypertag
    │   ├── __init__.py
    │   ├── builtins.py
    │   ├── core
    │       ├── README_draft.md
    │       ├── __init__.py
    │       ├── ast.py
    │       ├── dom.py
    │       ├── errors.py
    │       ├── grammar.py
    │       ├── run_html.py
    │       ├── runtime.py
    │       ├── structs.py
    │       ├── tag.py
    │       └── xml_chars.py
    │   ├── django
    │       ├── __init__.py
    │       ├── backend.py
    │       └── filters.py
    │   ├── html.py
    │   ├── nifty
    │       ├── __init__.py
    │       ├── parsing.py
    │       └── util.py
    │   ├── std
    │       ├── __init__.py
    │       ├── builtins.py
    │       ├── html.py
    │       └── registry.py
    │   └── tests
    │       ├── __init__.py
    │       ├── package1
    │           ├── __init__.py
    │           └── sample3.hy
    │       ├── sample1.py
    │       ├── sample2.hy
    │       ├── sample3.hy
    │       └── sample4.hy
├── tests
    ├── __init__.py
    ├── sample_usage.html
    ├── sample_usage.py
    ├── sandbox.py
    ├── sandbox_py2.py
    └── test_main.py
└── tox.ini


/.github/FUNDING.yml:
--------------------------------------------------------------------------------
 1 | # These are supported funding model platforms
 2 | 
 3 | github: mwojnars
 4 | patreon: # Replace with a single Patreon username
 5 | open_collective: # Replace with a single Open Collective username
 6 | ko_fi: # Replace with a single Ko-fi username
 7 | tidelift: # Replace with a single Tidelift platform-name/package-name e.g., npm/babel
 8 | community_bridge: # Replace with a single Community Bridge project-name e.g., cloud-foundry
 9 | liberapay: # Replace with a single Liberapay username
10 | issuehunt: # Replace with a single IssueHunt username
11 | otechie: # Replace with a single Otechie username
12 | custom: # Replace with up to 4 custom sponsorship URLs e.g., ['link1', 'link2']
13 | 


--------------------------------------------------------------------------------
/.gitignore:
--------------------------------------------------------------------------------
 1 | *.pyc
 2 | *.py[cod]
 3 | *.kate-swp
 4 | *.npy
 5 | 
 6 | # Drafts
 7 | *.draft.*
 8 | 
 9 | # C extensions
10 | *.so
11 | 
12 | # Packages
13 | *.egg
14 | *.egg-info
15 | dist
16 | build
17 | eggs
18 | parts
19 | bin
20 | var
21 | sdist
22 | develop-eggs
23 | .installed.cfg
24 | lib
25 | lib64
26 | __pycache__
27 | 
28 | # Installer logs
29 | pip-log.txt
30 | 
31 | # Unit test / coverage reports
32 | .coverage
33 | .tox
34 | nosetests.xml
35 | .pytest_cache
36 | 
37 | # Translations
38 | *.mo
39 | 
40 | .project
41 | .pydevproject
42 | 


--------------------------------------------------------------------------------
/CHANGELOG.md:
--------------------------------------------------------------------------------
 1 | # Change Log
 2 | 
 3 | ## *upcoming*
 4 | 
 5 | - Grammar fix: easier use of dictionaries in assignment blocks, no conflict with embedding syntax.
 6 | - ...
 7 | 
 8 | ## [1.2.0] - 2021-09-16
 9 | 
10 | - Non-standard operator `<>` can be used as an alternative for `!=` 
11 |   to avoid ambiguity with a verbatim block mark (`!`) inside `if` blocks.
12 | - Shortcut attributes `.CLS` and `#ID` no longer need spaces around, 
13 |   so `div#main.left.top` is now a correct tag occurrence.
14 | - The *if-else* operator returns "" (empty string) rather than None
15 |   when the *else* branch is missing.
16 | - Grammar extended to support *expression blocks* (`$ EXPR`) without an assignment. 
17 |   Can be used for method calls, like in `$myList.append(3)`.
18 | - Fixed html-escaping of special symbols inside HTML attribute values.
19 | - Built-in tag/function `dedent` accepts now an optional argument `full`.
20 | - Built-in tag/function `inline` renamed to `merge`.
21 | - New built-in tag and function, `inline`, converts all newline characters
22 |   to HTML entities `&#10;` for safe inclusion in indentation-sensitive blocks
23 |   like `<pre>` or `<textarea>`.
24 | - New built-in functions: `$crop()`, alias `$truncate()`.
25 | - New built-in tag, `doctype`, to produce a DOCTYPE declaration at the beginning of a page.
26 | - New built-in tag, `custom "NAME"` in HyperHTML runtime. It outputs 
27 |   non-standard HTML tag names, so that HTML Custom Elements (Web Components)
28 |   can be used inside target documents.
29 | 
30 | ## [1.1.4] - 2021-07-20
31 | 
32 | - Bug fix: exception raised in a "try" block caused "dedent" operations to be skipped,
33 |   leading to incorrect calculation of indentations of subsequent blocks.
34 |   Changed to always revert to an initial indentation when translating a body node.
35 | - Bug fix: post-translation state of an imported module included only output symbols.
36 |   Changed to include all symbols (slots), since they can also be needed 
37 |   for hypertag expansion.
38 | 
39 | ## [1.1.3] - 2021-04-10
40 | 
41 | - Syntax extended: loops may have an `else` clause.
42 | - Syntax extended: tags can be referenced inside expressions using the _tag embedding_ operator `%`.
43 | 
44 | ## [1.1.2], [1.1.0] - 2021-04-02
45 | 
46 | - Added Django integration: a Hypertag backend (`hypertag.django.backend.Hypertag`) that can be plugged into a Django application.
47 | - Changed syntax for importing from dynamic context: added "context" blocks in place of "from ~ import ..." syntax.
48 | - Fixed and improved the import mechanism, classes: `Runtime`, `Loader`, `Module`.
49 | 
50 | ## [1.0.3] - 2021-03-23
51 | 
52 | ## [1.0.2] - 2021-03-22
53 | 
54 | ## [1.0.1] - 2021-03-22
55 | 
56 | Initial release.
57 | 


--------------------------------------------------------------------------------
/LICENSE:
--------------------------------------------------------------------------------
 1 | MIT License
 2 | 
 3 | Copyright (c) 2021 Marcin Wojnarski
 4 | 
 5 | Permission is hereby granted, free of charge, to any person obtaining a copy
 6 | of this software and associated documentation files (the "Software"), to deal
 7 | in the Software without restriction, including without limitation the rights
 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
 9 | copies of the Software, and to permit persons to whom the Software is
10 | furnished to do so, subject to the following conditions:
11 | 
12 | The above copyright notice and this permission notice shall be included in all
13 | copies or substantial portions of the Software.
14 | 
15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21 | SOFTWARE.
22 | 


--------------------------------------------------------------------------------
/README.md:
--------------------------------------------------------------------------------
  1 | <!---
  2 | <style type="text/css" rel="stylesheet">
  3 |   div.highlight {
  4 |     max-height: 200px;
  5 |     overflow-y: auto;
  6 |   }
  7 | </style>
  8 | --->
  9 | 
 10 | ## Introduction
 11 | 
 12 | Hypertag is a modern language for front-end development that allows
 13 | writing markup documents in a way similar to writing Python scripts,
 14 | where _indentation_ determines relationships between nested elements 
 15 | and removes the need for explicit closing tags.
 16 | Hypertag provides advanced control of page rendering with native control blocks;
 17 | high level of modularity thanks to Python-like imports and DOM manipulation; 
 18 | unprecedented support for code reuse with native custom tags (_hypertags_), 
 19 | and more. Compatible with HTML 5. See the [Quick Start](#quick-start) below, or the 
 20 | [Language Reference](http://hypertag.io) for details.
 21 | 
 22 | Authored by [Marcin Wojnarski](http://www.linkedin.com/in/marcinwojnarski)
 23 | from [Paperity](https://paperity.org/).
 24 | 
 25 | <!---
 26 | The concept of the language stems from many years of experiences 
 27 | in the development of academic web services: [Paperity](https://paperity.org/) and others.
 28 | --->
 29 | 
 30 | <!---
 31 | ### Why to use Hypertag
 32 | 
 33 | - **Concise syntax**: inspired by Python, the indentation-based syntax is a lot cleaner, 
 34 |   more readable and maintainable than raw markup; it requires less typing, is less redundant,
 35 |   and lets you concentrate on coding rather than chasing unmatched opening-closing tags.
 36 | - **Code reuse** by means of functions/classes is the corner stone of programming,
 37 |   yet it is missing from HTML; this is fixed now with Hypertag:
 38 |   programmers can create reusable components in a form of **custom tags** (_hypertags_), 
 39 |   defined either as Python functions (_external tags_) 
 40 |   or directly in a document using Hypertag syntax (_native tags_);
 41 |   hypertags can be parameterized and may represent complex pieces 
 42 |   of combined: content, style and layout - for a reuse across documents.
 43 | - **Fine-grained control** over rendering process is possible with
 44 |   a range of native _control blocks_ (for, while, if-elif-else, try-else) 
 45 |   constituting a core part of Hypertag syntax, unlike in templating languages, 
 46 |   where control structures are artificially overlaid on top of another language (HTML).
 47 | - **Modularity** in Hypertag is modeled after Python's: 
 48 |   every script may import tags and variables from other scripts,
 49 |   from Python modules, and from the dynamic _context_ of script rendering;
 50 |   scripts and modules are arranged into packages;
 51 |   with these mechanisms in place, building libraries of reusable components is easy and fun.
 52 | - **Applicability** to different target languages. 
 53 |   Hypertag is _not_ just a templating system put on top of HTML. 
 54 |   Hypertag is a full-featured standalone programming language tailored to the generation
 55 |   of documents of all kinds. By defining new tags, Hypertag can be adapted to produce an arbitrary
 56 |   document description language.
 57 | --->
 58 |   
 59 | <!---
 60 | - **Code reuse** by means of functions/classes is the corner stone of programming,
 61 |   yet it is missing from HTML; this is fixed now with Hypertag:
 62 |   programmers can create reusable components in a form of **custom tags** (_hypertags_), 
 63 |   defined either as Python functions (_external tags_) 
 64 |   or directly in a document using Hypertag syntax (_native tags_);
 65 |   hypertags can be parameterized and may represent complex pieces 
 66 |   of combined: content, style and layout - for a reuse across documents.
 67 | - **DOM manipulation**:
 68 | 
 69 |   Hypertag is *not* limited to (X)HTML; by defining new tags,
 70 |   it can be adapted to produce an arbitrary document description language.
 71 |   HTML templating is one of applications, but Hypertag's capabilities are much bigger than that.
 72 |   whose one of use cases is being a replacement for web templating languages.
 73 | - **consistency** (monolithic architecture): Hypertag combines presentation and logic in one language; 
 74 |   you no longer have to mix presentation code (HTML) with foreign syntax of a 
 75 |   templating language, or PHP etc.
 76 | - **Object-Oriented Programming** (OOP) inside markup, through native language structures (??)
 77 | - **high performace** in web applications achieved through caching of parsed AST,
 78 |   combined with their **compactification**: constant parts of the AST are
 79 |   pre-rendered and merged into single nodes, to avoid repeated rendering
 80 |   with every web page request.
 81 | 
 82 | If you try Hypertag, you will never miss old-school HTML templating.
 83 | --->
 84 | 
 85 | ## Quick Start
 86 | 
 87 | Install in Python 3:
 88 | ```
 89 | pip install hypertag-lang               # watch out the name, it is "hypertag-lang"
 90 | ```
 91 | 
 92 | Usage:
 93 | ```python
 94 | from hypertag import HyperHTML
 95 | html = HyperHTML().render(script)       # rendering of a Hypertag `script` to HTML
 96 | ```
 97 | 
 98 | <!---
 99 | Run in Python 3 (example):
100 | ```python
101 | from hypertag import HyperHTML
102 | 
103 | script = \
104 | """
105 |     context $blue
106 |     html: body:
107 |         h1 style="color: $blue"
108 |             / Example document with a <u>list of items</u>
109 |         ul
110 |             li | item <1>
111 |             li | item no. {1+1}
112 |         p : a href='http://hypertag.io' | See the Hypertag site for more!
113 | """
114 | 
115 | html = HyperHTML().render(script, blue = '#00f')
116 | print(html)
117 | ```
118 | 
119 | The `script` in the code above is rendered to `html` as shown below
120 | (see a [preview](http://htmlpreview.github.io/?https://github.com/mwojnars/hypertag/blob/main/test/sample_usage.html)):
121 | 
122 | ```html
123 | <html><body>
124 |     <h1 style="color: #00f">
125 |         Example document with a <u>list of items</u>
126 |     </h1>
127 |     <ul>
128 |         <li>item &lt;1&gt;</li>
129 |         <li>item no. 2</li>
130 |     </ul>
131 |     <p><a href="http://hypertag.io">See the Hypertag site for more!</a></p>
132 | </body></html>
133 | ```
134 | --->
135 | 
136 | ### Blocks
137 | 
138 | A typical Hypertag script consists of nested blocks with tags:
139 | 
140 |     ul
141 |         li 
142 |             | This is the first item of a "ul" list.
143 |             | Pipe (|) marks a plain-text block. HTML is auto-escaped: & < >
144 |         li
145 |             / This is the second item. 
146 |               Slash (/) marks a <b>markup block</b> (no HTML escaping).
147 |               Text blocks may consist of multiple lines, like here.
148 | 
149 | Here, `ul` is a "tag", which renders to its HTML equivalent `<ul>...</ul>`. Indentation of blocks gets preserved in the output during rendering:
150 | 
151 | ```html
152 | <ul>
153 |     <li>
154 |         This is the first item of a "ul" list.
155 |         Pipe (|) marks a plain-text block. HTML is auto-escaped: &amp; &lt; &gt;
156 |     </li>
157 |     <li>
158 |         This is the second item.
159 |         Slash (/) marks a <b>markup block</b> (no HTML escaping).
160 |         Text blocks may consist of multiple lines, like here.
161 |     </li>
162 | </ul>
163 | ```
164 | 
165 | There are three types of _text_ blocks: _plain-text_ (|), _markup_ (/), _verbatim_ (!).
166 | 
167 |     | Plain-text block may contain {'em'+'bedded'} expressions & its output is HTML-escaped.
168 |     / Markup block may contain expressions; output is not escaped, so <b>raw tags</b> can be used.
169 |     ! In a verbatim $block$ {expressions} are left unparsed, no <escaping> is done.
170 | 
171 | output:
172 | 
173 | ```html
174 | Plain-text block may contain embedded expressions &amp; its output is HTML-escaped.
175 | Markup block may contain expressions; output is not escaped, so <b>raw tags</b> can be used.
176 | In a verbatim $block$ {expressions} are left unparsed, no <escaping> is done.
177 | ```
178 | 
179 | ### Tags
180 | 
181 | Content of a tagged block can be arranged as _inline_, _outline_ ("out of the line"),
182 | or mixed inline+outline. Inline content starts right after the tag in the _headline_ 
183 | and is rendered to a more compact form.
184 | 
185 |     h1 | This is inline text, no surrounding newlines are printed in the output.
186 |     p
187 |        / These are sub-blocks of an outline content...
188 |        | ...of the parent paragraph block.
189 |   
190 | output:
191 | 
192 | ```html
193 | <h1>This is inline text, no surrounding newlines are printed in the output.</h1>
194 | <p>
195 |    These are sub-blocks of an outline content...
196 |    ...of the parent paragraph block.
197 | </p>
198 | ```
199 | 
200 | Mixed inline+outline content is allowed if a colon `:` is additionally put in the headline:
201 | 
202 |     div: | This inline text is followed by a sub-block "p".
203 |       p
204 |         i | Line 1
205 |         b | Line 2
206 | 
207 | output:
208 | 
209 | ```html
210 | <div>This inline text is followed by a sub-block "p".
211 |   <p>
212 |     <i>Line 1</i>
213 |     <b>Line 2</b>
214 |   </p></div>
215 | ```
216 | 
217 | Without a colon, all content is interpreted as _multiline_ text:
218 | 
219 |     div |
220 |       Line 1
221 |       Line 2
222 | 
223 | output:
224 | 
225 | ```html
226 | <div>
227 | Line 1
228 | Line 2
229 | </div>
230 | ```
231 | 
232 | A special _null_ tag (`.`) can be used to better align tagged and untagged blocks in the code:
233 | 
234 |     p
235 |       i | This line is in italics ...
236 |       . | ... and this one is not, but both are vertically aligned in the script.
237 |       . | The null tag helps with code alignment when a tag is missing.
238 | 
239 | output:
240 | 
241 | ```html
242 | <p>
243 |   <i>This line is in italics ...</i>
244 |   ... and this one is not, but both are vertically aligned in the script.
245 |   The null tag helps with code alignment when a tag is missing.
246 | </p>
247 | ```
248 | 
249 | Tags may have _attributes_ and can be _chained_ together using a colon `:`, like here:
250 | 
251 |     h1 class='big-title' : b : a href="http://hypertag.io" style="color:DarkBlue"
252 |         | Tags can be chained together using a colon ":".
253 |         | Each tag in a chain can have its own attributes.
254 |         | Attributes are passed in a space-separated list, no parentheses.
255 | 
256 | output:
257 | 
258 | ```html
259 | <h1 class="big-title"><b><a href="http://hypertag.io" style="color:DarkBlue">
260 |     Tags can be chained together using a colon ":".
261 |     Each tag in a chain can have its own attributes.
262 |     Attributes are passed in a space-separated list, no parentheses.
263 | </a></b></h1>
264 | ```
265 | 
266 | Shortcuts are available for the two most common HTML attributes: 
267 | `.CLASS` is equivalent to `class=CLASS`, and `#ID` means `id=ID`.
268 | 
269 |     p #main-content .wide-paragraph | text...
270 |     
271 | output:
272 | 
273 | ```html
274 | <p id="main-content" class="wide-paragraph">text...</p>
275 | ```
276 | 
277 | 
278 | ### Expressions
279 | 
280 | A Hypertag script may define _variables_ to be used in _expressions_
281 | inside plain-text and markup blocks, or inside attribute lists.
282 | A variable is created by an _assignment block_ ($). 
283 | Expressions are embedded in text blocks using `{...}`  or `$...` syntax:
284 | 
285 |     $ k = 3
286 |     $ name = "Ala"
287 |     | The name repeated $k times is: {name * k}
288 |     | The third character of the name is: "$name[2]"
289 | 
290 | output:
291 | 
292 | ```html
293 | The name repeated 3 times is: AlaAlaAla
294 | The third character of the name is: "a"
295 | ```
296 | 
297 | Assignment blocks support _augmented assignments_:
298 | 
299 |     $ a, (b, c) = [1, (2, 3)]
300 | 
301 | Each variable points to a Python object and can be used with all the standard operators
302 | known from Python:
303 | 
304 |     ** * / // %
305 |     + - unary minus
306 |     << >>
307 |     & ^ |
308 |     == != >= <= < > in is "not in" "is not"
309 |     not and or
310 |     X if TEST else Y    - the "else" clause is optional and defaults to "else None"
311 |     A:B:C               - slice operator inside [...]
312 |     .                   - member access
313 |     []                  - indexing
314 |     ()                  - function call
315 | 
316 | Python collections: _lists_, _tuples_, _sets_, _dictionaries_, can be created in a standard way:
317 | 
318 |     | this is a list:   { [1,2,3] }
319 |     | this is a tuple:  { (1,2,3) }
320 |     | this is a set:    { {1,2,1,2} }
321 |     | this is a dict:   { {'a': 1, 'b': 2} }
322 | 
323 | output:
324 | 
325 |     this is a list:   [1, 2, 3]
326 |     this is a tuple:  (1, 2, 3)
327 |     this is a set:    {1, 2}
328 |     this is a dict:   {'a': 1, 'b': 2}
329 | 
330 | Variables can be imported from other Hypertag scripts and Python modules using an _import_ block:
331 | 
332 |     from python_module import $x, $y as z, $fun as my_function, $T as MyClass
333 |     from hypertag_script import $name
334 | 
335 |     | fun(x) is equal $my_function(x)
336 |     $ obj = MyClass(z)
337 | 
338 | Wildcard import is supported: 
339 | 
340 |     from PATH import *
341 | 
342 | 
343 | If your script needs to accept external data to work on, it can use a _context block_ 
344 | to declare a list of input variables (_dynamic context_) that must be passed
345 | by the caller to the `render()` method:
346 | 
347 |     context $width          # width [px] of the page
348 |     context $height         # height [px] of the page
349 |     
350 |     | Page dimensions imported from context are $width x $height
351 | 
352 | This script can be rendered in the following way:
353 | 
354 | ```python
355 | html = HyperHTML().render(script, width = 500, height = 1000)
356 | print(html)
357 | ```
358 | and the output is:
359 | 
360 |     Page dimensions imported from context are 500 x 1000
361 | 
362 | Context blocks, if present, constitute a _public interface_ of the script, 
363 | and so they must be placed at its beginning.
364 |     
365 | <!---
366 | A special import path `~` (tilde) denotes the _dynamic context_ of script execution: 
367 | a dictionary of all variables that have been passed to the rendering method 
368 | as extra keyword arguments.
369 | 
370 |     from ~ import $width, $height
371 |     
372 |     | Page dimensions imported from the context: $width x $height
373 | 
374 | The above script can be rendered in the following way:
375 | 
376 | ```python
377 | html = HyperHTML().render(script, width = 500, height = 1000)
378 | print(html)
379 | ```
380 | and the output is:
381 | 
382 |     Page dimensions imported from the context: 500 x 1000
383 | --->
384 | 
385 | 
386 | ### Custom tags
387 | 
388 | Custom tags (_hypertags_) can be defined directly in a Hypertag script 
389 | using _hypertag definition_ blocks (%):
390 | 
391 |     % tableRow name price='UNKNOWN'
392 |         tr        
393 |             td | $name
394 |             td | $price
395 | 
396 | A hypertag may declare attributes, possibly with default values. 
397 | In places of occurrence, hypertags accept positional (unnamed) and/or keyword (named) attributes:
398 | 
399 |     table
400 |         tableRow 'Porsche'  '200,000'
401 |         tableRow 'Jaguar'   '150,000'
402 |         tableRow name='Cybertruck'
403 | 
404 | output:
405 | 
406 | ```html
407 | <table>
408 |     <tr>
409 |         <td>Porsche</td>
410 |         <td>200,000</td>
411 |     </tr>
412 |     <tr>
413 |         <td>Jaguar</td>
414 |         <td>150,000</td>
415 |     </tr>
416 |     <tr>
417 |         <td>Cybertruck</td>
418 |         <td>UNKNOWN</td>
419 |     </tr>
420 | </table>
421 | ```
422 | 
423 | <!---
424 | If, at some point, you decided to add a CSS class `.style-price` to all cells in the price column,
425 | it is enough to modify the hypertag definition instead of walking through
426 | all the rows and manually updating corresponding `td` occurrences:
427 | 
428 | Imagine that at some point you decided to add a CSS class to all cells in the price column?
429 | In HTML, you'd have to walk through all the cells and manually modify 
430 | every single occurrence (HTML is notorious for [code duplication](https://en.wikipedia.org/wiki/Duplicate_code)!),
431 | taking care not to modify `<td>` cells of another column accidentally.
432 | In Hypertag, which provides powerful ways to deduplicate code, it is enough to modify
433 | the hypertag definition adding `.style-price` in one place, and voilà:
434 | 
435 |     % tableRow name price='UNKNOWN'
436 |         tr        
437 |             td | $name
438 |             td .style-price | $price
439 | 
440 | This definition can be moved out to a separate "utility" script,
441 | or stay in the same file where it is being used, for easy maintenance - 
442 | the programmer can choose whatever location is best in a given case.
443 | In traditional templating languages, there are not so many choices:
444 | often the best you can do is separate out duplicated HTML code into a Python function,
445 | introducing code fragmentation along the way and spreading presentation code over 
446 | different types of files (views vs. models) and languages (HTML vs. Python) - 
447 | a very unclean and confusing approach.
448 | 
449 | Let's say you wanted to add another column containing formatted (rich-text) information 
450 | about a car model: funny quotes, pictures etc. Passing it as a regular attribute would be
451 | difficult, as you'd have to somehow encode the entire HTML structure: paragraphs, styles, images.
452 | Instead, you can add a special _body attribute_ (@) to the hypertag definition:
453 | --->
454 | 
455 | If you want to pass structured (rich-text) data to a hypertag, you can declare 
456 | a _body attribute_ (@) in the hypertag definition block, and then paste its contents
457 | in any place you wish:
458 | 
459 |     % tableRow @info name price='UNKNOWN'
460 |         tr
461 |             td | $name
462 |             td | $price
463 |             td
464 |                @ info           # inline form can be used as well:  td @ info
465 | 
466 | This special attribute will hold the _actual body_ of hypertag's occurrence, 
467 | represented as a tree of nodes of Hypertag's native Document Object Model (DOM),
468 | so that all rich contents and formatting are preserved:
469 | <!---
470 | This special attribute (here, `info`; an arbitrary name can be chosen) will hold the 
471 | _actual body_ of hypertag's occurrence and will pass it in a structured form of the 
472 | Hypertag's native Document Object Model (DOM), so that all rich contents and formatting 
473 | are preserved:
474 | --->
475 | 
476 |     table
477 |         tableRow 'Porsche' '200,000'
478 |             img src="porsche.jpg"
479 |             / If you insist on <s>air conditioning</s>, 🤔
480 |             / you can always hit the track and roll down the window at <u>160 mph</u>. 😎 
481 |         tableRow 'Jaguar' '150,000'
482 |             img src="jaguar.jpg"
483 |             b | Money may not buy happiness, but I'd rather cry in a Jaguar than on a bus.
484 |         tableRow 'Cybertruck'
485 |             | If you liked Minecraft you will like this one, too.
486 |             / (Honestly, I did it for the memes. <i>Elon Musk</i>)
487 | 
488 | output:
489 | ```html
490 | <table>
491 |     <tr>
492 |         <td>Porsche</td>
493 |         <td>200,000</td>
494 |         <td>
495 |            <img src="porsche.jpg" />
496 |            If you insist on <s>air conditioning</s>, 🤔
497 |            you can always hit the track and roll down the window at <u>160 mph</u>. 😎
498 |         </td>
499 |     </tr>
500 |     <tr>
501 |         <td>Jaguar</td>
502 |         <td>150,000</td>
503 |         <td>
504 |            <img src="jaguar.jpg" />
505 |            <b>Money may not buy happiness, but I'd rather cry in a Jaguar than on a bus.</b>
506 |         </td>
507 |     </tr>
508 |     <tr>
509 |         <td>Cybertruck</td>
510 |         <td>UNKNOWN</td>
511 |         <td>
512 |            If you liked Minecraft you will like this one, too.
513 |            (Honestly, I did it for the memes. <i>Elon Musk</i>)
514 |         </td>
515 |     </tr>
516 | </table>
517 | ```
518 | 
519 | <!---
520 | There can only be one _body_ attribute in a hypertag; it must be the first one
521 | on the list; it can be missing (then the tag is _void_ and its 
522 | occurrences must have empty body); and it can have arbitrary name: we suggest _@body_ 
523 | if there is no other meaningful alternative... 
524 | Yes, any associations with Python's "self" are intended and well justified.
525 | --->
526 | 
527 | Like variables, tags can also be imported from Hypertag scripts and Python modules.
528 | Due to separation of namespaces (variables vs. tags), all symbols must be 
529 | prepended with either `$` (denotes a variable) or `%` (a tag):
530 | 
531 |     from my.utils import $variable
532 |     from my.utils import %tag
533 | 
534 | 
535 | ### Filters
536 | 
537 | Hypertag defines a colon `:` as a _pipeline operator_ that allows functions (and all callables)
538 | to be used in expressions as chained _filters_. A pipeline expression of the form:
539 | 
540 |     EXPR : FUN(*args, **kwargs)
541 | 
542 | gets translated internally to:
543 | 
544 |     FUN(EXPR, *args, **kwarg)
545 | 
546 | For example, the expression:
547 | 
548 |     'Hypertag' : str.upper : list : sorted(reverse=True)
549 | 
550 | evaluates to:
551 | 
552 |     ['Y', 'T', 'R', 'P', 'H', 'G', 'E', 'A']
553 | 
554 | Functions do _not_ have to be explicitly registered as filters before use, unlike in
555 | popular templating languages (Jinja, Django templates etc.).
556 | 
557 | <!---
558 | Hypertag defines a new operator not present in Python, the _pipeline_ (`:`), for use in expressions.
559 | It is applied in a similar way as pipes `|` in templating languages:
560 | to pass a result of an expression to a function (a _filter_) as its first argument,
561 | without putting the entire expression inside the function-call parentheses,
562 | as would normally be required. A typical example of filters in a templating language:
563 | 
564 |     title | truncate(50) | upper
565 | 
566 | this takes a `title` string, truncates it to no more than 50 characters and then converts 
567 | to upper case. In Hypertag, this expression will look the same, except the pipes are
568 | replaced with colons:
569 | 
570 |     title : truncate(50) : upper
571 | 
572 | Templating languages, like Jinja or Django's templates, require that functions are explicitly
573 | declared as filters before they can be used in template code.
574 | In Hypertag, there are _no_ such restrictions. Rather, all _callables_ (functions, methods,
575 | class instantiation etc.) can be used in pipelines without special preparation. 
576 | 
577 | Obviously, pipeline operators can be chained together, so `EXPR:FUN1:FUN2` is
578 | equivalent to `FUN2(FUN1(EXPR))`. The function can be given as a compound expression,
579 | such that the use of `obj.fun` or similar constructs is possible. For example, the standard
580 | `str.upper` can be used instead of implementing a custom `upper()` function:
581 | 
582 |     'Hypertag' : str.upper : list : sorted(reverse=True)
583 | 
584 | output:
585 | 
586 |     ['Y', 'T', 'R', 'P', 'H', 'G', 'E', 'A']
587 | 
588 | Remember that all Python built-ins are available in Hypertag, that is why `str`, `list`,
589 | `sorted` etc. are accessible without an explicit import.
590 | --->
591 | 
592 | Hypertag seamlessly integrates all of Django's [template filters](https://docs.djangoproject.com/en/3.1/ref/templates/builtins/#built-in-filter-reference).
593 | They can be imported from `hypertag.django.filters` and either called as regular functions
594 | or used inside pipelines. The extra filters from [django.contrib.humanize](https://docs.djangoproject.com/en/3.1/ref/contrib/humanize/)
595 | (the "human touch" to data) are also available. Django must be installed on the system.
596 | 
597 |     from hypertag.django.filters import $slugify, $upper
598 |     from hypertag.django.filters import $truncatechars, $floatformat
599 |     from hypertag.django.filters import $apnumber, $ordinal
600 | 
601 |     | { 'Hypertag rocks' : slugify : upper }
602 |     | { 'Hypertag rocks' : truncatechars(6) }
603 |     | { '123.45' : floatformat(4) }
604 | 
605 |     # from django.contrib.humanize:
606 |     | "5" spelled out is "{ 5:apnumber }"
607 |     | example ordinals {1:ordinal}, {2:ordinal}, {5:ordinal}
608 |     
609 | output:
610 | 
611 |     HYPERTAG-ROCKS
612 |     Hyper…
613 |     123.4500
614 | 
615 |     "5" spelled out is "five"
616 |     example ordinals 1st, 2nd, 5th
617 | 
618 | 
619 | ### Control blocks
620 | 
621 | There are _control blocks_ in Hypertag: "if", "try", "for", "while". For example:
622 | 
623 |     $size = 5
624 |     if size > 10      
625 |         | large size
626 |     elif size > 3:
627 |         | medium size
628 |     else
629 |         | small size
630 | 
631 | output:
632 | 
633 |     medium size
634 | 
635 | Clauses may have inline body; notice the parentheses around expressions:
636 | 
637 |     $size = 5
638 |     if (size > 10)    | large size
639 |     elif (size > 3)   | medium size
640 |     else              | small size
641 | 
642 | Examples of loops:
643 | 
644 |     for i in [1,2,3]  | $i
645 | 
646 |     for i in [1,2,3]:
647 |         li | item no. $i
648 | 
649 |     $s = 'abc'
650 |     while len(s) > 0               -- Python built-ins ("len") can be used
651 |         | letter "$s[0]"
652 |         $s = s[1:]                 -- assignments can occur inside loops
653 | 
654 | output:
655 | 
656 | ```html
657 | 123
658 | 
659 | <li>item no. 1</li>
660 | <li>item no. 2</li>
661 | <li>item no. 3</li>
662 | 
663 | letter "a"
664 | letter "b"
665 | letter "c"
666 | ```
667 | 
668 | The "try" block consists of a single `try` clause plus any number (possibly none) of `else` clauses.
669 | The first clause that does _not_ raise an exception is returned.
670 | All exceptions that inherit from Python's Exception are caught (this cannot be changed). Example:
671 | 
672 |     $cars = {'ford': 60000, 'audi': 80000}
673 |     try
674 |         | Price of Opel is $cars['opel'].
675 |     else
676 |         | Price of Opel is unknown.
677 | 
678 | output (the 'opel' key is missing in the dictionary):
679 | 
680 |     Price of Opel is unknown.
681 | 
682 | <!---
683 | Similar code as above, but with inline body:
684 | 
685 |     $cars = {'ford': 60000, 'audi': 80000}
686 |     
687 |     try  | Price of Opel is $cars['opel'].
688 |     else | Price of Opel is not available, but how about Seat: $cars['seat'].
689 |     else | Neither Opel nor Seat is available.
690 |            Let's stick with a Ford: $cars['ford'].
691 | 
692 | output:
693 | 
694 |     Neither Opel nor Seat is available.
695 |     Let's stick with a Ford: 60000.
696 | --->
697 | 
698 | There is a shortcut version "?" of the "try" syntax, which has no "else" clauses, so its 
699 | only function is to suppress exceptions:
700 | 
701 |     ? | Price of Opel is $cars['opel'].
702 | 
703 | Importantly, unlike the basic form of "try", the shortcut "?" can prepend a tagged block. 
704 | The code below renders empty string instead of raising an exception:
705 | 
706 |     ? b : a href=$cars.url | the "a" tag fails because "cars" has no "url" member
707 | 
708 | The "try" block is particularly useful when combined with _qualifiers_: 
709 | "optional" (`?`) and "obligatory" (`!`), placed at the end of (sub)expressions to mark 
710 | that a given piece of calculation either:
711 | 
712 | - can be ignored (replaced with `''`) if it fails with an exception (`?`); or
713 | - must be non-empty (not false), otherwise an exception will be raised (`!`).
714 | 
715 | Together, these language constructs enable fine-grained control over data post-processing,
716 | sanitization and display.
717 | They can be used to verify the availability of particular elements of data
718 | (keys in dictionaries, attributes of objects) and create alternative paths of calculation 
719 | that will handle multiple edge cases at once:
720 | 
721 |     | Price of Opel is {cars['opel']? or cars['audi'] * 0.8}
722 | 
723 | In the above code, the price of Opel is not present in the dictionary, but thanks 
724 | to the "optional" qualifier `?`, a KeyError is caught early, and a fallback is used 
725 | to approximate the price from another entry:
726 | 
727 |     Price of Opel is 64000.0
728 | 
729 | With the "obligatory" qualifier `!` one can verify that a variable has a non-default 
730 | (non-empty) value, and adapt the displayed message if not:
731 | 
732 |     % display name='' price=0
733 |         try  | Product "$name!" costs {price}!.
734 |         else | Product "$name!" is available, but the price is unknown yet.
735 |         else | There is a product priced at {price!}.
736 |         else | Sorry, we're closed.
737 | 
738 |     display 'Pen' 100
739 |     display 'Pencil'
740 |     display price=25
741 | 
742 | output:
743 | 
744 |     Product "Pen" costs 100.
745 |     Product "Pencil" is available, but the price is unknown yet.
746 |     There is a product priced at 25.
747 | 
748 | Qualifiers can also be used in loops to test for non-emptiness of the collections 
749 | to be iterated over:
750 | 
751 |     try
752 |         for p in products!
753 |             | $p.name costs $p.price
754 |     else
755 |         | No products currently available.
756 | 
757 | When initialized with `$products=[]`, the above code outputs:
758 | 
759 |     No products currently available.
760 | 
761 | <!---
762 | Qualifiers can be placed after all atomic expressions and embeddings, no space is allowed.
763 | --->
764 | 
765 | ### Built-ins
766 | 
767 | All Python built-ins, including the common types and functions: 
768 | `list`, `set`, `dict`, `int`, `min`, `max`, `enumerate`, `sorted` etc., 
769 | are automatically imported and can be used in a Hypertag script:
770 | 
771 |     | $len('cat'), $list('cat')
772 |     | $int('123'), $min(4,5,6)
773 |     for i, c in enumerate(sorted('cat')):
774 |         | $i, $c  
775 | 
776 | output:
777 | 
778 |     3, ['c', 'a', 't']
779 |     123, 4
780 |     0, a
781 |     1, c
782 |     2, t
783 | 
784 | 
785 | ### Further reading
786 | 
787 | There are still many Hypertag features that have not been mentioned 
788 | in this Quick Start: 
789 | block layout [modifiers](http://hypertag.io/#modifiers) (_dedent_, _append_); 
790 | [comments](http://hypertag.io/#comments) (`#` and `--`);
791 | [built-in](http://hypertag.io/#hypertag-built-ins) tags and functions;
792 | raw (r-) vs. formatted (f-) strings; 
793 | _pass_ keyword; concatenation operator; 
794 | [DOM manipulation](http://hypertag.io/#dom-manipulation); and more.
795 | 
796 | See the [Language Reference](http://hypertag.io/#language-reference) for details.
797 | 
798 | 
799 | ## Cheat Sheet
800 | 
801 | ### Text blocks
802 | 
803 | | &nbsp;<br> Syntax <br><img width=200/> | &nbsp;<br> Description <br>&nbsp; |
804 | | ------------- | --------------- | 
805 | | &#124; _text_   | plain-text block; may contain embedded expressions; output is HTML-escaped |
806 | | / markup        | markup block; may contain embedded expressions; output is *not* HTML-escaped |
807 | | ! _verbatim_    | verbatim block; expressions are *not* parsed; output is *not* HTML-escaped |
808 | | &#124; _multi-line_<br>&nbsp;&nbsp;_text....._ | text blocks may span *multiple lines*, also when preceded by a tag; subsequent lines must be indented |
809 | | -- _comment_ <br> # _comment_ | line of comment; is excluded from output; may occur at the end of a block's headline (_inline comment_) or on a separate line (_block comment_) |
810 | | < BLOCK       | _dedent_ modifier (<): when put on the 1st line of a BLOCK, causes the output to be dedented by one level during rendering; applies to blocks of all types (text, control etc.) |
811 | | ... BLOCK     | _append_ modifier (...): when put on the 1st line of a BLOCK, causes the output to be appended to the previous block without a newline |
812 | 
813 | ### Expressions
814 | 
815 | | &nbsp;<br> Syntax <br><img width=150/> | &nbsp;<br> Description <br>&nbsp; |
816 | | :------:          | --------------- | 
817 | | $x = a-b          | assignment block; space after $ is allowed |
818 | | $x <br> $x.v[1]   | embedding of a factor expression (a variable with 0+ tail operators) in a text block or string |
819 | | {x+y}             | embedding of an arbitrary expression in a text block or string |
820 | | x! <br> {x*y}!    | "obligatory" qualifier (!) for an atomic or embedded expression; raises as exception if the expression is false |
821 | | x? <br> {x*y}?    | "optional" qualifier (?) for an atomic or embedded expression; replaces exceptions and false values with empty strings |
822 | | 'text'    | f-string (formatted string), may contain embedded expressions: $... and {...} |
823 | | "text"    | f-string (formatted string), may contain embedded expressions: $... and {...} |
824 | | r'text'   | r-string (raw string), expressions are left unparsed |
825 | | r"text"   | r-string (raw string), expressions are left unparsed |
826 | | $$ {{ }}  | escape strings; render $ or { or } in a plaintext/markup block and inside formatted strings |
827 | <!---
828 | | %TAG      | reference to a tag in an expression (_not implemented yet_) |
829 | --->
830 | 
831 | ### Tags
832 | 
833 | | &nbsp;<br> Syntax <br><img width=700/> | &nbsp;<br> Description <br>&nbsp; |
834 | | ------------- | --------------- | 
835 | | div <br>&nbsp;&nbsp;&nbsp;&nbsp; p &#124; _text_  | _tagged block_ starts with a tag name (_header_) that can be followed by contents (_body_) on the same line (_inline body_) and/or on subsequent lines (_outline body_) |
836 | | p &#124; _this is_<br>&nbsp;&nbsp;&nbsp;&nbsp; _a multiline_<br>&nbsp;&nbsp;&nbsp;&nbsp; _paragraph_ | a tagged block with exclusively text contents may span multiple lines |
837 | | box: &#124; _Title_<br>&nbsp;&nbsp;&nbsp; li &#124; _item1_<br>&nbsp;&nbsp;&nbsp; li &#124; _item2_ | mixing inline and outline contents is possible when the colon (:) and a text-block marker (&#124;/!) are both present |
838 | | h1 : b : a href='' : <br>&nbsp;&nbsp;&nbsp;&nbsp; &#124; _text_  | tags can be chained together using colon (:); trailing colon is optional |
839 | | box "top" x=1.5 <br>a href=$url <br>a href={url} | unnamed and named (keyword) attributes can be passed to a tag in a space-separated list, no parentheses; expressions can be used as values |
840 | | %SUM x y=0 &#124; _sum is_ {x+y}<br>%TAB @cells n:<br>&nbsp;&nbsp;&nbsp;&nbsp; &#124; _Table no. $n_<br>&nbsp;&nbsp;&nbsp;&nbsp; @cells | _hypertag definition_ block (%) may declare attributes, possibly with defaults, and may have a body; the "at" sign (@) marks a special attribute that will hold actual body of hypertag's occurrence: without this attribute the hypertag is _void_ (must have empty contents in places of occurrence) |
841 | | @body <br> @body[2:] | _embedding block_ (@): inserts DOM nodes represented by an expression (typically a body attribute inside hypertag definition) |
842 | | div .CLASS        | (shortcut) equiv. to *class="CLASS"* on attributes list of a tag |
843 | | div #ID           | (shortcut) equiv. to *id="ID"* on attributes list of a tag |
844 | | pass              | "empty block" placeholder; generates no output, does *not* accept attributes nor a body |
845 | | . <br> . &#124; _text_ | the special _null tag_ (.) outputs its body without changes; helps improve vertical alignment of text in adjecent blocks; does *not* accept attributes |
846 | <!---
847 | | TAG x=1.0 y={v+1} | named (keyword) attributes of a tag occurrence; space-separated, no parentheses |
848 | | TAG "yes" 3 True  | unnamed attributes of a tag occurrence; values are matched to formal attributes in a way similar to how Python matches function arguments (by order) |
849 | --->
850 | 


--------------------------------------------------------------------------------
/docs/CNAME:
--------------------------------------------------------------------------------
1 | hypertag.io


--------------------------------------------------------------------------------
/docs/_config.yml:
--------------------------------------------------------------------------------
1 | theme: jekyll-theme-leap-day


--------------------------------------------------------------------------------
/docs/pypi.md:
--------------------------------------------------------------------------------
 1 | 
 2 | **Hypertag** is a modern language for front-end development that allows
 3 | writing markup documents in a way similar to writing Python scripts,
 4 | where [indentation](http://hypertag.io/#layout) determines relationships between nested elements 
 5 | and removes the need for explicit closing tags. Hypertag provides:
 6 | 
 7 | - Advanced control of rendering process with native [control blocks](http://hypertag.io/#control-blocks) 
 8 |   (_if_, _for_, _while_, _try_).
 9 | - High level of modularity thanks to Python-like [imports](http://hypertag.io/#imports) and explicit [context](http://hypertag.io/#context) specification.
10 | - Unprecedented support for code reuse with native [custom tags](http://hypertag.io/#custom-tags).
11 | - Native [DOM](http://hypertag.io/#dom) representation and [DOM manipulation](http://hypertag.io/#dom-manipulation) during rendering.
12 | - Embedded [compound expressions](http://hypertag.io/#expressions).
13 | - All Python [operators](http://hypertag.io/#operators) to manipulate arbitrary Python objects.
14 | - Custom [pipeline](http://hypertag.io/#filters) operator (`:`) for chaining multiple functions as [filters](http://hypertag.io/#filters).
15 | - Expression [qualifiers](http://hypertag.io/#qualifiers) (`!?`) to create alternative paths of calculation and handle edge cases easily.
16 | - Built-in tags for [HTML5](http://hypertag.io/#html-specific-symbols) generation and for [general](http://hypertag.io/#hypertag-built-ins) purposes.
17 | - Integrated [Python built-ins](http://hypertag.io/#python-built-ins).
18 | - Integrated [Django filters](http://hypertag.io/#django-filters).
19 | - [Django connector](http://hypertag.io/#django-connector).
20 | 
21 | More information:
22 | - **[Quick Start](https://github.com/mwojnars/hypertag#quick-start)**
23 | - **[Language Reference](http://hypertag.io/#language-reference)**
24 | 
25 | 
26 | ## Setup
27 | 
28 | Install:
29 | ```
30 | pip install hypertag-lang               # watch out the name, it is "hypertag-lang"
31 | ```
32 | 
33 | Usage:
34 | ```python
35 | from hypertag import HyperHTML
36 | html = HyperHTML().render(script)       # rendering of a Hypertag `script` to HTML
37 | ```
38 | 
39 | ## Why to use Hypertag
40 | 
41 | - **Concise syntax**: inspired by Python, the indentation-based syntax is a lot cleaner, 
42 |   more readable and maintainable than raw markup; it requires less typing, is less redundant,
43 |   and lets you concentrate on coding rather than chasing unmatched opening-closing tags.
44 | - **Code reuse** by means of functions/classes is the corner stone of programming,
45 |   yet it is missing from HTML; this is fixed now with Hypertag:
46 |   programmers can create reusable components in a form of **custom tags** (_hypertags_), 
47 |   defined either as Python functions (_external tags_) 
48 |   or directly in a document using Hypertag syntax (_native tags_);
49 |   hypertags can be parameterized and may represent complex pieces 
50 |   of combined: content, style and layout - for a reuse across documents.
51 | - **Fine-grained control** over rendering process is possible with
52 |   a range of native _control blocks_ (for, while, if-elif-else, try-else) 
53 |   constituting a core part of Hypertag syntax, unlike in templating languages, 
54 |   where control structures are artificially overlaid on top of another language (HTML).
55 | - **Modularity** in Hypertag is modeled after Python's: 
56 |   every script may import tags and variables from other scripts,
57 |   from Python modules, and from the dynamic _context_ of script rendering;
58 |   scripts and modules are arranged into packages;
59 |   with these mechanisms in place, building libraries of reusable components is easy and fun.
60 | - **Applicability** to different target languages. 
61 |   Hypertag is _not_ just a templating system put on top of HTML. 
62 |   Hypertag is a full-featured standalone programming language tailored to the generation
63 |   of documents of all kinds. By defining new tags, Hypertag can be adapted to produce an arbitrary
64 |   document description language.
65 |   
66 | 
67 | ## Acknowledgements
68 | 
69 | Hypertag was partially modeled on [Python's](https://www.python.org/) syntax, and 
70 | was inspired by indentation-based templating languages:
71 | [Slim](http://slim-lang.com/), [Plim](https://plim.readthedocs.io/en/latest/index.html),
72 | [Shpaml](http://shpaml.com/), [Haml](https://haml.info/).
73 | 
74 | 


--------------------------------------------------------------------------------
/hypertag:
--------------------------------------------------------------------------------
1 | src/hypertag


--------------------------------------------------------------------------------
/pyproject.toml:
--------------------------------------------------------------------------------
1 | [build-system]
2 | requires = [
3 |     "setuptools>=42",
4 |     "wheel"
5 | ]
6 | build-backend = "setuptools.build_meta"
7 | 


--------------------------------------------------------------------------------
/setup.cfg:
--------------------------------------------------------------------------------
 1 | [metadata]
 2 | name = hypertag-lang
 3 | version = 1.2.0
 4 | author = Marcin Wojnarski
 5 | author_email = hypertag@paperity.org
 6 | description = Modern language for markup generation with Python-like concise syntax, custom tags & Django integration. HTML templates reinvented.
 7 | long_description = file: docs/pypi.md
 8 | long_description_content_type = text/markdown
 9 | url = http://hypertag.io/
10 | project_urls =
11 |     Source = https://github.com/mwojnars/hypertag
12 | classifiers =
13 |     Programming Language :: Python :: 3
14 |     License :: OSI Approved :: MIT License
15 |     Operating System :: OS Independent
16 | keywords = markup, language, html, html5, css, templates, templating, tags, jinja2, mako, django, DOM
17 | 
18 | [options]
19 | package_dir =
20 |     = src
21 | packages = find:
22 | python_requires = >=3.5
23 | setup_requires =
24 |     wheel
25 | install_requires =
26 |     parsimonious==0.8.1
27 |     six>=1.12.0
28 | 
29 | [options.packages.find]
30 | where = src
31 | 


--------------------------------------------------------------------------------
/src/hypertag/__init__.py:
--------------------------------------------------------------------------------
1 | from hypertag.core.run_html import Runtime, HyperHTML
2 | from hypertag.core.tag import Tag, TagFunction, Markup
3 | from hypertag.core.dom import DOM
4 | 


--------------------------------------------------------------------------------
/src/hypertag/builtins.py:
--------------------------------------------------------------------------------
 1 | #
 2 | # Hypertag built-ins that are common to all target languages.
 3 | #
 4 | # They can be imported manually into a script with a statement of the form:
 5 | #
 6 | #    from hypertag.builtins import %dedent
 7 | #
 8 | 
 9 | 
10 | from hypertag.std.builtins import register
11 | 
12 | __tags__ = register.tags                # make all tags available for import
13 | globals().update(register.vars)         # make all variables available for import
14 | 
15 | del globals()['register']
16 | 


--------------------------------------------------------------------------------
/src/hypertag/core/README_draft.md:
--------------------------------------------------------------------------------
  1 | ## Introduction
  2 | 
  3 | <!---
  4 | If you want to build commercial websites with Hypertag
  5 | I can supervise or consult on your projects.
  6 | Please drop me an e-mail or contact me via LinkedIn.
  7 | 
  8 | Hypertag enables advanced control of document rendering process through
  9 | native **control blocks** (if-elif-else, for, while, try);
 10 | and let the programmer define **custom tags** (_hypertags_), either directly
 11 | in the document using Hypertag syntax (_native tags_), 
 12 | or as Python functions (_external tags_).
 13 | and provides high level of **modularity** through custom tags and import blocks. 
 14 | --->
 15 | 
 16 | ## Cheet Sheet
 17 | 
 18 | ### Control blocks
 19 | 
 20 | | &nbsp;<br> Syntax <br><img width=600/> | &nbsp;<br> Description <br>&nbsp; |
 21 | | ------------- | --------------- | 
 22 | | import $x <br> from ~ import $x as y <br> from hypertag.html import %a, %img <br> from PATH import $x, %TAG <br> | _import block_ reads a variable ($NAME) or a tag (%NAME) from another Hypertag or Python module (PATH) or from the dynamic context of rendering ("~" default) and adds it to the namespace of the current script |
 23 | | if COND &#124; _text_ <br> if COND:<br>&nbsp;&nbsp;&nbsp;&nbsp;BLOCKS...<br> elif: ... <br>else: ... | _if-elif-else_ construct behaves similarly as in Python; trailing colons (:) are optional |
 24 | | try-else | may have a body (inline, outline), but cannot be followed by tags on the same line |
 25 | 
 26 | 
 27 | | Symbol    | Syntax     | Description     |
 28 | | :-------: | --------------- | --------------- | 
 29 | | ?         |       | *optional block*; like a "try" without an "else" branch; can be applied to a tagged block (on the same line) |
 30 | | for       | for VAR in SEQ |
 31 | | while     | while EXPR |
 32 | 
 33 | Trailing colon (:) in a headline of a control block (for/while) or a clause (if/elif/else/try)
 34 | is _optional_ and can be omitted in most cases - with an exception for a mixed
 35 | text+structural body (text title in a headline + structural sub-blocks beneath),
 36 | which requires that a text marker in a headline is preceeded by a colon.
 37 | 
 38 |     box: | This box has both an inline "title" and a list of sub-blocks
 39 |         li : b | item 1
 40 |         li : b | item 2
 41 | 
 42 | ## Terminology
 43 | 
 44 | **Headline**. The first line of a tagged or control block. 
 45 | Contains a tag, or multiple tags, or a keyword.
 46 | 
 47 | **Header**. The initial part of a headline, up to the last marker (see below).
 48 | 
 49 | **Marker**. A special symbol (one of &#124;/!@:) that marks the beginning of a body 
 50 | of a tagged or control block. Typically, there is exactly one marker in a headline,
 51 | though it can be omitted altogether (when a body starts in a new line).
 52 | There is also a special case when the colon (:) can be followed 
 53 | ().
 54 | 
 55 | If the entire body of a block fits in the headline, it is called an **inline body**.
 56 | If the body starts on a new line after the headline, it is called an **outline body**.
 57 |  - is called a **header**.
 58 | 
 59 | **headline** = **header** + **marker** of body type + **title** (inline body)
 60 | 
 61 | **inline body** - consists of a title only
 62 | **multiline body** - 
 63 | **outline body** - consists of sub-blocks, no title
 64 | 
 65 | Trailing colon (:) after a header indicates that subsequent lines will contain blocks
 66 | rather than multi-line text - even if there is a text-block marker .
 67 | 
 68 | 
 69 | ## Blocks
 70 | 
 71 | Contents of a text block may span multiple lines. The additional lines
 72 | must be indented more than the first line. Sub-indentation is preserved in the output:
 73 | 
 74 |     |   a text block
 75 |       may span
 76 |         multiple
 77 |       lines...
 78 | 
 79 | renders to
 80 | 
 81 |       a text block
 82 |     may span
 83 |       multiple
 84 |     lines...
 85 | 
 86 | 
 87 | ### Tagged blocks
 88 | 
 89 | <!---
 90 | body layout: inline / outline / headline
 91 | 
 92 | The entire header must fit in the first line of a block.
 93 | For a body, there are several possibilities: it may follow directly after
 94 | the header on the same line (_inline body_), occupy subsequent lines
 95 | (_outline body_), mix both of these approaches (__), or contain a multi-line
 96 | text contents (_multiline body_).
 97 | --->
 98 | 
 99 | A **tag** is a function-like object belonging to the _tags namespace_ 
100 | (see section ........) that performs some kind of transformation 
101 | of the block's body represented by a DOM (see section .......).
102 | The execution of this transformation is called **tag expansion**:
103 | a DOM or a string is produced as a result, which is subsequently concatenated
104 | with the output of adjacent blocks to form final output of script rendering.
105 | Tag expansion can be moderated by values of attributes.
106 | 
107 | For example, the built-in tag `div` as provided by HyperHTML runtime 
108 | wraps up the rendered body of a block with `<div>...</div>`
109 | prefix and suffix during expansion, and pastes into the opening `<div>` 
110 | any attributes that have been declared in the block.
111 | 
112 | Technically speaking, every tag is an instance of `hypertag.core.tag.Tag` that
113 | accepts - through the method `Tag.expand()` - any number of positional 
114 | and/or keyword arguments (attributes) plus the special _body_ argument 
115 | that holds the DOM of a translated body of the tag's occurrence -
116 | in the example above this would be a DOM holding two plain-text nodes.
117 | 
118 | <!---
119 | The tag can perform any kind of processing of input DOM and 
120 | returns a DOM or a string that will be concatenated with the output of adjacent blocks
121 | to form the final output of rendering.
122 | (See section ...... for details about DOM.)
123 | --->
124 | 
125 | A tag can be implemented in Python (_external tag_) and imported to a script 
126 | using an _import_ block (see section ......), or defined directly in Hypertag
127 | using a _hypertag definition_ block (_native tag_, see section ......).
128 | In both cases, the parser represents the tag internally as an instance
129 | of a `hypertag.core.tag.Tag` subclass (`ExternalTag` or `NativeTag`, respectively).
130 | 
131 | Every Hypertag runtime may define a list of _built-in tags_ that will be imported
132 | automatically when script parsing begins.
133 | For example, `HyperHTML` provides built-in Tags for all valid HTML tags,
134 | and there are two variants available for each tag: lower-case and upper-case.
135 | (see section ....... for details).
136 | 
137 |  
138 | ### Special tags
139 | 
140 | #### Null tag (.)
141 | 
142 | Normally, a Hypertag script reads top-to-bottom through a vertical sequence of blocks;
143 | however, it can also exhibit left-to-right arrangement of tags vs. textual contents. 
144 | In blocks that could have a tag, but don't need one in a particular place,
145 | you can put a dot "." instead of a tag name to provide valid indentation and still 
146 | ensure consistent vertical alignment of *tags* on the left and *text* on the right,
147 | like here:
148 | 
149 |     div:
150 |         p
151 |             i   | this line is in italics and so it requires a tag
152 |             .   | this line needs no tags, hence the null tag is used as a placeholder 
153 |             
154 | The dot represents a _null tag_ that creates a node in the DOM (like any regular tag) 
155 | but during rendering it passes the body unchanged to the output.
156 | In other words, a _null tag_ serves as a placeholder that performs no processing 
157 | of its body, similar to an identity function.
158 | 
159 | The null tag can also be used as a root node for a group of sub-blocks, 
160 | to provide visual grouping in the source code and/or allow finer control
161 | over node filtering and selection if the blocks were to be passed to a hypertag (as _@body_)
162 | and processed with DOM selectors.
163 | 
164 | <!---
165 | Typically, a null tag is used in either of two cases:
166 | 
167 | 1. to vertically align text contents of mixed: tagged and untagged blocks;
168 |    using a null tag allows all blocks to have the same indentation of text contents,
169 |    even despite some blocks need more horizontal space for their (regular) tags;
170 | 1. to visually group in the script related blocks; note that sub-indentation of the
171 |    dotted block will be preserved in the output.
172 | --->
173 | 
174 | Both use cases are illustrated in the example below:
175 | 
176 |     h1                | TITLE
177 |     .
178 |         a href=$url   | click HERE!
179 |         .             | to visit our website
180 | 
181 | 
182 | #### Pass tag
183 | 
184 | A block consisting of a single keyword `pass` (the _"pass" tag_), no attributes, no body - 
185 | constitutes a "pass block" that serves as a placeholder that does simply nothing, 
186 | not even rendering a newline (unlike the null tag). The pass tag corresponds 
187 | to Python's "pass" keyword. Typically, a pass block is used inside control blocks 
188 | (for/if/else) to mark an empty branch, which may appeal to aesthetics or be a way to mark 
189 | unfinished implementation. Example:
190 | 
191 |     if condition
192 |         p | render something
193 |     else
194 |         pass
195 | 
196 | The above code is equivalent to:
197 | 
198 |     if condition
199 |         p | render something
200 | 
201 | 
202 | ### Hypertag definition
203 | 
204 | A Hypertag script may contain definitions of custom tags, called **hypertags**,
205 | which can be viewed as equivalents of Python functions
206 | and be used in all the same places as built-in tags.
207 | Like functions in traditional programming languages, hypertags enable _modularization_
208 | and _code de-duplication_: repeated parts of a script can be moved out
209 | to a hypertag and re-used instead of being copy-pasted multiple times; also, hypertags 
210 | can be placed in separate files, then imported and re-used across different scripts
211 | and applications, as a library of predefined **widgets**.
212 | This realization of the DRY principle is an important aspect of the Hypertag language,
213 | one that has been missing in standard front-end languages (HTML, XML).
214 | 
215 | A hypertag definition block starts with "%" followed by spaces (optional),
216 | then a name of new tag (obligatory), its attributes (optional) and formal body (optional). 
217 | Default values in a form of literals or expressions can be provided for attributes.
218 | If the first attribute on the list is prefixed with a special symbol "@",
219 | this attribute will be assigned the actual body of the block of occurrence
220 | (where the new tag is used); the "body" attribute can have an arbitrary name, 
221 | but it cannot have a default value. If a body attribute is missing, the hypertag
222 | is "void", which means its occurrences must always have empty contents.
223 | 
224 | An example hypertag definition looks like this:
225 | 
226 |     % fancy_text @body size='10px':
227 |         | *****
228 |         p style=("color: blue; font-size: " size)
229 |             @body
230 |         | *****
231 | 
232 | The above block defines a new tag "fancy_text", which can be used like this:
233 | 
234 |     fancy_text '20px'
235 |         | This text is rendered through a FANCY hypertag!
236 | 
237 | During rendering of this block the definition body of `fancy_tag` gets expanded
238 | and the actual `@body` is inserted, resulting in the following output:
239 | 
240 |     *****
241 |     <p style="color: blue; font-size: 20px">
242 |         This text is rendered through a FANCY hypertag!
243 |     </p>
244 |     *****
245 | 
246 | 
247 | Hypertag definition blocks can be nested in tagged blocks and in other
248 | definition blocks (_nested definitions_), but not in control blocks.
249 | 
250 | 
251 | ### DOM embedding
252 | 
253 |     @ body
254 |     @ body[0]
255 |     @ body.METHOD()
256 | 
257 | 
258 | ### Assignments
259 | 
260 | A block that starts with $ marks an assignment to a local variable:
261 | 
262 |     $ x = 5
263 |     $ dash = x * '-'
264 |     $ title = dash + ' TITLE ' + dash
265 | 
266 | 
267 | ### Control blocks
268 | 
269 | #### Block "try"
270 | 
271 | Basic form:
272 | 
273 |     try ...
274 |     else ...
275 |     else ...
276 | 
277 | If there are no "else" branches, a try block can be written in a shorter form using `?`:
278 | 
279 |     ?tag ...
280 |     ? tag ...
281 |     ?
282 |         block1
283 |         block2
284 | 
285 | During translation, a "try" block returns a DOM generated by the first clause 
286 | that does _not_ raise an exception; or an empty DOM if all clauses have failed
287 | (the exception from the last clause is _not_ re-raised).
288 | 
289 | Note that here, the semantics of "else" is _opposite_ to what it is in Python, where
290 | the "else" clause of a "try-else" statement only gets executed if _no_ exception occured.
291 | 
292 | All exceptions that inherit from Python's Exception are caught, including Hypertag exceptions.
293 | There is no way to define a different set of exceptions to be caught.
294 | Python's special exceptions that inherit directly from BaseException:
295 | SystemExit, KeyboardInterrupt, GeneratorExit - are being passed up the tree.
296 | 
297 | Note that exceptions are caught during _translation_ of the block only.
298 | If there are syntactical or name resolution errors (e.g., an undefined variable
299 | was found in any clause), these errors are still passed up the tree.
300 | 
301 | 
302 | #### Block "if"
303 | 
304 | #### Block "for"
305 |     
306 | - break & continue
307 | 
308 | #### Block "while"
309 | 
310 | #### Block "import"
311 | 
312 | 1. Import from the global context:
313 |   
314 |         import $OBJECT
315 |         import %TAG
316 | or
317 |         from ~ import $OBJECT
318 |         from ~ import %TAG
319 | 
320 | The special import path "~" denotes the current global context of script execution.
321 | When the `from PATH` clause is omitted the `from ~` path is assumed.
322 | When a script is imported to another script, the context stays the same for both.
323 | 
324 | 2. Import from a Python module:
325 | 
326 | from PATH import NAME -- import from a module denoted by PATH, which can be any string
327 |   consisting of [...] characters that can be correctly interpreted by the currently used
328 |   runtime
329 | 
330 | 3. Import from a Hypertag script:
331 | 
332 | ...
333 | 
334 | 4. Import from a combined Python-Hypertag module:
335 | 
336 | ...
337 | 
338 | 
339 | #### Inline comments
340 | 
341 | An inline comment occurs at the end of a line containing a header of any structural block
342 | (a block that may contain nested blocks, i.e., any block except text blocks). Examples:
343 | 
344 |     p      -- comment
345 |     p:     -- comment
346 |     p:     #  comment
347 |     for i in [1,2,3]:     -- comment
348 |         pass
349 |     if test               # comment
350 |         | test is true
351 | 
352 | 
353 | 
354 | ### Indentation
355 | 
356 | Normally, _script indentation_ is stored in the DOM and translates to 
357 | _output indentation_ of the text produced during rendering.
358 | You can modify this behaviour and remove output indentation in two ways, by using:
359 | 
360 | - Dedent marker (<): removes output indentation of a single block
361 |   the marker is used with
362 | - Dedent tag (builtin): removes output indentation of all nested blocks
363 |   (but the indentation of the dedent block itself stays untouched)
364 | 
365 | The _dedent marker_ and _dedent tag_ can be used together. Example:
366 | 
367 |     div
368 |         < dedent : a
369 |             p:    / These <p> blocks will be aligned with their parent (<a>) after rendering.
370 |                 b | Deeper-level nested block are dedented, too!
371 |             p     / And the parent's indentation is removed, as well
372 |                     (<a> is aligned with top-level <div>), thanks to 
373 |                     the use of the <i>dedent marker</i>.
374 | 
375 | 
376 | ## Symbols
377 | 
378 | ### Namespaces
379 | 
380 | During translation of a script to a DOM, Hypertag maintains two separate namespaces:
381 | 
382 | - **tags namespace**
383 | - **variables namespace**
384 | 
385 | Only symbols in the tags namespace can be used as tags in tagged blocks.
386 | Only symbols in the variables namespace can occur in expressions,
387 | unless the `%` prefix is added to inform that a given symbol should be taken
388 | from the tags namespace instead (NOT IMPLEMENTED YET).
389 | 
390 | The same name can be present in both namespaces at the same time and link to different 
391 | objects _without_ causing a collission. This is an important feature when 
392 | generating HTML documents. The HTML defines dozens of tags, whose names are fixed 
393 | and often very short (i, b, p, code, form, head, body, ...) - 
394 | without separation of namespaces, they would often collide with names 
395 | commonly used by programmers for naming variables.
396 | 
397 | <!---
398 | Maintaining separate namespaces is justified by the fact that in 
399 | typical use cases - like HTML generation - there are dozens of predefined tags
400 | with short names, all of which must be directly accessible. 
401 | Many of these tags have very common names (i, b, p, code, form, head, body, ...),
402 | and without separation of name spaces, name collissions between tags and local variables 
403 | would be very frequent and lead to confusion. 
404 | --->
405 | 
406 | The ways to add symbols to the two namespaces:
407 | 
408 | - Both namespaces can be initialized with built-in symbols by the runtime.
409 | - Both namespaces can be extended with new symbols by `import` blocks.
410 | - A hypertag definition block (`%...`) adds the name of a new tag to the tags namespace
411 |   in the outer scope, while adding symbols of formal attributes to the variables
412 |   namespace in the inner scope.
413 | - Assignments (`$...`) and `for` blocks add symbols to the variables namespace.
414 | 
415 | ### Name scoping
416 | 
417 | The two main namespaces (tags namespace, variables namespace) are _not_ flat, 
418 | rather they are arranged in a hierarchy that follows the structure 
419 | of the document (_hierarchical name scoping_). Every tagged block, 
420 | as well as a hypertag definition block, creates a new branch in the namespace:
421 | new symbols are only added to this branch; they are visible to sibling blocks 
422 | at the same depth and to their sub-blocks, but are _not_ accessible from the outer scope.
423 | For example:
424 | 
425 |     p
426 |         $x = 1
427 |         -- "x" can be accessed inside the paragraph
428 |         | $x
429 |     -- "x" cannot be accessed outside the paragraph
430 | 
431 | Obviously, symbols defined at a higher level can be overwritten at a lower level
432 | down the document tree:
433 | 
434 |     $ x = 1
435 |     p:
436 |         $ x = 2
437 |         | x inside the paragraph equals $x
438 |     | x outside the paragraph equals $x
439 | 
440 | outputs:
441 | 
442 |     <p>
443 |         x inside the paragraph equals 2
444 |     </p>
445 |     x outside the paragraph equals 1
446 | 
447 | Note that unlike tagged and definition blocks, control blocks do _not_ create new branches
448 | in namespaces by themselves. Therefore, it is correct to assign a variable inside 
449 | an if/try/for/while block and access its value in sibling blocks:
450 | 
451 |     if True:
452 |         $x = 1
453 |     else:
454 |         $x = 2
455 |     | x=$x is accessible in a sibling of a control block
456 | 
457 | outputs:
458 | 
459 |     x=1 is accessible in a sibling of a control block
460 | 
461 | 
462 | ## Expressions
463 | 
464 | Text blocks (normal and markup) may contain embedded expressions.
465 | An embedding may come in two forms, either using the {...} syntax:
466 | 
467 |     | value of x = {x}
468 | 
469 | or using the $... syntax:
470 | 
471 |     | value of x = $x
472 | 
473 | The former syntax can handle all types of expressions, while the latter 
474 | can only be used for variables, possibly extended with tail operators:
475 | 
476 |     | $x.value
477 |     | $x[index]
478 |     | $x(arg1, arg2)
479 | 
480 | Both forms, {...} and $..., allow the use of a trailing qualifier (? or !)
481 | to test for errors or false results:
482 | 
483 |     | {x+y}?  -- is converted to empty string if false, or if exception was raised
484 |     | $x.call()!  -- raises an exception if x.call() is false
485 | 
486 | 
487 | ### Operators
488 | 
489 | Hypertag implements majority of standard operators available in Python.
490 | 
491 | **Arithmetic and binary operators:**
492 | 
493 |     ** * / // %
494 |     + - unary minus
495 |     << >>
496 |     & ^ |
497 | 
498 | **Logical operators:**
499 | 
500 |     == != >= <= < > in is "not in" "is not"
501 |     not and or
502 | 
503 | A more general variant of the ternary "if-else" operator is available, 
504 | with the "else" branch being optional, imputed with "else None" if missing:
505 | 
506 |     X if TEST else Y
507 |     X if TEST
508 | 
509 | **Tail operators:**
510 | 
511 |     .     member access
512 |     []    indexing
513 |     ()    function call
514 | 
515 | Access to attributes of Python objects and elements of collections.
516 | Calls to methods and functions.
517 | 
518 | **Slice operator** when used inside [...]:
519 | 
520 |     start : stop : step
521 | 
522 | Above these, Hypertag implements a non-standard binary **concatenation operator** (space),
523 | as well as tail operators: **optional value** ("?") and **obligatory value** ("!").
524 | They are described in next sections.
525 | 
526 | 
527 | ### Qualifiers: ? and !
528 | 
529 | ? = _optional value_: fall back to an empty string if an error/None/False/0/... was returned
530 | 
531 | ! = _obligatory value_: raise an exception if an empty value (None/False/0/''/...) was returned
532 | 
533 | A qualifier (? or !) can be appended at the end of an atomic expression (X?, X!)
534 | to test against errors during evaluation, or emptiness (falseness) of the returned value.
535 | 
536 | With ? qualifier, if X evaluates to a false value or an exception was raised during evaluation,
537 | empty string '' is returned instead. A value, X, is false, if bool(X) == False.
538 | Empty string '', None, 0, False are examples of false values.
539 | 
540 | With ! qualifier, if X is false, MissingValue exception is raised. 
541 | Typically, this exception is caught with a surrounding "optional value" qualifier:
542 | 
543 |     (... expr! ...)?
544 | 
545 | or with a "try" block higher in the script.
546 | 
547 | In any case (X? or X!), if X is true, the value of X is returned unchanged.
548 | 
549 | Examples:
550 | 
551 |     {(post.authors ', ')? post.title}  -- prints title only if "authors" field is missing in "post"
552 |     
553 | 
554 | ## DOM
555 | 
556 | DOM = Document Object Model
557 | 
558 | ### DOM classes
559 | 
560 | ### DOM manipulation
561 | 
562 | Selectors as methods of DOM ...
563 | 
564 | ### Selectors (??)
565 | 
566 | **Terminal tag** is a tag that does not use any tags inside its (formal) body and thus 
567 | its expansion does NOT create any new tagged nodes, except possibly those that have been
568 | passed to it in the @body attribute.
569 | Terminal tags are important, because they are the only tags that may occur in a DOM tree
570 | after translation of the AST; and consequently, they are the only tags that are visible 
571 | to **selectors** and can be operated on during expansion of other (non-terminal) tags.
572 | 
573 | Note that:
574 | - All external tags that return plain text (their property `terminal = True`) are terminal.
575 | - All native hypertags that .......
576 | 
577 | 
578 | ## Modules
579 | 
580 | ### Import
581 | 
582 | Special symbols available inside a Hypertag script: `__file__`, `__package__` - 
583 | for proper interpretation of import paths. They can be initialized
584 | through same-named arguments of Runtime.render() and Runtime.translate().
585 | 
586 | - absolute vs. relative import paths
587 | 
588 | ### Export
589 | 
590 | Regular Python modules (.py files) can export symbols for use in Hypertag scripts.
591 | Any global variable defined by a module that can be imported into another Python module
592 | can also be imported to a Hypertag script as an object ($NAME).
593 | 
594 | Additionally, if a module defines a `__tags__` global - which should be a dict
595 | with strings (tag names) as keys and Tag objects as values - the content of this 
596 | dictionary can be imported as tags (%NAME) into a Hypertag script.
597 | 
598 | 
599 | ## SDK ??
600 | 
601 | - class Tag, ExternalTag, Markup -- for implementing custom tags
602 | - class DOM and inner classes (Node, Root, Text)
603 | - class Runtime
604 | 
605 | ## Questions & Answers
606 | 
607 | If you have technical questions related to Hypertag, please post them
608 | on [StackOverflow](https://stackoverflow.com/questions/ask) 
609 | and tag them with, guess... the "**hypertag**" tag.
610 | 
611 | Other questions can be posted in the _Discussions_ section on Hypertag's page in GitHub.
612 | 
613 | 


--------------------------------------------------------------------------------
/src/hypertag/core/__init__.py:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/mwojnars/hypertag/8cd989127c12d78f6902d4b3dc972de31a3e88d9/src/hypertag/core/__init__.py


--------------------------------------------------------------------------------
/src/hypertag/core/dom.py:
--------------------------------------------------------------------------------
  1 | """
  2 | Classes that represent Hypertag's native DOM (Document Object Model) that is produced as an output
  3 | of translation of the AST. The DOM can subsequently undergo "rendering" to a document in a target language.
  4 | 
  5 | DOM manipulation methods:
  6 | - find(path): generate a sequence of all (sub)nodes that match a given path.
  7 | - remove(path): remove from the DOM and return all (sub)nodes that match a given path;
  8 | 
  9 | @author:  Marcin Wojnarski
 10 | """
 11 | 
 12 | import re, itertools
 13 | from copy import copy
 14 | from types import GeneratorType
 15 | 
 16 | from hypertag.nifty.util import Object
 17 | 
 18 | from hypertag.core.errors import VoidTagEx, TypeErrorEx
 19 | 
 20 | 
 21 | ########################################################################################################################################################
 22 | #####
 23 | #####  UTILITIES
 24 | #####
 25 | 
 26 | 
 27 | def add_indent(text, indent, re_start = re.compile(r'\n(?=.)')):        # re.compile(r'(?m)^(?=.)')
 28 |     """
 29 |     Append `indent` string at the beginning of each line of `text` excluding the 1st line (!).
 30 |     Empty lines (containing zero characters, not even a space) are left untouched!
 31 |     """
 32 |     if not indent: return text
 33 |     return re_start.sub('\n' + indent, text)
 34 |     # if not text: return text
 35 |     # return indent + text.replace('\n', '\n' + indent)
 36 |     
 37 | def del_indent(text, indent = None):
 38 |     """
 39 |     Remove `indent` string from the beginning of each line of `text`, wherever it is present as a line prefix.
 40 |     If indent=None, maximum common indentation (get_indent()) is truncated.
 41 |     """
 42 |     if indent is None: indent = get_indent(text)
 43 |     if text.startswith(indent): text = text[len(indent):]
 44 |     return text.replace('\n' + indent, '\n')
 45 | 
 46 | def get_indent(text):
 47 |     """
 48 |     Retrieve the longest indentation string fully composed of whitespace
 49 |     that is shared by ALL non-empty lines in `text`, including the 1st line (if it contains a non-whitespace).
 50 |     """
 51 |     lines = text.split('\n')
 52 |     lines = list(filter(None, [l.rstrip() for l in lines]))             # filter out empty or whitespace-only lines
 53 |     # lines = list(filter(None, [l if l.strip() else '' for l in lines]))          # filter out empty lines
 54 |     if not lines: return ''
 55 | 
 56 |     # iterate over columns (!) of `text`, from left to right
 57 |     for i, column in enumerate(zip(*lines)):        # zip() only eats up as many characters as the shortest line
 58 |         if not column[0].isspace() or min(column) != max(column):
 59 |             return lines[0][:i]
 60 |     else:
 61 |         size = min(map(len, lines))
 62 |         return lines[0][:size]                      # when all lines are prefixes of each other take the shortest one
 63 |     
 64 | 
 65 | ########################################################################################################################################################
 66 | #####
 67 | #####  DOCUMENT OBJECT MODEL (DOM)
 68 | #####
 69 | 
 70 | class DOM:
 71 |     """
 72 |     List of DOM.Nodes that comprise (a part of) a body of a parent Node, or was produced as an intermediate
 73 |     collection of nodes during DOM manipulation.
 74 |     Provides methods for traversing DOM (sub)trees and selecting nodes,
 75 |     as well as flattening and cleaning up the list during node construction.
 76 |     
 77 |     All DOM node classes: Node, Root, Text - are defined as inner classes of this one.
 78 |     """
 79 |     nodes = None
 80 |     
 81 |     def __init__(self, *nodes, **params):  #_strict = True
 82 |         _strict = params.pop('_strict', True)
 83 |         if params: raise TypeErrorEx('unrecognized keyword argument "%s" in DOM.__init__()' % list(params.keys())[0])
 84 |         self.nodes = self._flatten(nodes) if _strict else list(nodes)
 85 |         # self.set_nodes(nodes, _strict)
 86 |         
 87 |     # def set_nodes(self, nodes, strict = True):
 88 |     #     self.nodes = self._flatten(nodes) if strict else list(nodes)
 89 |     
 90 |     def __bool__(self):             return bool(self.nodes)
 91 |     def __len__(self):              return len(self.nodes)
 92 |     def __iter__(self):             return iter(self.nodes)
 93 |     def __getitem__(self, pos):
 94 |         
 95 |         if isinstance(pos, int):
 96 |             return self.nodes[pos]
 97 |         if isinstance(pos, (int, slice)):
 98 |             return DOM(self.nodes[pos], _strict = False)
 99 |         # if isinstance(pos, (str, Tag)):
100 |         return self.select(pos)
101 |     
102 |     @staticmethod
103 |     def node(*args, **kwargs):
104 |         """
105 |         Create a DOM consisting of a single DOM.Node instance initialized with given arguments.
106 |         A shortcut for DOM(DOM.Node(...)).
107 |         """
108 |         return DOM(DOM.Node(*args, **kwargs))
109 |     
110 |     @staticmethod
111 |     def text(*args, **kwargs):
112 |         """
113 |         Create a DOM consisting of a single DOM.Text instance initialized with given arguments.
114 |         A shortcut for DOM(DOM.Text(...)).
115 |         """
116 |         return DOM(DOM.Text(*args, **kwargs))
117 |     
118 |     @staticmethod
119 |     def _flatten(nodes):
120 |         """Flatten nested lists of nodes by concatenating them into the top-level list; drop None's."""
121 |         result = []
122 |         for n in nodes:
123 |             if n is None: continue
124 |             if isinstance(n, (list, DOM, GeneratorType)):
125 |                 result += DOM._flatten(n)
126 |             elif isinstance(n, DOM.Node):
127 |                 result.append(n)
128 |             else:
129 |                 raise TypeErrorEx("found %s in a DOM, expected DOM.Node" % type(n))
130 |         return result
131 |         
132 |     def set_indent(self, indent):
133 |         for n in self.nodes:
134 |             n.set_indent(indent)
135 |             
136 |     def render(self):
137 |         return ''.join(node.render() for node in self.nodes)
138 | 
139 |     def tree(self, indent = '', step = '  '):
140 |         r"""Return a multiline \n-terminated string that presents this DOM's structure as a list of trees."""
141 |         return ''.join(node.tree(indent, step) for node in self.nodes)
142 |         
143 |     def copy(self):
144 |         """
145 |         Mostly deep copy of self, with an exception for nodes' attributes, which are shallow-copied, and their `tag` links (no copy).
146 |         """
147 |         dup = copy(self)
148 |         dup.nodes = [node.copy() for node in self.nodes]
149 |         return dup
150 |         
151 | 
152 |     ### SELECTORS API
153 |     
154 |     def walk(self, skip = None, order = 'preorder'):
155 |         """
156 |         Generator of all nodes inside this DOM: parent nodes and descendants.
157 |         Parents are yielded before descendants if order='preorder' (default), or after descendants if order='postorder'.
158 |         An optional argument `skip` is a function that takes a Node instance and returns True if the subtree
159 |         rooted at this node should be omitted, False otherwise.
160 |         The stream of nodes returned is NOT wrapped in a DOM - use select() instead if you need a DOM.
161 |         """
162 |         # if order not in ('preorder', 'postorder'): raise TypeErrorEx("incorrect value of order ({order})")
163 |         return itertools.chain(*(node.walk(skip, order) for node in self.nodes))
164 | 
165 |     def alter(self, transform, skip = None, order = 'preorder'):
166 |         """
167 |         Apply `transform` function to nodes in this DOM, and in nested DOMs.
168 |         Inside every DOM instance, starting from `self`, a list of nodes is replaced with a concatenated
169 |         list of nodes returned by transform(child) when applied to each child separately.
170 |         This is done before recursive calls to alter() on child nodes (if order='preorder'),
171 |         or after these calls if order='postorder'.
172 |         The `transform` function can be a generator, or a regular function that returns a list of nodes.
173 |         For each input node, `transform` can return any number of nodes: none, or one, or multiple;
174 |         the input node itself can also be returned.
175 |         The alter() function returns self.
176 |         """
177 |         if order == 'preorder':
178 |             for node in self.nodes: node.alter(transform, skip, order)
179 |         
180 |         new_nodes  = itertools.chain(*(transform(node) for node in self.nodes))
181 |         self.nodes = self._flatten(new_nodes)
182 |         
183 |         if order == 'postorder':
184 |             for node in self.nodes: node.alter(transform, skip, order)
185 | 
186 |         return self
187 |         
188 | 
189 |     ATTR_DEFINED = Object(name = 'ATTR_DEFINED')
190 | 
191 |     def select(self, tag = None, attr = None, value = ATTR_DEFINED, order = 'preorder', **attrs):
192 |         """
193 |         Collect a list of all nodes (including descendants) that match the search criteria and return it wrapped up in a new DOM instance.
194 |         Some of the nodes can be related, i.e., an ancestor and its descendant can be returned together,
195 |         with the ancestor still linking to its children and (directly or indirectly) to this descendant.
196 |         When a non-matching node gets removed, its subtree is NOT removed and descendants can still be included in the result.
197 |         :param tag: desired tag name (<str>) or Tag instance that should be present in a matching node
198 |         """
199 |         tag, name, attrs = self._constraint(tag, attr, value, attrs)
200 |         nodes = [node for node in self.walk(order = order) if self._test(node, tag, name, attrs)]
201 |         
202 |         # nodes = []
203 |         # for node in self.walk(order = order):
204 |         #     if tag  is not None and node.tag is not tag: continue
205 |         #     if name is not None and (node.tag is None or node.tag.name != name): continue
206 |         #
207 |         #     stop = False
208 |         #     for a, v in attrs.items():
209 |         #         if a not in node.kwattrs: stop = True; break
210 |         #         if v is not DOM.ATTR_DEFINED and node.kwattrs[a] != v: stop = True; break
211 |         #     if stop: continue
212 |         #
213 |         #     nodes.append(node)
214 |         
215 |         return DOM(*nodes, _strict = False)
216 |         
217 |         
218 |     def skip(self, tag = None, attr = None, value = ATTR_DEFINED, **attrs):
219 |         """
220 |         Return a copy of `self` that has the same structure (nodes, relations, parents and descendants),
221 |         but the subtrees rooted at nodes matching the provided criteria are removed.
222 |         The original DOM (self) and its nodes are left unmodified. All nodes in the returned DOM are copies
223 |         of nodes in self.
224 |         """
225 |         tag, name, attrs = self._constraint(tag, attr, value, attrs)
226 |         dom = self.copy()
227 |         
228 |         def drop(node):
229 |             """Drop subtrees whose root node satisfies the constraints."""
230 |             if not self._test(node, tag, name, attrs):
231 |                 yield node
232 |         
233 |         return dom.alter(drop)
234 | 
235 | 
236 |     def _constraint(self, tag, attr, value, attrs):
237 |         """Normalization of constraints for select() and skip()."""
238 |         
239 |         name = None
240 |         if tag:
241 |             if isinstance(tag, str):
242 |                 name = tag
243 |                 tag  = None
244 |         if attr is not None:
245 |             attrs[attr] = value
246 |             
247 |         return tag, name, attrs
248 |         
249 |     def _test(self, node, tag, name, attrs):
250 |         """Returns True if the `node` satisfies all of the constraints."""
251 |         
252 |         if tag  is not None and node.tag is not tag: return False
253 |         if name is not None and (node.tag is None or node.tag.name != name): return False
254 |         
255 |         for a, v in attrs.items():
256 |             if a not in node.kwattrs: return False
257 |             if v is not DOM.ATTR_DEFINED and node.kwattrs[a] != v: return False
258 | 
259 |         return True
260 |     
261 |         
262 |     # Selectors TODO:
263 |     # - https://github.com/scrapy/cssselect (Scrapy converts all CSS selectors to XPath)
264 |     
265 | 
266 |     ####################################################################################################################
267 |     #####
268 |     #####  DOM nodes
269 |     #####
270 |     
271 |     class Node:
272 |         """"""
273 |         
274 |         tag     = None      # Tag instance whose expand() will be called to post-process the body, in a non-terminal node
275 |         attrs   = None      # list of positional (unnamed) attributes to be passed to tag.expand() during rendering
276 |         kwattrs = None      # dict of keyword (named) attributes to be passed to tag.expand()
277 |         
278 |         body    = None      # DOM (possibly empty) containing child nodes of a non-terminal node; None in DOM.Text
279 |         
280 |         outline = False     # True/False denotes an "outline" block or an "inline" node; adds a leading newline during rendering if True
281 |         indent  = None      # indentation string of this block: absolute (when starts with \n) or relative
282 |                             # to its parent (otherwise); None means this is an inline (headline) block, no indentation
283 |     
284 |         def __init__(self, body = None, indent = None, **params):
285 |             
286 |             # assign secondary parameters
287 |             for name, value in params.items():
288 |                 setattr(self, name, value)
289 |                 
290 |             # assign a list of body nodes, with flattening of nested lists and filtering of None's
291 |             self.body = DOM(body)
292 |             
293 |             # assign indentation, with proper handling of absolute (in parent) vs. relative (in children) indentations
294 |             self.set_indent(indent)
295 |             
296 |             # assert not self.tag or isinstance(self.tag, Tag)
297 |             
298 |         def __getitem__(self, attr):
299 |             """Return value of a keyword attribute `attr` if present. Raise an exception otherwise. Shortcut for self.kwattrs[attr]."""
300 |             return self.kwattrs[attr]
301 |             
302 |         def get(self, attr, default = None):
303 |             """Return value of a keyword attribute `attr` if present, or `default` otherwise. Shortcut for self.kwattrs.get(attr, default)."""
304 |             return self.kwattrs.get(attr, default) if self.kwattrs else default
305 |             
306 |         def set_outline(self, outline = True):
307 |             self.outline = outline
308 |     
309 |         def set_indent(self, indent):
310 |             """
311 |             Sets absolute indentation on self. This calls relative_indent() on all children
312 |             to make their indentations relative to the parent's.
313 |             """
314 |             self.indent = indent
315 |             if self.indent is not None:
316 |                 for child in self.body:
317 |                     child.relative_indent(self.indent)
318 |     
319 |         def relative_indent(self, parent_indent):
320 |             """
321 |             Convert self.indent from absolute to relative by subtracting `parent_indent`.
322 |             If this node is inline (indent=None), the method is called recursively on child nodes.
323 |             """
324 |             if self.indent is None:
325 |                 for child in self.body: child.relative_indent(parent_indent)
326 |             elif self.indent[:1] == '\n':
327 |                 assert self.indent.startswith(parent_indent)
328 |                 self.indent = self.indent[len(parent_indent):]
329 |             else:
330 |                 pass        # self.indent is relative already
331 |                 
332 |         def render(self):
333 |             
334 |             text = self.outline * '\n' + self._render_body()
335 |             
336 |             # if self.outline and self.indent:
337 |             if self.indent is not None:
338 |                 assert self.indent[:1] != '\n'                      # self.indent must have been converted already to relative
339 |                 text = add_indent(text, self.indent)                # indentation is only added where \n is present, the 1st line is left untouched!
340 |                 # text = text.replace('\n', '\n' + self.indent)
341 |                 
342 |             return text
343 |         
344 |         def _render_body(self):
345 |             
346 |             if self.tag:
347 |                 return self.tag.expand(self.body, self.attrs or (), self.kwattrs or {})
348 |             else:
349 |                 return self.body.render()
350 |             
351 |         def tree(self, indent = '', step = '  '):
352 |             r"""Return a multiline \n-terminated string that presents this node's structure as a tree."""
353 |             
354 |             name = self.tag.name if self.tag else "<%s>" % self.__class__.__name__
355 |             heading = str(name)
356 |             
357 |             for attr in self.attrs or []:
358 |                 heading += " %s" % attr
359 |             for key, attr in (self.kwattrs or {}).items():
360 |                 heading += " %s=%s" % (key, attr)
361 |                 
362 |             return indent + heading + '\n' + self.body.tree(indent + step, step)
363 |         
364 |         def copy(self):
365 |             """Deep copy of self, with an exception for attributes, which are shallow-copied, and self.tag (no copy)."""
366 |             
367 |             dup = copy(self)
368 |             dup.attrs = copy(self.attrs)
369 |             dup.kwattrs = copy(self.kwattrs)
370 |             
371 |             if self.body is not None:
372 |                 dup.body = self.body.copy()
373 |             
374 |             return dup
375 |             
376 |         
377 |         def walk(self, skip = None, order = 'preorder'):
378 |             """
379 |             Generator of all nodes inside the tree rooted at self: parent nodes and descendants.
380 |             Parents are yielded before descendants if order='preorder' (default), or after descendants if order='postorder'.
381 |             An optional argument `skip` is a function that takes a Node instance and returns True if the subtree
382 |             rooted at this node should be omitted, False otherwise.
383 |             """
384 |             if skip is not None and skip(self): return
385 |             if order == 'preorder': yield self
386 |             if self.body:
387 |                 yield from self.body.walk(skip, order)
388 |                 # for node in self.body.walk(skip, order): yield node
389 |             if order == 'postorder': yield self
390 | 
391 |         def alter(self, transform, skip = None, order = 'preorder'):
392 |             """Transforms a list of child nodes by calling alter() on self.body. See DOM.alter() for details."""
393 |             
394 |             if skip is not None and skip(self): return self
395 |             if self.body: self.body.alter(transform, skip, order)
396 |             return self
397 |             
398 | 
399 |     class Root(Node):
400 |         """Root node of a Hypertag DOM tree."""
401 |     
402 |         def render(self, drop_line = True):
403 |             """
404 |             If this Root node represents an entire translated document that was originally fed to a Hypertag parser,
405 |             an extra empty line have been prepended by the parser in Grammar.preprocess() and should be removed now
406 |             - set drop_line=True (default) to perform this correction or drop_line=False to skip it.
407 |             """
408 |             output = super(DOM.Root, self).render()
409 |             return output[1:] if drop_line and output.startswith('\n') else output
410 |             # if not output or not drop_line: return output
411 |             # assert output[0] == '\n'
412 |             # return output[1:]
413 |             
414 |     
415 |     class Text(Node):
416 |         """A leaf node containing plain text."""
417 |         
418 |         text = None         # text of this node, either plain text or markup after preprocessing; consists of two parts:
419 |                             #  1) headline (head) - 1st line of `text`, without trailing newline
420 |                             #  2) tailtext (tail) - all lines after the 1st one including the leading newline (!);
421 |                             #     tailtext may contain trailing newline(s), but this is not obligatory
422 |         
423 |         def __init__(self, text = '', **kwargs):
424 |             super(DOM.Text, self).__init__(text = text, **kwargs)
425 |         
426 |         def __str__(self):
427 |             return self.text
428 |             
429 |         def set_indent(self, indent):
430 |             self.indent = indent
431 |     
432 |         def _render_body(self):
433 |             return self.text
434 |     
435 |         # def indent(self, spaces = 1, gap = 0, re_lines = re.compile(r'^(\s*\n)+|\s+$')):
436 |         #     """
437 |         #     Like self.text, but with leading/trailing empty lines removed and indentation fixed at a given number of `spaces`.
438 |         #     Optionally, a fixed number (`gap`) of empty lines are added at the beginning.
439 |         #     """
440 |         #     text = self.text
441 |         #     text = re_lines.sub('', text)           # strip leading and trailing empty lines
442 |         #
443 |         #     # replace current indentation with a `spaces` number of spaces; existing tabs treated like a single space
444 |         #     lines  = text.splitlines()
445 |         #     indent = ' ' * spaces
446 |         #     offset = min(len(line) - len(line.lstrip()) for line in lines if line.strip())
447 |         #     text   = '\n'.join(indent + line[offset:] for line in lines)
448 |         #     return gap * '\n' + text
449 |         
450 |         
451 | 


--------------------------------------------------------------------------------
/src/hypertag/core/errors.py:
--------------------------------------------------------------------------------
 1 | from hypertag.nifty.parsing import ParserError
 2 | 
 3 | ########################################################################################################################################################
 4 | 
 5 | class HError(ParserError):
 6 |     def make_msg(self, msg):
 7 |         if not self.pos: return
 8 |         
 9 |         # convert `line`, `column` to coordinates of the original script from before INDENT/DEDENT encoding
10 |         if self.node and self.column is not None:
11 |             prefix  = self.node.fulltext[:self.pos[0]]
12 |             symbols = self.node.tree.parser.symbols
13 |             indents = prefix.count(symbols['INDENT_S']) + prefix.count(symbols['INDENT_T'])
14 |             dedents = prefix.count(symbols['DEDENT_S']) + prefix.count(symbols['DEDENT_T'])
15 |             indent  = indents - dedents
16 |             column  = indent + self.column
17 |         else:
18 |             column  = None
19 |         
20 |         line = self.line - 1 if self.line else self.line
21 |         
22 |         if self.node and self.node.tree.filename:
23 |             return msg + " in '%s', line %s, column %s (%s)" % (self.node.tree.filename, line, column, self.text)
24 |         else:
25 |             return msg + " at line %s, column %s (%s)" % (line, column, self.text)
26 | 
27 | 
28 | # class HBaseException(BaseException):
29 | #     """
30 | #     Base class for exceptions that play internal role during execution of a Hypertag AST.
31 | #     Their occurrence does NOT signal an error, rather they are intended to be caught by Hypertag internal code
32 | #     at a higher level in the tree.
33 | #     """
34 | #
35 | # class HBreak(HBaseException):
36 | #     pass
37 | 
38 | 
39 | ########################################################################################################################################################
40 | 
41 | class SyntaxErrorEx(HError, SyntaxError):               pass
42 | class ValueErrorEx(HError, ValueError):                 pass
43 | class TypeErrorEx(HError, TypeError):                   pass
44 | class NameErrorEx(HError, NameError):                   pass
45 | class UnboundLocalEx(HError, UnboundLocalError):        pass
46 | class UndefinedTagEx(HError):                           pass
47 | class NotATagEx(HError):                                pass
48 | class NoneStringEx(HError):                             pass
49 | class ImportErrorEx(HError, ImportError):               pass
50 | class ModuleNotFoundEx(ImportErrorEx):                  pass
51 | class ParseErrorEx(HError):                             pass
52 | 
53 | class FalseValueEx(HError):
54 |     """
55 |     Empty (false) value was returned by an expression marked with "!" (obligatory) qualifier.
56 |     Typically, this exception is caught at a higher level with a "try" block.
57 |     """
58 |     
59 | class VoidTagEx(HError):
60 |     """Raised when non-empty body is passed to a void tag (i.e., a tag that doesn't accept body)."""
61 |     
62 | 


--------------------------------------------------------------------------------
/src/hypertag/core/grammar.py:
--------------------------------------------------------------------------------
  1 | """
  2 | @author:  Marcin Wojnarski
  3 | """
  4 | 
  5 | from __future__ import unicode_literals
  6 | 
  7 | 
  8 | grammar = r"""
  9 | 
 10 | ###  Before the grammar is applied, indentation in the input text must be converted into
 11 | ###  "indent" and "dedent" special characters: INDENT_S/DEDENT_S for spaces, INDENT_T/DEDENT_T for tabs.
 12 | ###  These characters must be unique and not occur anywhere else in input text,
 13 | ###  except for places where they get inserted during conversion.
 14 | 
 15 | ###  DOCUMENT
 16 | 
 17 | document         =  core_blocks? margin?
 18 | 
 19 | tail_blocks      =  (indent_s core_blocks dedent_s) / (indent_t core_blocks dedent_t)
 20 | core_blocks      =  tail_blocks / block+
 21 | 
 22 | block            =  margin_out (modifier ws)? (block_control / block_def / block_context / block_import / block_struct / block_comment / special_tag)
 23 | modifier         =  dedent / append
 24 | dedent           =  '<'                         # marks the block shall be dedented by one level (to parent's indentation)
 25 | append           =  '...'                       # marks the block is a continuation and should be appended to a previous block without a newline (no top margin, inline mode)
 26 | 
 27 | ###  CONTROL BLOCKS
 28 | 
 29 | block_control    =  block_assign / block_expr / block_if / block_try / block_for / block_while
 30 | 
 31 | block_try        =  try_long / try_short
 32 | try_long         =  'try' generic_control? clause_else*
 33 | try_short        =  '?' ws (block_struct / body_control)?       # short version of "try" block:  ?tag ... or ?|...
 34 | 
 35 | block_expr       =  mark_eval expr_assign
 36 | block_assign     =  mark_eval ws targets ws op_inplace? '=' expr_assign
 37 | expr_assign      =  ws (expr_augment / embedding) inline_comment?
 38 | op_inplace       =  ~"//|\%%|<<|>>|[-+*/&|^]"
 39 | 
 40 | block_while      =  'while' clause_if clause_else?
 41 | block_for        =  'for' space targets space 'in' space tail_for clause_else?
 42 | block_if         =  'if' clause_if (nl 'elif' clause_if)* clause_else?
 43 | 
 44 | clause_if        =  space tail_if
 45 | clause_else      =  nl 'else' generic_control?
 46 | 
 47 | targets          =  target (comma target)* (ws ',')?            # result object must be unpacked whenever at least one ',' was parsed
 48 | target           =  ('(' ws targets ws ')') / var_def           # left side of assignment: a variable, or a tuple of variables/sub-tuples
 49 | var_def          =  name_id ''                                  # definition (assignment) of a variable
 50 | 
 51 | tail_for         =  (expr_factor generic_control) / (expr_augment body_control?)
 52 | tail_if          =  (expr_factor generic_control) / (expr         body_control?)         # expr may contain hanging operators (/|) hence it can't be followed by generic_control which may contain /| as a text block marker
 53 | 
 54 | ###  DEFINITION BLOCKS
 55 | 
 56 | block_def        =  mark_def ws name_id attrs_def generic_struct
 57 | attrs_def        =  (space attr_body)? (space attr_def)*
 58 | 
 59 | attr_body        =  mark_embed ws name_id
 60 | attr_def         =  name_id (ws '=' ws value_of_attr)?
 61 | 
 62 | block_context    =  'context' space cntx_import (comma cntx_import)* inline_comment?
 63 | block_import     =  'from' space path_import space 'import' space item_import (comma item_import)* inline_comment?
 64 | path_import      =  ~"[^\s\x22\x27]+"                           # import path can be ANY string of 1+ characters unless it contains a whitespace, ' or "
 65 | item_import      =  wild_import / name_import
 66 | wild_import      =  '*'
 67 | cntx_import      =  symbol rename?
 68 | name_import      =  symbol rename?                              # imported name must always be prepended with percent or $ to denote whether we load it from (and save into) a tag namespace or a variable namespace
 69 | rename           =  space 'as' space name_id
 70 | 
 71 | 
 72 | ###  STRUCTURED BLOCK
 73 | 
 74 | block_struct     =  (tags_expand generic_struct) / body_text    # text block is a special case of a structural block (!), in this case block_struct gets
 75 |                                                                 # reduced after parsing to the underlying block_verbat/_normal/_markup
 76 | tags_expand      =  null / (tag_expand (ws mark_struct ws tag_expand)*)
 77 | tag_expand       =  name_id attrs_val
 78 | 
 79 | special_tag      =  pass ''   #/ break_tag / continue_tag
 80 | 
 81 | null             =  '.'
 82 | pass             =  'pass'
 83 | 
 84 | #break_tag       =  'break'
 85 | #continue_tag    =  'continue'
 86 | 
 87 | 
 88 | ###  HEAD, BODY
 89 | 
 90 | generic_control  =  (ws body_text) / body_control               # like body_control, but additionally allows full-text body
 91 | generic_struct   =  (ws body_text) / body_struct                # like body_struct, but additionally allows full-text body
 92 | 
 93 | body_control     =  (ws mark_struct inline_comment? tail_blocks?) / (inline_comment? tail_blocks)
 94 | body_struct      =  (ws mark_struct)? ((ws headline) / inline_comment)? tail_blocks?       # this rule matches empty string '' (!)
 95 | 
 96 | body_text        =  block_verbat / block_normal / block_markup / block_embed
 97 | headline         =  head_verbat / head_normal / head_markup
 98 | 
 99 | head_verbat      =  mark_verbat gap? line_verbat?
100 | head_normal      =  mark_normal gap? line_normal?
101 | head_markup      =  mark_markup gap? line_markup?
102 | 
103 | ###  TEXT BLOCKS, TAIL, LINE
104 | 
105 | block_embed      =  mark_embed ws expr inline_comment?          # @... embedding of a DOM fragment; not strictly a text block, but is treated as such to allow inline placement after a tag:  TAG @body ... @ body.child[0]
106 | block_comment    =  mark_comment line_verbat? tail_verbat?
107 | inline_comment   =  ws mark_comment verbatim?                   # inline (end-line) comment; full-line comments are parsed as block_comment
108 | 
109 | block_verbat     =  mark_verbat line_verbat? tail_verbat?
110 | block_normal     =  mark_normal line_normal? tail_normal?
111 | block_markup     =  mark_markup line_markup? tail_markup?
112 | 
113 | tail_verbat      =  (indent_s core_verbat dedent_s) / (indent_t core_verbat dedent_t)
114 | tail_normal      =  (indent_s core_normal dedent_s) / (indent_t core_normal dedent_t)
115 | tail_markup      =  (indent_s core_markup dedent_s) / (indent_t core_markup dedent_t)
116 | 
117 | core_verbat      =  (tail_verbat / (margin line_verbat))+
118 | core_normal      =  (tail_normal / (margin line_normal))+
119 | core_markup      =  (tail_markup / (margin line_markup))+
120 | 
121 | line_verbat      =  verbatim ''
122 | line_normal      =  line_markup ''                              # same as line_markup during parsing, but renders differently (performs HTML-escaping)
123 | line_markup      =  (escape / embedding / text)+                # line of plain text with {...} or $... expressions; no HTML-escaping during rendering
124 | 
125 | mark_struct      =  ':'
126 | mark_verbat      =  '!'
127 | mark_normal      =  '|'
128 | mark_markup      =  '/'
129 | 
130 | mark_def         =  '%%'                                        # double percent means single percent, only we need to escape for grammar string formatting
131 | mark_eval        =  '$'
132 | mark_embed       =  '@'
133 | mark_comment     =  ~"--|#"
134 | 
135 | gap              =  ~"[ \t]"                                    # 1-space leading gap before a headline, ignored during rendering
136 | 
137 | 
138 | ###  EMBEDDINGS
139 | 
140 | embedding        =  embedding_braces / embedding_eval
141 | embedding_braces =  '{' ws expr_augment ws '}' qualifier?
142 | embedding_eval   =  mark_eval !mark_eval expr_var
143 | 
144 | 
145 | ###  ATTRIBUTES of tags
146 | 
147 | # actual attributes as passed to a tag
148 | attrs_val        =  attr_short* (space (attr_val / attr_short+))*       #/ ws '(' attr_val (',' ws attr_val)* ')'
149 | attr_val         =  attr_named / attr_unnamed
150 | 
151 | attr_short       =  ('.' / '#') (attr_short_lit / embedding)        # shortcuts: .class for class="class", #id for id="id" ... or #{var} or #$var
152 | attr_short_lit   =  ~"[a-z0-9_-]+"i                                 # shortcut literal value MAY contain "-", unlike python identifiers!
153 | attr_named       =  name_xml ws '=' ws value_of_attr                # name="value" OR name=value OR name=$(...)
154 | attr_unnamed     =  value_of_attr ''
155 | value_of_attr    =  embedding / expr_strict
156 | 
157 | ###  ARGUMENTS of functions
158 | 
159 | args             =  arg (comma arg)* (ws ',')?
160 | arg              =  kwarg / expr
161 | kwarg            =  name_id ws '=' ws expr
162 | 
163 | ###  EXPRESSIONS
164 | 
165 | # the expression...
166 | # built bottom-up, starting with inner-most components built of high-priority operators (arithmetic)
167 | # and proceeding outwards, to operators of lower and lower priority (logical);
168 | # after parsing, the expression nodes with only 1 child are reduced (compactified),
169 | # to avoid long (~10 nodes) branches in the syntax tree that don't perform any operations
170 | # other than blindly propagating method calls down to the leaf node.
171 | 
172 | expr         =  expr_root ''                # basic (standard) form of an expression
173 | expr_var     =  factor_var ''               # reduced form of an expression: a variable, with optional trailer; used for inline $... embedding (embedding_eval) and attributes lists
174 | expr_factor  =  factor ''                   # reduced form of an expression: any atom, with optional trailer; used for non-embedded attribute values
175 | expr_strict  =  factor_strict ''            # reduced form of an expression: any atom with a trailer, but no spaces inside (before the trailer)
176 | expr_augment =  expr_tuple / expr_root      # augmented form of an expression: includes unbounded tuples (no parentheses); used in augmented assignments
177 | expr_tuple   =  expr ws ',' (ws expr ws ',')* (ws expr)?      # unbounded tuple, without parentheses ( ); used in `expr_augment` only
178 | expr_bitwise =  or_expr ''                  # reduced form of an expression: only arithmetic and bitwise operators; other ops must be enclosed in (...)
179 | 
180 | subexpr      =  '(' ws expr ws ')'
181 | 
182 | tag_use      =  mark_def name_id ''                           # occurrence (use) of a tag inside an expression
183 | var_use      =  mark_eval? name_id ''                         # occurrence (use) of a variable; $ is allowed inside {...} for convenience
184 | tuple        =  '(' ws ((expr comma)+ (expr ws)?)? ')'
185 | list         =  '[' ws (expr comma)* (expr ws)? ']'
186 | set          =  '{' ws expr (comma expr)* ws (',' ws)? '}'    # obligatory min. 1 element in a set
187 | dict         =  '{' ws (dict_pair comma)* (dict_pair ws)? '}'
188 | dict_pair    =  expr_bitwise ws ':' ws expr                   # here, `expr_bitwise` is used instead of `expr` to avoid ambiguity with pipeline operator
189 | 
190 | atom         =  literal / var_use / tag_use / subexpr / tuple / list / dict / set
191 | factor_var   =  var_use trailer* qualifier?                   # reduced form of `factor`: a variable with optional trailer, no spaces allowed
192 | factor_strict=  atom trailer* qualifier?                      # reduced form of `factor`: no spaces allowed before the trailer
193 | factor_filt  =  atom trailer_filt*                            # reduced form of `factor`: no function call () allowed; for use as filter function in `pipeline`
194 | factor       =  atom (ws trailer)* qualifier?                 # operators: () [] .
195 | 
196 | pow_expr     =  factor (ws op_power ws factor)*
197 | term         =  pow_expr (ws op_multiplic ws pow_expr)*       # operators: * / // percent
198 | arith_expr   =  neg? ws term (ws op_additive ws term)*        # operators: neg + -
199 | shift_expr   =  arith_expr (ws op_shift ws arith_expr)*
200 | 
201 | and_expr     =  shift_expr (ws '&' ws shift_expr)*
202 | xor_expr     =  and_expr (ws '^' ws and_expr)*
203 | or_expr      =  xor_expr (ws '|' ws xor_expr)*
204 | 
205 | pipeline     =  or_expr (ws ':' ws filter)*                   # "pipeline" operator, x:fun(a,b) gets executed as fun(x,a,b)
206 | concat_expr  =  pipeline (space pipeline)*                    # string concatenation: space-delimited list of items
207 | 
208 | comparison   =  concat_expr (ws op_comp ws concat_expr)*
209 | not_test     =  (not space)* comparison                       # spaces are obligatory around: not, and, or, if, else,
210 | and_test     =  not_test (space 'and' space not_test)*        # even if subexpressions are enclosed in (...) - unlike in Python
211 | or_test      =  and_test (space 'or' space and_test)*
212 | ifelse_test  =  or_test (space 'if' space or_test (space 'else' space ifelse_test)?)?       # "else" branch is optional, defaults to None
213 | 
214 | #empty_test  =  ifelse_test op_empty (ifelse_test)?           # test for emptiness (falseness) of 1st operand: if empty, it's replaced with either '' or None, depending on operator: ? or !
215 | 
216 | expr_root    =  ifelse_test ''
217 | 
218 | 
219 | ###  TAIL OPERATORS:  call, slice, member access, filter application, qualifier ...
220 | 
221 | slice_value  =  ws (expr_bitwise ws)?           # empty value '' serves as a placeholder, so that we know which part of *:*:* we're at; expr_bitwise is used to avoid collission with pipeline operator
222 | slice        =  slice_value ':' slice_value (':' slice_value)?
223 | subscript    =  slice / (ws expr_augment ws)
224 | 
225 | call         =  '(' ws (args ws)? ')'
226 | index        =  '[' subscript ']'               # handles atomic indices [i] and all types of [*:*:*] slices
227 | member       =  '.' ws name_id
228 | trailer      =  call / index / member
229 | trailer_filt =  index / member                  # reduced form of `trailer` for use in `factor_filt` and `pipeline`
230 | 
231 | partial_call =  '(' ws (args ws)? ')'
232 | filter       =  factor_filt partial_call?       # no space is allowed between the function and its arguments
233 | 
234 | qualifier    =  ~"[\?!]"                        # ? means that None/empty(false)/exceptions shall be converted to '' ... ! means that empty (false) value triggers exception
235 | # obligatory   =  '!'
236 | # optional     =  '?'
237 | 
238 | 
239 | ###  SIMPLE OPERATORS
240 | 
241 | op_power     =  '**'
242 | neg          =  '-'                             # multiple negation, e.g., "---x", not allowed -- unlike in Python
243 | op_multiplic =  '*' / '//' / '/' / '%%'         # double percent means single percent, only we need to escape for grammar string formatting
244 | op_additive  =  '+' / '-'
245 | op_shift     =  '<<' / '>>'
246 | op_empty     =  '?' / '!'
247 | 
248 | not          =  'not'
249 | op_comp      =  ~"==|!=|<>|>=|<=|<|>|not\s+in|is\s+not|in|is"
250 | 
251 | ###  IDENTIFIERS
252 | 
253 | symbol           =  (mark_def / mark_eval) name_id
254 | name_id          =  !name_reserved ~"[a-z_][a-z0-9_]*"i
255 | name_reserved    =  ~"(from|import|if|else|elif|try|for|while|break|continue|pass|is|in|not|and|or)\\b"     # names with special meaning inside expressions, disallowed for hypertags & variables; \\b is a regex word boundary and is written with double backslash bcs single backslash-b is converted to a backspace by Python
256 | 
257 | # names allowed in XML, defined liberally, with nearly all characters allowed to match all valid HTML/XML identifiers (esp. attributes);
258 | # EXCEPTION: colon ':' is NOT allowed as the 1st or the last character, to avoid collision with a trailing ":" used in blocks
259 | name_xml         =  ~"[%(XML_StartChar)s]([%(XML_Char)s]*[%(XML_EndChar)s])?"i
260 | 
261 | 
262 | ###  ATOMS
263 | 
264 | literal          =  number / string / boolean / none
265 | 
266 | number_signed    =  ~"[+-]?" number
267 | number           =  ~"((\.\d+)|(\d+(\.\d*)?))([eE][+-]?\d+)?"      # the leading +- is added during expression construction (<neg>)
268 | #string           =  ~"'[^']*'" / ~'"[^"]*"'         # '...' or "..." string; no escaping of ' and " inside!
269 | boolean          =  'True' / 'False'
270 | none             =  'None'
271 | 
272 | string           =  string_format / string_raw
273 | string_raw       =  ~"r'[^']*'" / ~'r"[^"]*"'         # '...' or "..." string; must not contain ' or " inside (no escaping available)
274 | 
275 | string_format    =  string_quot1 / string_quot2
276 | string_quot1     =  "'" (escape / embedding / text_quot1)* "'"
277 | string_quot2     =  '"' (escape / embedding / text_quot2)* '"'
278 | 
279 | 
280 | ###  BASIC TOKENS
281 | 
282 | escape      =  ~'\$\$|{{|}}'                 # escape sequences: $$ {{ }}
283 | 
284 | verbatim    =  ~"[^%(INDENT_S)s%(DEDENT_S)s%(INDENT_T)s%(DEDENT_T)s\n]+"su       # 1 line of plain text, may include special symbols (left unparsed)
285 | text        =  ~"[^%(INDENT_S)s%(DEDENT_S)s%(INDENT_T)s%(DEDENT_T)s\n${}]+"su    # 1 line of plain text, excluded $ { }
286 | text_quot1  =  ~"[^%(INDENT_S)s%(DEDENT_S)s%(INDENT_T)s%(DEDENT_T)s\n${}']+"su   # 1 line of plain text, excluded $ { } '
287 | text_quot2  =  ~'[^%(INDENT_S)s%(DEDENT_S)s%(INDENT_T)s%(DEDENT_T)s\n${}"]+'su   # 1 line of plain text, excluded $ { } "
288 | 
289 | indent_s    = "%(INDENT_S)s"
290 | dedent_s    = "%(DEDENT_S)s"
291 | indent_t    = "%(INDENT_T)s"
292 | dedent_t    = "%(DEDENT_T)s"
293 | 
294 | margin_out  =  nl ''                         # margin that preceeds an outlined block; its trailing \n will be moved out into the block before rendering
295 | margin      =  nl ''                         # top margin of a block; same as `nl` in grammar, but treated differently during analysis (`nl` is ignored)
296 | nl          =  ~"\n+"                        # vertical space = 1+ newlines
297 | 
298 | comma       =  ws ',' ws
299 | space       =  ~"[ \t]+"                     # obligatory whitespace, no newlines
300 | ws          =  ~"[ \t]*"                     # optional whitespace, no newlines
301 | 
302 | ###  SYMBOLS that mark TYPES of blocks or text spans
303 | 
304 | """
305 | 
306 | #####################################################################################################################################################
307 | #####
308 | #####  DEFINITIONS for use together with the grammar
309 | #####
310 | 
311 | MARK_TAG = '%'
312 | MARK_VAR = '$'
313 | 
314 | def TAG(name):
315 |     """Convert a tag name to a symbol, for insertion to (and retrieval from) a Context."""
316 |     return MARK_TAG + name
317 | 
318 | def VAR(name):
319 |     """Convert a variable name to a symbol, for insertion to (and retrieval from) a Context."""
320 |     return MARK_VAR + name
321 | 
322 | def TAGS(names):
323 |     """Mapping of a dict of tag names (and their linked objects) to a dict of symbols."""
324 |     return {MARK_TAG + name: link for name, link in names.items()}
325 | 
326 | def VARS(names):
327 |     """Mapping of a dict of variable names (and their linked objects) to a dict of symbols."""
328 |     return {MARK_VAR + name: link for name, link in names.items()}
329 | 
330 | def IS_TAG(symbol):
331 |     return symbol.startswith(MARK_TAG)
332 | 
333 | def IS_VAR(symbol):
334 |     return symbol.startswith(MARK_VAR)
335 | 
336 | 


--------------------------------------------------------------------------------
/src/hypertag/core/run_html.py:
--------------------------------------------------------------------------------
 1 | from hypertag.core.runtime import Runtime
 2 | from hypertag.std.html import html_escape
 3 | import hypertag.html
 4 | 
 5 | 
 6 | ########################################################################################################################################################
 7 | #####
 8 | #####  HTML runtime
 9 | #####
10 | 
11 | 
12 | class HyperHTML(Runtime):
13 |     
14 |     language = 'HTML'
15 |     escape   = staticmethod(html_escape)
16 |     BUILTINS = Runtime.BUILTINS + ['hypertag.html']
17 |     


--------------------------------------------------------------------------------
/src/hypertag/core/runtime.py:
--------------------------------------------------------------------------------
  1 | import os, sys, importlib
  2 | 
  3 | from hypertag.core.errors import ImportErrorEx, ModuleNotFoundEx
  4 | from hypertag.core.grammar import MARK_VAR, MARK_TAG, VAR, TAG, TAGS
  5 | from hypertag.core.ast import HypertagAST
  6 | 
  7 | PATH_SEP = os.path.sep
  8 | 
  9 | 
 10 | #####################################################################################################################################################
 11 | #####
 12 | #####  UTILITIES
 13 | #####
 14 | 
 15 | # set Python's global recursion limit to at least MIN_RECURSION_LIMIT to avoid RecursionError
 16 | # when parsing deeply nested scripts
 17 | MIN_RECURSION_LIMIT = 20000
 18 | sys.setrecursionlimit(max(sys.getrecursionlimit(), MIN_RECURSION_LIMIT))
 19 | 
 20 | 
 21 | def join_path(base, path, sep = '.', ext = None):
 22 |     """
 23 |     Convert a relative import `path` to an absolute one by appending it to an absolute `base` path.
 24 |     The `base` can be any type of path: a file path (slash-separated) or an import path (dot-separated) -
 25 |     the `sep` argument determines what character is treated as a separator; the same character is inserted
 26 |     in the result instead of dots, in the `path` part of the output string. If `ext` is provided,
 27 |     it is appended as a file extension of the output path; it should start with a dot '.'.
 28 |     """
 29 |     if base is None: return None
 30 |     
 31 |     orig_base = base
 32 |     orig_path = path
 33 |     
 34 |     assert path[:1] == '.'                  # `path` is a relative import path
 35 |     path = path[1:]
 36 |     
 37 |     if base[-1:] == sep:                    # truncate a trailing separator in the base path
 38 |         base = base[:-1]
 39 | 
 40 |     while path[:1] == '.':                  # move upwards the package tree if `path` starts with multiple dots
 41 |         if sep not in base:
 42 |             raise ImportErrorEx("attempted relative import (%s) beyond top-level package or folder (%s)" % (orig_path, orig_base))
 43 |         base = base.rsplit(sep, 1)[0]
 44 |         path = path[1:]
 45 | 
 46 |     fullpath = base + sep + path.replace('.', sep)
 47 |     if ext: fullpath += ext
 48 |     
 49 |     return fullpath
 50 | 
 51 | 
 52 | #####################################################################################################################################################
 53 | #####
 54 | #####  MODULES
 55 | #####
 56 | 
 57 | class Module:
 58 |     """Wrapper around all types of imported modules (Hypertag's, Python's)."""
 59 |     
 60 |     runtime  = None     # Runtime that created this module
 61 |     location = None     # canonical path of this module, for deduplication and caching; there can be many
 62 |                         # non-canonical (e.g., relative) paths pointing to the same module, but only one canonical path
 63 |     filename = None     # name of the file if this module was loaded from disk; mapped to __file__ inside a script
 64 |     package  = None     # Python package path of this module's package, if available; mapped to __package__ inside a script
 65 |     symbols  = None     # cached dict of this module's symbols; each symbol has a leading mark % or $
 66 |     
 67 |     def __init__(self, **kwattrs):
 68 |         for attr, value in kwattrs.items():
 69 |             setattr(self, attr, value)
 70 |     
 71 |     def __getitem__(self, key):
 72 |         return self.symbols[key]
 73 |         
 74 |     def get(self, key, default = None):
 75 |         return self.symbols.get(key, default)
 76 |         
 77 | 
 78 | class PyModule(Module):
 79 |     """Wrapper around a regular Python module and its global symbols."""
 80 |     
 81 |     module = None       # the original Python module
 82 |     
 83 |     def __init__(self, module):
 84 |         """
 85 |         Pull symbols (tags & variables) from a Python module and return as a dict.
 86 |         All top-level symbols are treated as variables; tags are pulled from a special module-level dictionary `__tags__`.
 87 |         """
 88 |         self.module   = module
 89 |         self.location = getattr(module, '__name__', None)
 90 |         self.filename = getattr(module, '__file__', None)
 91 |         self.package  = getattr(module, '__package__', None)
 92 |         
 93 |         self.symbols  = {VAR(name) : getattr(module, name) for name in dir(module)}
 94 |         
 95 |         # retrieve __tags__
 96 |         tags = self.symbols.pop(VAR('__tags__'), None)
 97 |         if tags:
 98 |             if not isinstance(tags, dict): raise ImportErrorEx("__tags__ must be a dict in %s" % module)
 99 |             
100 |             # make sure that the tags being imported are assigned to the same names as configured in their Tag.name property
101 |             for name, tag in tags.items():
102 |                 if name != tag.name: raise ImportErrorEx("tag's internal name (%s) differs from its public name (%s) in %s" % (tag.name, name, module))
103 |             
104 |             self.symbols.update({name if name[0] == MARK_TAG else TAG(name) : tag for name, tag in tags.items()})
105 | 
106 | class HyModule(Module):
107 |     """Wrapper around a Hypertag script to be executed (translated). Stores results of translation."""
108 |     
109 |     script  = None      # plain text of the Hypertag script
110 |     dom     = None      # output DOM produced by translation
111 |     symbols = None      # dict of output symbols produced during script parsing, {name: value}
112 |     state   = None      # local State at the end of translation; needed when hypertags from this module are to be expanded
113 |     
114 |     def translate(self, builtins, __tags__, **variables):
115 |     
116 |         ast = HypertagAST(self.script, self)
117 |         self.dom, self.symbols, self.state = ast.translate(builtins, __tags__, **variables)
118 |         return self
119 |         
120 |     
121 | class RootModule(HyModule):
122 |     """Wrapper module for a top-level script, to provide a referrer module for other (imported) scripts."""
123 |     
124 | 
125 | #####################################################################################################################################################
126 | #####
127 | #####  LOADERS
128 | #####
129 | 
130 | class Loader:
131 |     """
132 |     Finding, loading, and caching modules.
133 |     A loader takes an import path as specified in `import` block of a Hypertag script,
134 |     together with the location of the referrer module; converts the path to its canonical form (subclass-dependent);
135 |     loads the module and returns it as a dict of symbols. Caching of modules can be performed internally.
136 |     Each Loader subclass represents a specific naming convention and an ontology of valid import paths.
137 |     """
138 |     cache = None        # dict of cached modules indexed by their canonical path (module.location);
139 |                         # each Loader subclass is responsible for managing the cache by itself
140 |     
141 |     def __init__(self):
142 |         self.cache = {}
143 |     
144 |     def load(self, path, referrer, runtime):
145 |         """
146 |         Try to load a module given its (non-canonical) path and a referrer module, and return a Module instance.
147 |         If the path is invalid or the module can't be found, None is returned.
148 |         If this method is overriden by subclasses, `self.cache` should be checked to avoid repeated loading
149 |         of the same module every time after a `path` is converted to a canonical `location`; every newly loaded module
150 |         should be put into cache.
151 |         """
152 |         location = self._find(path, referrer)
153 |         if location is None: return None
154 |         if location in self.cache: return self.cache[location]
155 |         
156 |         module = self.cache[location] = self._read(location, runtime)
157 |         return module
158 | 
159 |     def _find(self, path, referrer):
160 |         """
161 |         Convert a path to its canonical form. The `referrer` is the module where the path occured.
162 |         Return None if the path is invalid for this type of modules.
163 |         """
164 |         return None
165 | 
166 |     def _read(self, location, runtime):
167 |         """Read a module identified by `location`. Return None if the module is missing."""
168 |         return None
169 |     
170 |     @staticmethod
171 |     def _make_absolute(path, referrer):
172 |         """Utility method for internal purposes: converting a python import path to an absolute path."""
173 |         if path[:1] != '.': return path                 # `path` is absolute, return it unchanged
174 |         return join_path(referrer.package, path)        # `path` is relative, convert it to an absolute one
175 |         
176 |         
177 | class PyLoader(Loader):
178 |     """
179 |     Loader of regular Python modules. They are identified by absolute package paths (dot-separated, no initial dot),
180 |     which are used as modules' unique locations. When importing a Python module from a Hypertag script using
181 |     a RELATIVE package path, the script's module must have its `package` property set, otherwise it will be impossible
182 |     to infer the referred module's package path.
183 |     """
184 | 
185 |     def _find(self, path, referrer):
186 |         """Absolute and relative Python paths are supported. The latter require that referrer's package is set."""
187 |         return self._make_absolute(path, referrer)
188 |         
189 |     def _read(self, location, runtime):
190 |         try:
191 |             module = importlib.import_module(location)
192 |             return PyModule(module)
193 |         
194 |         except ModuleNotFoundError:
195 |             return None
196 | 
197 | 
198 | class HyLoader(Loader):
199 |     """
200 |     Loader of Hypertag scripts located on disk.
201 |     Import paths have the same structure as in Python: with a dot '.' separating subfolders (packages).
202 |     Location of a module is defined as an absolute file system path to the script file.
203 | 
204 |     A relative import path (.XYZ) is converted to a file path rooted at the referrer's module folder
205 |     (referrer.filename must be present) to obtain the location.
206 |     
207 |     For absolute paths, different methods are tried to obtain the location. The first one that succeeds
208 |     (i.e., the file exists) is selected:
209 |     
210 |     1. resolve(import_path, referrer) is called to get a location; `resolve` function is an optional argument
211 |        of HyLoader instantiation;
212 |     2. location is being searched for using Python's import mechanism, like if the script was a Python file - this only
213 |        works when the import path contains a package specifier, which points to a valid Python package (with __init__.py);
214 |     3. location is created as a directory path based at the current folder of the process;
215 |     4. location is created as a directory path based at the referrer's folder (like for relative imports).
216 |     
217 |     Import paths CAN refer to folders which are NOT valid Python packages (don't have __init__.py inside).
218 |     HyLoader does not check the existence of __init__.py on directory paths to scripts.
219 |     """
220 |     
221 |     SCRIPT_EXTENSION = '.hy'         # default file extension of Hypertag scripts
222 | 
223 |     def __init__(self, resolve = None):
224 |         super(HyLoader, self).__init__()
225 |         self.resolve = resolve
226 | 
227 |     def load(self, path, referrer, runtime):
228 |         
229 |         # whenever possible, python package path (python_path) of the module is inferred, so that its `package` can be set
230 |         location = python_path = None
231 |         ref_root = os.path.dirname(referrer.filename) if referrer.filename else None
232 |         
233 |         # relative import path is always resolved relative to the referrer's folder
234 |         if path[:1] == '.':
235 |             if ref_root:
236 |                 location = self._join_path(ref_root, path)
237 |                 python_path = self._make_absolute(path, referrer)
238 |                 
239 |         # absolute import path can be resolved in several different ways; the first one that returns a valid file path is used
240 |         else:
241 |             # 1. try calling a resolve() function
242 |             if self.resolve:
243 |                 location = self.resolve(path, referrer)
244 |                 if location in self.cache: return self.cache[location]
245 |                 if location and not os.path.exists(location):
246 |                     location = None
247 | 
248 |             # 2. try the Python's standard import mechanism (importlib); must be applied to the parent package, not the script itself
249 |             if not location and '.' in path:
250 |                 package_name, filename = path.rsplit('.', 1)
251 |                 pkg = importlib.import_module(package_name)                 # package_name is an absolute path, so referrer is not needed here
252 |                 package_path = pkg.__file__
253 |                 if package_path.endswith('.py'):
254 |                     package_path = os.path.dirname(package_path)            # truncate /__init__.py part of a package file path
255 |                 
256 |                 location = package_path + PATH_SEP + filename + self.SCRIPT_EXTENSION
257 |                 python_path = package_path + '.' + filename
258 |                 if location and not os.path.exists(location):
259 |                     location = python_path = None
260 |                 
261 |             # 3. try using the process' current folder as a root
262 |             if not location:
263 |                 location = self._join_path(os.getcwd(), '.' + path)
264 |                 if location in self.cache: return self.cache[location]
265 |                 if location and not os.path.exists(location):
266 |                     location = None
267 | 
268 |             # 4. try using the referrer's folder as a root
269 |             if not location and ref_root:
270 |                 location = self._join_path(ref_root, '.' + path)
271 |                 python_path = self._make_absolute('.' + path, referrer)
272 | 
273 |         if not location or not os.path.exists(location):
274 |             return None
275 |         
276 |         package = python_path.rsplit('.', 1)[0] if python_path else None
277 |         
278 |         script = open(location).read()
279 |         
280 |         module = self.cache[location] = runtime.translate(script, location, package)
281 |         module.location = location
282 |         module.package = package
283 |         
284 |         return module
285 |         
286 |     
287 |     def _join_path(self, root, path):
288 |         """Convert a python import `path` (with dots) to a file path (with slashes) and append to a root folder's path."""
289 |         
290 |         return join_path(root, path, PATH_SEP, self.SCRIPT_EXTENSION)
291 |         
292 | 
293 | #####################################################################################################################################################
294 | #####
295 | #####  RUNTIME
296 | #####
297 | 
298 | class Runtime:
299 |     """
300 |     Runtime execution environment of Hypertag scripts.
301 |     Defines a set of built-in symbols, a target language, and an escape function.
302 |     Manages modules loading through a collection of predefined loaders.
303 |     Modules are identified by "paths". The meaning of a particular path is determined by its loader.
304 |     """
305 | 
306 |     # list of module paths to be imported automatically into a script upon startup of translation (built-in symbols)
307 |     BUILTINS = ['builtins', 'hypertag.builtins']
308 |     
309 |     # cached dict of built-in symbols, to avoid recalculation for every new AST
310 |     _builtins = None
311 | 
312 |     language = None     # target language the documents will be compiled into, defined in subclasses
313 |     #compact  = True    # if True, compactification is performed after analysis: pure (static, constant) nodes are replaced with their pre-computed render() values,
314 |                         # which are returned on all subsequent render() requests; this improves performance when
315 |                         # a document contains many static parts and variables occur rarely
316 |     escape   = None     # escaping function or static method that converts plaintext to target language; typically, when assigned
317 |                         # in a subclass, staticmethod() must be applied as a wrapper to prevent this attr be treated as a regular method:
318 |                         #   escape = staticmethod(custom_function)
319 | 
320 |     loaders  = None     # list of Module subclasses whose static load() is called in sequence to find the first one
321 |                         # that is able to locate and load a module by a given path
322 |     
323 |     default_loaders = [HyLoader, PyLoader]      # loaders to be used when no others are passed to __init__
324 |     
325 |     
326 |     def __init__(self, loaders = None):
327 |         self.loaders = loaders or self.default_loaders
328 |         self.loaders = [loader if isinstance(loader, Loader) else loader() for loader in self.loaders]
329 | 
330 |     def import_builtins(self):
331 |         """
332 |         Import default symbols that shall be available to every script upon startup.
333 |         This typically means all general-purpose symbols + standard tags/variables specific for a target language.
334 |         """
335 |         if self._builtins is None:
336 |             self._builtins = {}
337 |             for path in self.BUILTINS:
338 |                 module = self.import_module(path, RootModule(), None)
339 |                 self._builtins.update(module.symbols)
340 |                 
341 |         return dict(self._builtins)
342 |         
343 |     def import_module(self, path, referrer, ast_node):
344 |         """Import symbols that are defined in a module identified by `path`. Return as an instance of Module."""
345 |         
346 |         assert path
347 | 
348 |         for loader in self.loaders:
349 |             module = loader.load(path, referrer, self)
350 |             if module: return module
351 | 
352 |         raise ModuleNotFoundEx("import path not found '%s', try setting __package__ or __file__ when calling render()" % path, ast_node)
353 |         
354 |     def translate(self, __script__, __file__ = None, __package__ = None, __tags__ = None, **variables):
355 |         """Parse a given script, translate to a DOM tree, and return wrapped up in a HyModule instance."""
356 |         
357 |         builtins = self.import_builtins()
358 |         builtins[VAR('__file__')]    = __file__
359 |         builtins[VAR('__package__')] = __package__
360 |         
361 |         module = HyModule(runtime = self, script = __script__, filename = __file__, package  = __package__)
362 |         module.translate(builtins, __tags__, **variables)
363 |         
364 |         return module
365 |         
366 |     def render(self, __script__, __file__ = None, __package__ = None, __tags__ = None, **variables):
367 |         
368 |         module = self.translate(__script__, __file__, __package__, __tags__, **variables)
369 |         return module.dom.render()
370 |         
371 | 
372 | 


--------------------------------------------------------------------------------
/src/hypertag/core/structs.py:
--------------------------------------------------------------------------------
  1 | """
  2 | Data structures for semantic analysis of Hypertag's AST.
  3 | """
  4 | 
  5 | import six
  6 | from copy import copy
  7 | from six import text_type as unicode
  8 | 
  9 | if six.PY2:
 10 |     from collections import OrderedDict as odict
 11 | else:
 12 |     odict = dict
 13 |     
 14 | from hypertag.core.errors import UnboundLocalEx, UndefinedTagEx
 15 | from hypertag.core.grammar import MARK_VAR
 16 | 
 17 | 
 18 | ########################################################################################################################################################
 19 | 
 20 | class Stack(list):
 21 |     """
 22 |     Stack of current symbols encountered during semantic analysis. Implementation based on standard <list>.
 23 |     Additionally, it keeps track of current indentation.
 24 |     """
 25 | 
 26 |     # current indentation string, as a combination of ' ' and '\t' characters;
 27 |     # initial \n is used to mark that an indentation is absolute rather than relative to a parent node
 28 |     indentation = '\n'
 29 |     
 30 |     @property
 31 |     def size(self):
 32 |         return len(self)
 33 |     @size.setter
 34 |     def size(self, _size_):
 35 |         del self[_size_:]
 36 | 
 37 |     #push    = list.append
 38 |     pushall  = list.extend
 39 |     get      = list.__getitem__
 40 |     set      = list.__setitem__
 41 |     
 42 |     def push(self, value):
 43 |         """Append `value` to the stack and return its index in the list."""
 44 |         self.append(value)
 45 |         return len(self) - 1
 46 | 
 47 |     def indent(self, whitechar):
 48 |         self.indentation += whitechar
 49 |     
 50 |     def dedent(self, whitechar):
 51 |         assert self.indentation[-1] == whitechar, 'Trying to dedent a different character than was appended'
 52 |         self.indentation = self.indentation[:-1]
 53 |     
 54 |     # def next_slot(self):
 55 |     #     """
 56 |     #     Index in the list of symbols where a new symbol will be inserted if push() is called right after.
 57 |     #     This method can be merged with push() in the future.
 58 |     #     """
 59 |     #     return len(self)
 60 |     
 61 |     def position(self):
 62 |         """
 63 |         Position in a Stack that can be passed to reset().
 64 |         NOTE: indentation is NOT included and must always be reset manually through dedent()
 65 |         """
 66 |         return len(self) #, len(self.indentation)
 67 | 
 68 |     def reset(self, position):
 69 |         "If anything was added on top of the stack, reset the stack to a previous position and forget those elements."
 70 |         # if len(self) < position: raise Exception("Stack.reset(), can't return to a point (%s) that is higher than the current size (%s)" % (position, self.size))
 71 |         del self[position:]
 72 |         # pos_symbols, pos_indent = position
 73 |         # self.indentation = self.indentation[:pos_indent]
 74 |         # del self[pos_symbols:]
 75 |         
 76 |     def copy(self):
 77 |         dup = Stack(self)
 78 |         dup.indentation = self.indentation
 79 |         return dup
 80 |         
 81 | 
 82 | class MultiDict(object):
 83 |     """
 84 |     An ordered multi-dictionary with push/pop operations. Or, in other words, a stack of (name,value) pairs
 85 |     that additionally keeps a dict of the names and their most recent values, for fast name lookup.
 86 |     Each element of the stack keeps also an index of the previous element with the same name,
 87 |     to enable pop() implementation that replaces a dictionary value of a given name with its previous value.
 88 |     
 89 |     >>> md = MultiDict()
 90 |     >>> md.push('a', 123); md.push('b', 321); md.push('a', 9)
 91 |     >>> md.pop()
 92 |     ('a', 9)
 93 |     >>> md['a']
 94 |     123
 95 |     >>> md.push('b', 456); md.push('b', 654)
 96 |     >>> md.reset(md.position() - 2)
 97 |     >>> md['b']
 98 |     321
 99 |     """
100 |     
101 |     stack  = None           # Stack object; elements are triples of the form: (name, value, index_of_previous)
102 |     lookup = None           # {name: index} dict of current names and their most recent positions in the stack
103 |     
104 |     def __init__(self, maxlen = 10):
105 |         self.stack = Stack()
106 |         self.lookup = {}
107 |     
108 |     def __contains__(self, name):
109 |         return name in self.lookup
110 |     
111 |     def __getitem__(self, name):
112 |         """Current value assigned to `name`."""
113 |         idx = self.lookup[name]
114 |         return self.stack[idx][1]
115 |     
116 |     def get(self, name, default = None):
117 |         idx = self.lookup.get(name, None)
118 |         if idx is None: return default
119 |         return self.stack[idx][1]
120 |     
121 |     def keys(self): return self.lookup.keys()
122 |     def values(self):
123 |         for i in self.lookup.values(): yield self.stack[i][1]
124 |     def items(self):
125 |         for k in self.lookup.keys(): yield k, self[k]
126 |         
127 |     def push(self, name, value):
128 |         prev = self.lookup.get(name, None)
129 |         self.lookup[name] = self.stack.size
130 |         self.stack.push((name, value, prev))
131 |         
132 |     def pop(self):
133 |         "Pop the top element off the stack and return as a (name, value) pair, with proper update of the lookup dictionary."
134 |         top = self.stack.pop()
135 |         name, value, prev = top
136 |         if prev is None: del self.lookup[name]
137 |         else: self.lookup[name] = prev
138 |         return (name, value)
139 |     
140 |     def pushall(self, symbols):
141 |         """A repeated push() of all symbols from a dictionary."""
142 |         for name, value in symbols.items():
143 |             self.push(name, value)
144 | 
145 |     def pushnew(self, symbols):
146 |         """A conditional push() of symbols from a dictionary: only the symbols not present yet in self.stack are pushed."""
147 |         for name, value in symbols.items():
148 |             if name not in self.lookup: self.push(name, value)
149 | 
150 |     def position(self):
151 |         """Does NOT include self.stack's indentation, which must be reset manually through dedent()."""
152 |         return self.stack.position()
153 | 
154 |     def reset(self, position):
155 |         """
156 |         Same as repeated pop(), but a bit more efficient.
157 |         Does NOT reset self.stack's indentation, which must be reset manually through dedent().
158 |         """
159 |         size = self.stack.size
160 |         if size < position: raise Exception("MultiDict.reset(), can't return to a point (%s) that is higher than the current size (%s)" % (position, size))
161 |         for i in range(size-1, position-1, -1):
162 |             name, _, prev = self.stack[i]
163 |             if prev is None: del self.lookup[name]
164 |             else: self.lookup[name] = prev
165 |         self.stack.size = position
166 | 
167 |     def asdict(self, start_position = 0):
168 |         """Return all current symbols and their values as an ordered dict, starting at `start_position`."""
169 |         rng = range(start_position, self.stack.size)
170 |         return odict((name, value) for name, value, _ in [self.stack[i] for i in rng])
171 | 
172 |     def indent(self, whitechar):
173 |         self.stack.indent(whitechar)
174 | 
175 |     def dedent(self, whitechar):
176 |         self.stack.dedent(whitechar)
177 | 
178 |     def copy(self):
179 |         dup = copy(self)
180 |         dup.stack = self.stack.copy()
181 |         dup.lookup = self.lookup.copy()
182 |         return dup
183 |     
184 |     def __unicode__(self): return unicode(self.lookup)
185 |     def __repr__(self):
186 |         items = [(repr(self.stack[i][0]), self.stack[i][1]) for i in sorted(self.lookup.values())]
187 |         return "{%s}" % ', '.join("%s: %s" % item for item in items)
188 | 
189 | 
190 | class Context(MultiDict):
191 |     """Context data passed through a HyperML syntax tree during semantic analysis."""
192 |     
193 |     depth = 0               # current depth of nested hypertag definitions during analyse()
194 |     ref_depth = None        # the top-most (minimum as a value) def-depth of a non-pure variable or hypertag referenced inside the current subtree;
195 |                             # this value is passed bottom-up through the tree, from inner nodes (variable occurences)
196 |                             # to outer nodes (hypertag def-nodes); def-depth = -1 indicates an external non-pure variable/hypertag
197 |     
198 |     # depth counters, below, are incremented/decremented directly by corresponding nodes during analyse()
199 |     hypertag_depth = 0      # no. of (nested) hypertag definition nodes on the path from tree root to the current node;
200 |                             # for analysing which hypertags are pure (always produce the same output) and can be compacted
201 |     control_depth  = 0      # no. of control instructions (if/for/try) on the path from tree root to the current node;
202 |                             # hypertag definitions are disallowed inside control blocks
203 |     regular_depth  = 0      # no. of tags or tag-def nodes (control nodes excluded) on the path from tree root,
204 |                             # or from the closest nesting hypertag definition node, to the current node;
205 |                             # sibling nodes located at the same regular_depth share a namespace
206 | 
207 |     in_prolog = True        # True inside the document prolog, i.e., from the document beginning till the first block
208 |                             # different than a context specification or a comment
209 |     
210 |     def add_refdepth(self, d, symbol = None):
211 |         """Update self.ref_depth with the depth of one more definition of a variable/hypertag.
212 |         'symbol': optional name of the variable/hypertag being referenced, for debugging.
213 |         """
214 |         if d is None: return                            # special case, for adding back initial ref_depth value, which can be None
215 |         if self.ref_depth is None:                      # when ref_depth is still uninitialized, just use 'd'
216 |             self.ref_depth = d
217 |         else:                                           # otherwise, use 'd' if it's lower than the current value
218 |             self.ref_depth = min(self.ref_depth, d)
219 |         #if DEBUG: print(' ', symbol or '', d, 'ref_depth =', self.ref_depth)
220 | 
221 | 
222 | ########################################################################################################################################################
223 | 
224 | # class Frame(object):
225 | #     """Activation frame. A list of actual values of hypertag/function arguments, plus values of local variables.
226 | #     Parent-linked to the caller's activation frame and linked through *access link* to the frame of the immediate lexical encapsulating hypertag.
227 | #     Through the parent link, frames create an *activation list*, a counterpart of a stack in traditional runtime implementations.
228 | #     Activation list enables easy creation of a parent-pointer-tree for representation of execution of closures (hypertags passed as variables):
229 | #     https://en.wikipedia.org/wiki/Parent_pointer_tree
230 | #     """
231 | #     name = None             # name of the hypertag that created this frame, for debugging
232 | #
233 | #     vars = []               # the fixed stack of variables' values
234 | #     parent = None
235 | #     accesslink = None
236 | 
237 | 
238 | # class StackBranch(object):
239 | #     """A stack that exposes Stack interface, but internally consists of 2 disjoint parts, each one being a Stack object:
240 | #     1) the "lower" Stack or StackBranch object (the "trunk") containing stack of a fixed length - can only be used for read access;
241 | #     2) the "upper" Stack object (the "branch") that initially has 0 length, but can grow and shrink like any regular Stack
242 | #     StackBranch is used in Closure implementation.
243 | #     """
244 | #     def __init__(self, lower):
245 | #         self.lower = lower
246 | #         self.upper = Stack()
247 | #         self.lowersize = self.lower.size
248 | #
249 | #     ### all Stack methods delegated to the appropriate Stack object...
250 | #
251 | #     def push(self, x):
252 | #         self.upper.push(x)
253 | #     def pop(self):
254 | #         return self.upper.pop()
255 | #     def pushall(self, elems):
256 | #         self.upper.pushall(elems)
257 | #
258 | #     def __getitem__(self, pos):
259 | #         if pos < self.lowersize:
260 | #             return self.lower[pos]
261 | #         return self.upper[pos - self.lowersize]
262 | #     def get(self, pos):
263 | #         if pos < 0:
264 | #             pos += self.lowersize + self.upper.size
265 | #         return self[pos]
266 | #     def set(self, pos, val):
267 | #         # transform 'pos' to a positive index in the 'upper' stack
268 | #         if pos < 0:
269 | #             upos = pos + self.upper.size
270 | #         else:
271 | #             upos = pos - self.lowersize
272 | #         if upos < 0:
273 | #             raise Exception("StackBranch.set(), can't modify an element (#%s) that's located on the trunk (size %s)" % (pos, self.lowersize))
274 | #         self.upper.set(upos, val)
275 | #
276 | #     def position(self):
277 | #         return self.lowersize + self.upper.size
278 | #     def reset(self, position):
279 | #         "If anything was added on top of the stack, reset the top position to a previous state and forget those elements."
280 | #         size = self.lowersize + self.upper.size
281 | #         if position > size:
282 | #             raise Exception("StackBranch.reset(), can't return to a point (%s) that is higher than the current size (%s)" % (position, size))
283 | #         if position < self.lowersize:
284 | #             raise Exception("StackBranch.reset(), can't return to a point (%s) that is located on the trunk (size %s)" % (position, self.lowersize))
285 | #         self.upper.reset(position - self.lowersize)
286 | #
287 | #     def __repr__(self):
288 | #         return repr(self.lower) + '/' + repr(self.upper)
289 | #
290 | #
291 | # class Closure(object):
292 | #     """
293 | #     Frozen hypertag expansion, created when a hypertag is passed as a variable for later execution: $H ... $fun(H).
294 | #     Closure keeps the stack as was present at the point of closure creation, to pass it to the hypertag's expand()
295 | #     even if the original stack has changed before the point of expansion.
296 | #     Note that in HyML, hypertags can only be passed downwards through the chain of hypertag expansions or function calls,
297 | #     not upwards, so the stack can only grow, never shrink, before the closure execution point, and consequently
298 | #     all the frames that the closure needs are still on the stack. However, the closure may need to extend the stack in its own way
299 | #     starting from the stack position from the point of closure creation, which would override the frames added between closure creation
300 | #     and execution. To avoid this, we have to implement *stack branching*, done by temporary re-mapping of stack indices
301 | #     to "hide" a part of the stack without its physical removal. The hidden part can be unhidden in constant time, and there can be many
302 | #     overlapping parts hidden at the same time (happens when a chain of several closures are executed).
303 | #     """
304 | #     ishypertag = None       # the HypertagSpec
305 | #     hypertag = None         # the xhypertag node to be expanded when the request comes
306 | #     stack = None            # current stack from the point of closure creation
307 | #     occurDepth = None       # call depth at the point of closure creation
308 | #
309 | #     def __init__(self, hypertag, stack, occurDepth):
310 | #         self.ishypertag = hypertag.ishypertag
311 | #         self.hypertag = hypertag
312 | #         self.occurDepth = occurDepth
313 | #         self.stack = StackBranch(stack)
314 | #         #self.stack = stack.copy()
315 | #
316 | #     def definition(self): return self.hypertag.definition()
317 | #
318 | #     def expand(self, unnamed, kwattrs, caller = None):
319 | #         # now we use the recorded stack and depth from the place of closure creation, instead of the ones from the point of expansion
320 | #         return self.hypertag.expand(unnamed, kwattrs, self.stack, self.occurDepth, caller)
321 | #
322 | #
323 | # class LazyVariable(object):
324 | #     """
325 | #     A variable whose value calculation is delayed until the first use.
326 | #     If the variable is never used, its value won't be calculated at all.
327 | #     The value is calculated from a "value function" set during initialization and
328 | #     should be retrieved with getvalue() method. Repeated calls to getvalue() return
329 | #     the same value as calculated on the 1st call, without further re-calculation.
330 | #     """
331 | #     def __init__(self, fun):
332 | #         self.fun = fun          # no-arg "value function"; will be called to calculate the actual value of the variable
333 | #         self.value = None
334 | #         self.hasvalue = False
335 | #         #self.getvalue()
336 | #
337 | #     def getvalue(self):
338 | #         if not self.hasvalue:
339 | #             self.value = self.fun()
340 | #             self.hasvalue = True
341 | #         return self.value
342 | #
343 | # lazyEmptyString = LazyVariable(lambda: '')
344 | # lazyEmptyString.getvalue()
345 | 
346 | 
347 | ########################################################################################################################################################
348 | #####
349 | #####  STATE
350 | #####
351 | 
352 | # UNDEFINED = ('UNDEFINED',)      # token that marks a variable hasn't been assigned a value, yet; similar to Python:
353 | #                                 # UnboundLocalError: local variable 'x' referenced before assignment
354 | 
355 | class State:
356 |     """
357 |     State of translation/rendering, as a dict of Slots and their current values.
358 |     Substitute for a Stack when keeping a history of frames is not necessary and values can be indexed
359 |     by nodes of an AST instead of integer positions in a stack. State class resembles Context more than Stack.
360 |     """
361 |     values = None       # dict of (slot, value) pairs
362 |     
363 |     # current indentation string, as a combination of ' ' and '\t' characters;
364 |     # initial \n is used to mark that an indentation is absolute rather than relative to a parent node
365 |     indentation = '\n'
366 |     
367 |     def __init__(self, values = None):
368 |         self.values = values.copy() if values else {}   #odict()
369 | 
370 |     def __contains__(self, slot):
371 |         return slot in self.values
372 | 
373 |     def __getitem__(self, slot):
374 |         return self.values[slot]
375 | 
376 |     def __setitem__(self, slot, value):
377 |         #assert slot not in self.values     # this assert is not true inside "for" loops
378 |         self.values[slot] = value
379 |         
380 |     def copy(self):
381 |         dup = State()
382 |         dup.values = self.values.copy()
383 |         return dup
384 |         
385 |     def update(self, values):
386 |         self.values.update(values)
387 |         
388 |     def indent(self, whitechar):
389 |         self.indentation += whitechar
390 |     
391 |     def dedent(self, whitechar):
392 |         assert self.indentation[-1] == whitechar, 'Trying to dedent a different character than was appended'
393 |         self.indentation = self.indentation[:-1]
394 |     
395 |     # def savepoint(self):
396 |     #     return len(self.values)
397 |     #
398 |     # def rollback(self, savepoint):
399 |     #     """
400 |     #     Does NOT rollback indentation, which must always be reset manually through dedent().
401 |     #     """
402 |     #     size = len(self.values)
403 |     #     if size == savepoint: return
404 |     #     if size < savepoint: raise Exception("State.rollback(), can't rollback to a point (%s) that is higher than the current size (%s)" % (savepoint, size))
405 |     #     for _ in range(size - savepoint):
406 |     #         self.values.popitem()
407 |     
408 |     
409 | class Slot:
410 |     """
411 |     Representation of a variable or a tag for assignments.
412 |     Slot instances are used as keys inside `state` during translation to uniquely identify variables and tags.
413 |     Slots do NOT hold values by themselves (!), they only indicate where to save a current value inside `state`.
414 |     
415 |     Created during analysis of definition / assignment / import blocks; provides global identification
416 |     of a symbol inside `state` during translation; enables correct name scoping and dynamic re-assignment of the actual
417 |     value or reference during translate(), so that imports and hypertag definitions can be placed inside
418 |     control blocks (i.e., dynamic name resolution of hypertags and imported symbols is possible).
419 |     """
420 |     name    = None
421 |     symbol  = None
422 |     
423 |     primary = None      # 1st definition of this symbol, if self represents a re-assignment (override)
424 |     depth   = None      # ctx.regular_depth of this slot, for correct identification of re-assignments that occur
425 |                         # at the same depth (in the same namespace)
426 |     
427 |     def __init__(self, symbol, ctx):
428 |         assert len(symbol) >= 2 and symbol[0] in '$%'
429 |         self.name   = symbol[1:]
430 |         self.symbol = symbol
431 |         self.depth  = ctx.regular_depth
432 | 
433 |         link = ctx.get(symbol)
434 |         
435 |         from .ast import NODES
436 |         assert link is None or isinstance(link, (Slot, NODES.attribute, NODES.variable)), 'unrecognized type of slot: %s' % type(link)
437 |         
438 |         # if link and link.depth == self.depth:
439 |         #     self.primary = link
440 |         # else:
441 |         ctx.push(symbol, self)
442 |         
443 |     def set(self, state, value):
444 |         state[self.primary or self] = value
445 |         
446 |     def get(self, state):
447 |         """Raises KeyError if this slot is uninitialized in `state`."""
448 |         return state[self.primary or self]
449 |     
450 | 
451 | class ValueSlot(Slot):
452 |     """
453 |     A slot that remembers its (constant) initial value, as assigned by the creator node, and writes it into `state`
454 |     on every call to set_value(). NOTE: the value of this slot in `state` can be overwritten by other nodes later on,
455 |     just like for any Slot!
456 |     """
457 | 
458 |     value = None
459 |     
460 |     def __init__(self, symbol, value, ctx):
461 |         super(ValueSlot, self).__init__(symbol, ctx)
462 |         self.value = value
463 | 
464 |     def set_value(self, state):
465 |         self.set(state, self.value)
466 | 
467 | 
468 | 


--------------------------------------------------------------------------------
/src/hypertag/core/tag.py:
--------------------------------------------------------------------------------
  1 | from six import text_type
  2 | from xml.sax.saxutils import quoteattr
  3 | 
  4 | from hypertag.core.errors import VoidTagEx, TypeErrorEx
  5 | from hypertag.core.dom import DOM
  6 | 
  7 | 
  8 | ########################################################################################################################################################
  9 | #####
 10 | #####  TAG
 11 | #####
 12 | 
 13 | class Tag:
 14 |     """
 15 |     Base class for custom tags defined as a Python functions (external tags).
 16 |     Instances can be imported to a Hypertag script and used as tags.
 17 |     """
 18 |     name = None         # [str] name that indentifies this tag in a DOM and can be used in DOM selectors
 19 |     
 20 |     pure = True         # if True, the tag is assumed to always return the same result for the same arguments (no side effects),
 21 |                         # which enables caching of expand() calls and compactification of DOM nodes tagged with this tag
 22 |     
 23 |     def expand(self, body, attrs, kwattrs):
 24 |         """
 25 |         Subclasses should NOT append trailing \n nor add extra indentation during tag expansion
 26 |         - both things are added by the parser later on, if desired so by programmer.
 27 |         
 28 |         :param body: DOM of a translated body of tag occurrence, possibly empty; if a tag is void and does not expect a body,
 29 |                      but a non-empty body was passed, the tag should raise VoidTagEx
 30 |         :param attrs: list/tuple of positional attributes
 31 |         :param kwattrs: dict of keyword attributes; attributes are guaranteed to have valid XML names,
 32 |                         but NOT necessarily valid Python names
 33 |         :return: a string
 34 |         """
 35 |         raise NotImplementedError
 36 | 
 37 |     def __call__(self, body_string = None, *attrs, **kwattrs):
 38 |         """
 39 |         Wrapper around expand() to let the tag be used like a function and to allow its direct rendering,
 40 |         without an intermediate DOM. This method can be used as a shortcut inside Hypertag expressions
 41 |         to skip DOM creation entirely.
 42 |         """
 43 |         body = DOM.Text(text_type(body_string)) if body_string is not None else DOM()
 44 |         return self.expand(body, attrs, kwattrs)
 45 |         
 46 | 
 47 | ########################################################################################################################################################
 48 | 
 49 | class TagFunction(Tag):
 50 |     """A wrapper that creates a Tag instance from a function. The `body` attribute is passed as a string to the function."""
 51 |     
 52 |     def __init__(self, fun, dom = False):
 53 |         self.fun  = fun
 54 |         self.name = fun.__name__
 55 |         self.dom  = dom
 56 |     
 57 |     def expand(self, body, attrs, kwattrs):
 58 |         if not self.dom: body = body.render()
 59 |         return self.fun(body, *attrs, **kwattrs)
 60 |         
 61 | 
 62 | class Null(Tag):
 63 |     """Null tag '.' is represented in the DOM tree. Its expand() passes the body unchanged."""
 64 |     
 65 |     name = 'null'
 66 |     
 67 |     def expand(self, body, attrs, kwattrs):
 68 |         return body.render()
 69 |     
 70 | null = Null()
 71 | 
 72 | 
 73 | ########################################################################################################################################################
 74 | #####
 75 | #####  STANDARD MARKUP TAG
 76 | #####
 77 | 
 78 | class Markup(Tag):
 79 |     """
 80 |     A hypertag whose expand() outputs the body unchanged, surrounded by <name>...</name> strings, with proper handling
 81 |     of void tags <name /> and HTML/XHTML format differences for boolean attributes.
 82 |     This class is used for all built-in (X)HTML tags. It can also be used to define custom markup tags in an application.
 83 |     """
 84 |     
 85 |     name = None         # tag <name> to be printed into markup; may differ from the Hypertag name used inside a script (!)
 86 |     void = False        # True if this tag is void: the `body` in expand() should be empty and the element is rendered as self-closing: <NAME />
 87 |     mode = 'HTML'       # (X)HMTL compatibility mode: either 'HTML' or 'XHTML'
 88 |     
 89 |     def __init__(self, name = None, void = False, mode = 'HTML'):
 90 |         if name: self.name = name
 91 |         self.void = void
 92 |         self.mode = mode
 93 |     
 94 |     def expand(self, body, attrs, kwattrs):
 95 |         
 96 |         if attrs: raise TypeErrorEx("markup tag '%s' does not accept positional attributes" % self.name)
 97 |         return self._expand(self.name, body, kwattrs)
 98 |         
 99 |     def _expand(self, name, body, kwattrs):
100 |         
101 |         # render attributes
102 |         kwattrs = filter(None, map(self._render_attr, kwattrs.items()))
103 |         tag = ' '.join([name] + list(kwattrs))
104 | 
105 |         body = body.render()
106 |         
107 |         # render output
108 |         if self.void:
109 |             if body: raise VoidTagEx("non-empty body passed to a void markup tag '%s'" % name)
110 |             return "<%s />" % tag
111 |         else:
112 |             # if the block contains a headline, the closing tag is placed on the same line as body;
113 |             # a newline is added at the end, otherwise
114 |             nl = '\n' if body[:1] == '\n' else ''
115 |             return "<%s>" % tag + body + nl + "</%s>" % name
116 | 
117 |     def _render_attr(self, name_value):
118 |         
119 |         name, value = name_value
120 |         if value is True:               # name=True   -- converted to:  name (HTML)  or  name="name" (XHTML)
121 |             if self.mode == 'HTML':
122 |                 return name
123 |             else:
124 |                 return '%s="%s"' % (name, name)
125 |         if value is False:              # name=False  -- removed from attr list
126 |             return None
127 |         
128 |         value = quoteattr(str(value))        # escaping of <,>,&," characters; string wrapped up in "..." or '...'
129 |         # value = str(value)
130 |         # if '"' not in value:
131 |         #     value = '"%s"' % value
132 |         # elif "'" not in value:
133 |         #     value = "'%s'" % value
134 |         # else:
135 |         #     value = quoteattr(value)    # escaping of <,>,&," chars is performed ONLY when the value contains a quote "
136 |         
137 |         return '%s=%s' % (name, value)
138 |         
139 | 
140 | 


--------------------------------------------------------------------------------
/src/hypertag/core/xml_chars.py:
--------------------------------------------------------------------------------
 1 | ########################################################################################################################################################
 2 | ###
 3 | ###  Regex patterns for character sets allowed in XML identifiers, to be put inside [...] in a regex.
 4 | ###  XML identifiers differ substantially from typical name patterns in other computer languages. Main differences:
 5 | ###   1) national Unicode characters are allowed, specified by ranges of unicode point values
 6 | ###   2) special characters are allowed:  ':' (colon) '.' (dot) '-' (minus)
 7 | ###      Colon is allowed as the 1st character according to XML syntax spec., although such a name may be treated as malformed during semantic analysis.
 8 | ###      Others (dot, minus), are allowed on further positions in the string, after the 1st character.
 9 | ###  Specification: http://www.w3.org/TR/REC-xml/#NT-NameStartChar
10 | ###
11 | 
12 | # human-readable:  [:_A-Za-z] | [\u00C0-\u00D6] | [\u00D8-\u00F6] | [\u00F8-\u02FF] | [\u0370-\u037D] | [\u037F-\u1FFF] | [\u200C-\u200D] | [\u2070-\u218F] | [\u2C00-\u2FEF] | [\u3001-\uD7FF] | [\uF900-\uFDCF] | [\uFDF0-\uFFFD] | [\U00010000-\U000EFFFF]
13 | XML_StartChar  =  r"_A-Za-z\u00C0-\u00D6\u00D8-\u00F6\u00F8-\u02FF\u0370-\u037D\u037F-\u1FFF\u200C-\u200D\u2070-\u218F\u2C00-\u2FEF\u3001-\uD7FF\uF900-\uFDCF\uFDF0-\uFFFD\U00010000-\U000EFFFF"
14 | 
15 | # human-readable:  XML_StartChar | [0-9.\u00B7-] | [\u0300-\u036F] | [\u203F-\u2040]
16 | XML_Char       =  XML_StartChar + r"0-9\.\-" + r"\u00B7\u0300-\u036F\u203F-\u2040" + ":"
17 | 
18 | # XML_EndChar is like XML_Char, but ":" is NOT allowed, to avoid collision with a trailing ":" used in Hypertag blocks
19 | XML_EndChar    =  XML_Char[:-1]
20 | 
21 | 


--------------------------------------------------------------------------------
/src/hypertag/django/__init__.py:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/mwojnars/hypertag/8cd989127c12d78f6902d4b3dc972de31a3e88d9/src/hypertag/django/__init__.py


--------------------------------------------------------------------------------
/src/hypertag/django/backend.py:
--------------------------------------------------------------------------------
 1 | """
 2 | Hypertag's template processing backend for Django.
 3 | """
 4 | 
 5 | from django.template import Origin, TemplateDoesNotExist, TemplateSyntaxError
 6 | from django.template.backends.base import BaseEngine
 7 | from django.template.backends.utils import csrf_input_lazy, csrf_token_lazy
 8 | 
 9 | from hypertag import HyperHTML
10 | 
11 | 
12 | 
13 | class Hypertag(BaseEngine):
14 |     """
15 |     Hypertag's template processing backend for Django. Produces HTML5 documents.
16 |     This class should be installed in TEMPLATES, in Django project's settings.py:
17 | 
18 |     TEMPLATES = [
19 |         {
20 |             'BACKEND': 'hypertag.django.backend.Hypertag',
21 |             'DIRS': [],
22 |             'APP_DIRS': True,
23 |             'OPTIONS': {},
24 |         },
25 |         # ... other engines here ...
26 |     ]
27 |     """
28 |     
29 |     # Name of the subdirectory containing the templates for this engine
30 |     # inside an installed application.
31 |     app_dirname = 'hypertag'
32 | 
33 | 
34 |     def __init__(self, params):
35 |         # print('setting up Hypertag backend...')
36 |         params  = params.copy()
37 |         options = params.pop('OPTIONS').copy()
38 |         super().__init__(params)
39 |         
40 |         # `options` are ignored currently
41 |         self.engine = HyperHTML()  #**options
42 | 
43 |     def from_string(self, template_code):
44 |         
45 |         return Template(self.engine, template_code)
46 | 
47 |     def get_template(self, template_name):
48 |         
49 |         tried = []
50 |         for template_file in self.iter_template_filenames(template_name):
51 |             try:
52 |                 script = open(template_file, 'rt').read()
53 |                 return Template(self.engine, script, template_file)
54 | 
55 |             except FileNotFoundError:
56 |                 tried.append((
57 |                     Origin(template_file, template_name, self),
58 |                     'Source does not exist',
59 |                 ))
60 | 
61 |         raise TemplateDoesNotExist(template_name, tried = tried, backend = self)
62 | 
63 | 
64 | class Template:
65 | 
66 |     def __init__(self, engine, view, filename = None):
67 |         self.engine = engine
68 |         self.view = view
69 |         self.filename = filename
70 | 
71 |     def render(self, context=None, request=None):
72 |         if context is None:
73 |             context = {}
74 |         if request is not None:
75 |             context['request'] = request
76 |             context['csrf_input'] = csrf_input_lazy(request)
77 |             context['csrf_token'] = csrf_token_lazy(request)
78 |         if self.filename and '__file__' not in context:
79 |             context['__file__'] = self.filename
80 |         
81 |         return self.engine.render(self.view, **context)
82 |     
83 |     


--------------------------------------------------------------------------------
/src/hypertag/django/filters.py:
--------------------------------------------------------------------------------
 1 | """
 2 | Django standard filters for use with Hypertag.
 3 | A complete list and documentation of filters:  https://docs.djangoproject.com/en/3.1/ref/templates/builtins/#built-in-filter-reference
 4 | 
 5 | All standard Django filters are loaded to this module's namespace, so that they can be imported into other files
 6 | and used as functions or Hypertag filters (with a pipeline operator), for example:
 7 | 
 8 |     from hypertag.django.filters import slugify
 9 |     from hypertag.django.filters import dictsort
10 | 
11 | OR, if imported in a Hypertag script:
12 | 
13 |     from hypertag.django.filters import $slugify
14 | 
15 | """
16 | 
17 | import django as __django__
18 | from django.conf import settings as __settings__
19 | from django.template.defaultfilters import register as __register_default__
20 | from django.contrib.humanize.templatetags.humanize import register as __register_humanize__
21 | 
22 | ##
23 | ## for now, Django's default settings are always used in those filters that depend on them
24 | ##
25 | 
26 | __settings__.configure()
27 | __django__.setup()
28 | 
29 | 
30 | globals().update(__register_humanize__.filters)
31 | globals().update(__register_default__.filters)
32 | 
33 | del globals()['__django__']
34 | del globals()['__settings__']
35 | del globals()['__register_default__']
36 | del globals()['__register_humanize__']
37 | 


--------------------------------------------------------------------------------
/src/hypertag/html.py:
--------------------------------------------------------------------------------
 1 | #
 2 | # Hypertag built-ins for HTML.
 3 | #
 4 | # HTML tags can be imported manually into a Hypertag script with a statement of the form:
 5 | #
 6 | #    from hypertag.html import %div
 7 | #
 8 | 
 9 | from hypertag.std.html import register
10 | 
11 | __tags__ = register.tags                # make all tags available for import
12 | globals().update(register.vars)         # make all variables available for import
13 | 
14 | del globals()['register']
15 | 


--------------------------------------------------------------------------------
/src/hypertag/nifty/__init__.py:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/mwojnars/hypertag/8cd989127c12d78f6902d4b3dc972de31a3e88d9/src/hypertag/nifty/__init__.py


--------------------------------------------------------------------------------
/src/hypertag/nifty/parsing.py:
--------------------------------------------------------------------------------
  1 | # -*- coding: utf-8 -*-
  2 | '''
  3 | COPIED FROM "nifty" TOOLBOX with MODIFICATIONS:  https://github.com/mwojnars/nifty/blob/master/parsing/parsing.py
  4 | 
  5 | LICENSE: MIT
  6 | '''
  7 | 
  8 | import copy
  9 | from .util import isstring, escape, flatten
 10 | 
 11 | 
 12 | ########################################################################################################################################################
 13 | ###
 14 | ###  UTILITIES
 15 | ###
 16 | 
 17 | class ParserError(Exception):
 18 |     
 19 |     MAXLEN = 20
 20 |     pos    = None       # (start,end) position where the error occured, or None
 21 |     line   = None       # derived from 'pos'
 22 |     column = None       # derived from 'pos'
 23 |     text   = None       # initial MAXLEN characters of the input fragment that caused the error
 24 |     node   = None
 25 | 
 26 |     def __init__(self, msg, node = None, cause = None):
 27 |         if node:
 28 |             self.node, self.pos, self.text = node, node.pos, node.text(self.MAXLEN)
 29 |         if self.pos:                                            # calculate the line number and column of self.pos, if possible
 30 |             prefix = node.fulltext[:self.pos[1]]
 31 |             self.line = prefix.count('\n') + 1
 32 |             self.column = len(prefix) - prefix.rfind('\n')
 33 |         msg = self.make_msg(msg) or msg
 34 |         if cause: msg += " because of %s: %s" % (type(cause).__name__, cause)
 35 |         super(ParserError, self).__init__(msg)
 36 |         
 37 |     def make_msg(self, msg):
 38 |         if self.pos:
 39 |             return msg + " at line %s, column %s (%s)" % (self.line, self.column, self.text)
 40 | 
 41 | 
 42 | ########################################################################################################################################################
 43 | ###
 44 | ###  TREE
 45 | ###
 46 | 
 47 | class Tree(object):
 48 |     """Semantic tree. Produced as a result of rewriting of the original raw AST generated by parser.
 49 |     In contrast to raw AST, which always comprises nodes of the same type, no matter what language is parsed,
 50 |     this Tree comprises custom domain-specific nodes which implement semantic analysis specific to a given language.
 51 |     That's why rewriting from AST to Tree is needed.
 52 |     Subclasses of Tree should contain inner node classes named after the names of non-terminals of the grammar, 
 53 |     precedeed by leading 'x' to differentiate node classes from Tree methods and attributes.
 54 |     Instead of inner classes, you can use methods.
 55 |     """
 56 |     parser  = None              # instance of AST parser class produced by parser generator; only one method is used: parse(text) -> AST
 57 |     #Context = None              # Context subclass that shall be used in semantic analysis
 58 | 
 59 |     _use_init = True            # only set to True in old code that uses init(...) with arguments; otherwise set False in subclasses and override setup() in nodes
 60 | 
 61 |     # the sets below can be given as a space-separated string of names instead of a set
 62 |     _ignore_  = set()           # set of non-terminals that should be ignored during rewriting (pruned from the tree)
 63 |     _reduce_  = set()           # set of non-terminals that should be replaced with a list of their children
 64 |     _compact_ = set()           # set of non-terminals that should be replaced with their child if there is exactly 1 child AFTER rewriting of all children
 65 | 
 66 |     text    = None              # full text of the input string fed to the parser
 67 |     ast     = None              # raw AST generated by the parser; for read access by the client
 68 |     root    = None              # root node of the final tree after rewriting
 69 |     
 70 |     def __init__(self, text = None, ast = None, stopAfter = None):
 71 |         """Build Tree, either from input 'text' (will be parsed to raw AST and then rewritten to Tree), 
 72 |         or from raw AST 'ast' (only rewriting will be done).
 73 |         """
 74 |         def _split(names):
 75 |             if isstring(names): names = names.split()
 76 |             if isinstance(names, list): names = set(names)
 77 |             return names
 78 | 
 79 |         self._ignore_  = _split(self._ignore_)
 80 |         self._reduce_  = _split(self._reduce_)
 81 |         self._compact_ = _split(self._compact_)
 82 |         
 83 |         self.text = text
 84 |         self.ast = self.parse(text) if text else ast                # parse input text to raw AST; keep AST for reference by the client
 85 |         if stopAfter == "parse": return
 86 |         
 87 |         self.root = self.rewrite(self.ast)                          # rewrite AST to a tree of <Tree.node> nodes
 88 |         if not self.root: return
 89 |         
 90 |         self.root._enrich()                                         # enrich nodes with basic semantic information
 91 |         
 92 |         # bottom-up initialization of nodes by calls to custom init(), no arguments
 93 |         if not self._use_init:
 94 |             self.root._setup_all()
 95 | 
 96 |     def __str__(self):
 97 |         "For printing of the tree. Walks the tree via node.children, collects info() lines and concatenates them with proper indentation."
 98 |         lines = self.info(self.root)
 99 |         return '\n'.join(flatten(lines))
100 |     
101 |     @staticmethod
102 |     def info(node, depth = 0):
103 |         prefix = '  ' * depth
104 |         if isstring(node): return prefix + node
105 |         return prefix + node.info(), [Tree.info(n, depth+1) for n in node.children]
106 | 
107 |     def parse(self, text):
108 |         "Parses raw text (string) into a syntax tree built of custom node classes (self.Tree). Can be overridden in subclasses."
109 |         return self.parser.parse(text)
110 | 
111 |     def rewrite(self, astroot):
112 |         "Recursively convert a node of raw AST and all its subtree into appropriate subclasses of ``Tree.node``. Generator-specific."
113 |         raise NotImplementedError()
114 |     
115 |     def _rewriteNode(self, astnode):
116 |         "Rewrites contents of a single node to corresponding Tree.node attributes: pos, type, children. Generator-specific."
117 |         raise NotImplementedError()
118 |     
119 |     def compile(self):                                                                                                          #@ReservedAssignment
120 |         "Full compilation, with postprocessing. Can be overriden in subclasses, but usually overriding postprocess() or analyse() is enough."
121 |         data = self.analyse()                                   # run top-down semantic analysis of the tree
122 |         code = self.root.compile()                              # generate output code
123 |         return self.postprocess(code, data)
124 | 
125 |     def analyse(self):
126 |         ctx = self.root.analyse(self.Context())
127 |         return ctx.data
128 | 
129 |     def postprocess(self, code, data):
130 |         "Override in subclasses to perform custom postprocessing of the compiled code and possibly change the return type, e.g., avoid returning 'data'."
131 |         return code, data
132 |         
133 |     ### Base node classes
134 |     
135 |     class node(object):
136 |         """Generic recursive node: passes all str() and compile() requests down to children, then concatenates returned strings.
137 |         Base class for all nodes in rewritten trees.
138 |         """
139 |         
140 |         tree     = None
141 |         astnode  = None                     # original AST node this node was created from
142 |         fulltext = None                     # full text of the input string fed to the parser; see text()
143 |         pos      = None                     # (start,end) positions of the substring represented by this node in 'fulltext'
144 |         type     = None                     # name of non-terminal that produced this node, [str]   @ReservedAssignment
145 |         display  = ""                       # string representation of this node for debugging purposes
146 |         
147 |         children = []                       # list of child nodes after rewriting
148 |         
149 |         # enriched information
150 |         parent = None                       # parent node; None if root
151 |         sibling_prev = None                 # older sibling (before this node) AFTER rewriting
152 |         sibling_next = None                 # younger sibling (after this node) AFTER rewriting
153 |         
154 |         @property
155 |         def line(self):
156 |             """
157 |             Line number in `fulltext` where this node's match begins.
158 |             NOTE: this method scans all fulltext up to self.pos, which can be costly.
159 |             """
160 |             prefix = self.fulltext[:self.pos[0]]
161 |             return prefix.count('\n') + 1
162 | 
163 |         @property
164 |         def column(self):
165 |             prefix = self.fulltext[:self.pos[0]]
166 |             return len(prefix) - prefix.rfind('\n')
167 | 
168 |         def __init__(self, tree, astnode): 
169 |             self.tree = tree
170 |             self.astnode = astnode
171 |             self.fulltext = tree.text
172 |             self.pos, self.type, self.children = tree._rewriteNode(astnode)
173 |             if tree._use_init:
174 |                 self.init(tree, astnode)
175 |             
176 |         def init(self, tree = None, astnode = None):
177 |             """
178 |             Subclasses should override this method (args can be dropped!) instead of __init__,
179 |             so that standard initialization is still performed beforehand, without explicit super() call.
180 |             DEPRECATED. In new code, override setup() instead.
181 |             """
182 |         def setup(self):
183 |             """
184 |             Like init(), but doesn't take arguments. Can be overriden in subclasses to perform
185 |             custom initialization of a node after standard rewriting and enrichment.
186 |             In new code, setup() should be overriden instead of init(...), with Tree._use_init = False.
187 |             """
188 |             
189 |         def _setup_all(self):
190 |             """Recursive bottom-up initialization of nodes with calls to custom setup() methods, no arguments (!)."""
191 |             for child in self.children: child._setup_all()
192 |             self.setup()
193 | 
194 |         def text(self, maxlen = None):
195 |             """The substring of source text matched by this node, or its leading 'maxlen' characters if maxlen != None. 
196 |             None if this node was derived from other nodes during tree post-processing."""
197 |             if not self.pos: return None
198 |             end = self.pos[1] if maxlen is None else min(self.pos[1], self.pos[0] + maxlen)
199 |             ret = self.fulltext[ self.pos[0] : end ]
200 |             if end < self.pos[1]: return ret + "..."
201 |             return ret
202 |     
203 |         def info(self):
204 |             #d = self.__dict__.copy(); del d['fulltext']
205 |             ds = "" #" %s" % d
206 |             return "%s%s at position %s matching: %s" % (self.infoName(), ds, self.pos, escape(str(self.text())))
207 |         def infoName(self):
208 |             # override in subclasses to customize info() printout
209 |             return "<%s>" % (self.type or self.__class__.__name__)
210 |         
211 |         def __str__(self): 
212 |             return self.display or ''.join(str(c) for c in self.children)
213 |         
214 |         def child(self, idx):
215 |             "Returns child with a given index, or None if missing."
216 |             return self.children[idx] if len(self.children) > idx else None
217 |         
218 |         def _enrich(self):
219 |             """Recursively enrich nodes with basic semantic information. Called right after rewriting."""
220 |             
221 |             prev = None
222 |             for child in self.children:
223 |                 child.parent = self
224 |                 child.sibling_prev = prev
225 |                 if prev: prev.sibling_next = child
226 |                 prev = child
227 | 
228 |             for child in self.children:
229 |                 child._enrich()
230 | 
231 |         def analyse(self, ctx):
232 |             """Top-down semantic analysis of the tree. 'ctx' is the opening Context (pre-context) of the current node. 
233 |             Returns closing context (post-context) of the node, which may be the same object 'ctx' (!), but doesn't have to.  
234 |             Parent nodes shall NEVER assume that 'ctx' doesn't get modified when passed down to a child.
235 |             If this object is going to be used again after passing to a child (even if it's only passing to another child!), 
236 |             a copy should be passed instead of the original.
237 |             """
238 |             for c in self.children: c.analyse(ctx.copy())
239 |             return ctx
240 | 
241 |         def compile(self): #@ReservedAssignment
242 |             "Top-down compilation of the tree. By default, returns concatenation of strings compiled by children nodes."
243 |             return ''.join(c.compile() for c in self.children)
244 | 
245 |     class virtual(node):
246 |         """A node that's artificially created after the tree is parsed, 
247 |         having no corresponding AST node nor a position in the source text."""
248 |         def __init__(self, tree):
249 |             "No 'astnode' argument, we only keep the tree reference."
250 |             self.tree = tree
251 |     
252 |     class static(node):
253 |         """Static string (or value of another type) that can be inferred already during parsing (node initialization) and then returned 
254 |         in str() and compile(). By default, the value is made up in __init__ by merging strings from all children."""
255 |         typecast = str              # type or method that will be applied to the parsed string to convert it to a value of appropriate type
256 |         
257 |         def __init__(self, tree, astnode, string = None):
258 |             "Additional argument 'string' is accepted, not present in base 'node'. It's used as a node value if children produce empty string."
259 |             self.tree = tree
260 |             self.fulltext = tree.text
261 |             self.pos, self.type, children = tree._rewriteNode(astnode) if astnode else (None, None, [])
262 |             string = ''.join(str(c) for c in children) or string or ''
263 |             self.value = self.typecast(string)
264 |             if tree._use_init:
265 |                 self.init(tree, astnode)
266 |         
267 |         def __str__(self): return str(self.value)
268 |         def compile(self): return self.value                                                        #@ReservedAssignment
269 |     
270 |     class const(node):
271 |         "A node that always generates the same output: cls.value. However, in general, it can be parsed from different substrings of input text (!)."
272 |         value = ""
273 |         def compile(self): return self.value                                                        #@ReservedAssignment
274 |     
275 |     class string(node):
276 |         """Artificial node created at some intermediate stage of tree post-processing that may or may not correspond
277 |         to a specific node of the initial AST. Typically produced from partial compilation of some other Tree nodes.
278 |         """
279 |         value = None
280 |         type  = "(string)"                                                                          #@ReservedAssignment
281 |         
282 |         def __init__(self, tree, value, pos = None):
283 |             self.tree = tree
284 |             self.fulltext = tree.text
285 |             self.value = value
286 |             self.pos = pos
287 |         def info(self):
288 |             return "<%s> %s" % (self.type, escape(self.value))
289 |         
290 |     
291 |     class empty(static): pass
292 | 
293 | 
294 | ########################################################################################################################################################
295 | ###
296 | ###  PARSIMONIOUS tree
297 | ###
298 | 
299 | class ParsimoniousTree(Tree):
300 | 
301 |     NODES = None                # the container (module/class/...) where node classes "x***" can be found; if None, self is used instead
302 | 
303 |     _reduce_anonym_ = True      # shall we reduce anonymous nodes? these are the nodes generated by unnamed expressions, typically groupings (...);
304 |                                 # leaf nodes (static strings) will be removed entirely from the tree, i.e., ignored rather than reduced
305 |     _reduce_string_ = True      # if a node to be reduce has no children but matched a non-empty part of the text, shall it be replaced with a string node
306 |                                 # instead of raising an exception? 
307 | 
308 |     class reduced(Tree.string):
309 |         "Like Tree.string, but adapted to reducing Parsimonious AST nodes."
310 |         type = "(reduced)"
311 |         
312 |         def __init__(self, tree, astnode):
313 |             Tree.string.__init__(self, tree, astnode.text, (astnode.start, astnode.end))
314 |         
315 | 
316 |     def rewrite(self, astnode):
317 |         def flat(): return self._rewriteNode(astnode)[-1]       # flattening: return children instead of the node
318 |             
319 |         if astnode is None: return None
320 |         name = astnode.expr_name
321 |         if not name and self._reduce_anonym_: return flat()     # flatten nodes without names; leaf nodes (static strings) removed entirely
322 |         if name in self._ignore_: return []                     # remove nodes listed in _ignore_, together with their subtrees
323 |         if name in self._reduce_:                               # flatten nodes listed in _reduce_
324 |             if astnode.children: return flat()
325 |             if astnode.start == astnode.end != None: return []
326 |             if self._reduce_string_: return self.reduced(astnode)
327 |             raise ParserError("Trying to reduce a node that has no children but consumed a non-empty part of the input at position (%s,%s): %s" % 
328 |                               (astnode.start, astnode.end, astnode))
329 |         
330 |         # find corresponding inner class of the tree and instantiate (this will recursively rewrite nodes down the tree)
331 |         nodeclass = getattr(self.NODES or self, 'x' + name)
332 |         node = nodeclass(self, astnode)
333 |     
334 |         if name in self._compact_:                              # flatten nodes listed in _compact_
335 |             if len(node.children) == 1: return node.children[0]
336 | 
337 |         return node
338 |     
339 |     def _rewriteNode(self, astnode):
340 |         children = flatten(self.rewrite(c) for c in astnode.children)
341 |         return (astnode.start, astnode.end), astnode.expr_name, children        # pos, type, children
342 |     
343 | 
344 | ########################################################################################################################################################
345 | ###
346 | ###  CONTEXT of semantic analysis
347 | ###
348 | 
349 | class Context(object):
350 |     """Base class for classes representing current context of semantic analysis in Tree.analyse(), 
351 |     for top-down transfer of contextual data from parent to child nodes and for bottom-up collection of global data from all nodes. 
352 |     Can be modified along the way, when passed through the tree, and copied with .copy() method 
353 |     when local modifications must be performed. 
354 |     """
355 |     
356 |     Data = None                             # class (or function that returns instantiated object) of self.data
357 |     
358 |     def __init__(self):
359 |         self.data = self.Data() if self.Data else None          # global semantic information about the syntax tree, shared by all copies of Context
360 |         
361 |     def copy(self):
362 |         "A safe - shallow - copy of Context. Global data are shared by all copies."
363 |         return copy.copy(self)
364 |     
365 | 


--------------------------------------------------------------------------------
/src/hypertag/std/__init__.py:
--------------------------------------------------------------------------------
1 | """
2 | This package contains implementations of the Hypertag's Standard Library:
3 | predefined tags and functions that can be used inside Hypertag scripts.
4 | """
5 | 


--------------------------------------------------------------------------------
/src/hypertag/std/builtins.py:
--------------------------------------------------------------------------------
  1 | import re, itertools
  2 | from hypertag.nifty.util import unique as unique_list, merge_spaces
  3 | from hypertag.core.dom import del_indent
  4 | from hypertag.std.registry import Registry
  5 | 
  6 | 
  7 | register = Registry()
  8 | 
  9 | 
 10 | ########################################################################################################################################################
 11 | #####
 12 | #####  GENERAL PURPOSE tags
 13 | #####
 14 | 
 15 | _re_dedent = re.compile(r'(?m)^\s+')
 16 | _re_merge  = re.compile(r'\s+')
 17 | 
 18 | 
 19 | @register.tag
 20 | @register.var
 21 | def doctype(text, value = 'html'):
 22 |     return f"<!DOCTYPE {value}>"
 23 | 
 24 | @register.tag
 25 | @register.var
 26 | def dedent(text, full = True):
 27 |     """
 28 |     Remove all line indentation in `text` (if full=True); or the longest common indentation shared by all
 29 |     lines with at least one non-whitespace character.
 30 |     The indentation may differ between lines and it still gets fully removed.
 31 |     """
 32 |     # return _re_dedent.sub('', text)
 33 |     if full: return _re_dedent.sub('', text)
 34 |     else:    return del_indent(text)
 35 | 
 36 | @register.tag
 37 | @register.var
 38 | def unique(text, strip = True):
 39 |     """
 40 |     Remove duplicate lines in `text`. The order of remaining lines is preserved.
 41 |     If strip=True (default), the lines are stripped of leading & trailing whitespace before comparison,
 42 |     and all empty lines are removed.
 43 |     """
 44 |     lines = text.splitlines()
 45 |     if strip: lines = list(filter(None, map(str.strip, lines)))
 46 |     uniq = unique_list(lines)
 47 |     return '\n'.join(uniq)
 48 | 
 49 | @register.tag
 50 | @register.var
 51 | def merge(text):
 52 |     """
 53 |     Strip leading/trailing whitespace in `text`, replace newlines and tabs with spaces, merge multiple adjacent spaces.
 54 |     Similar to normalize-space() in XPath.
 55 |     """
 56 |     return _re_merge.sub(' ', text).strip()
 57 | 
 58 | @register.tag
 59 | @register.var
 60 | def inline(text):
 61 |     """
 62 |     Convert all newline characters to HTML entities: &#10;
 63 |     This can be used to prevent Hypertag from indenting lines of `text` when rendering parent nodes,
 64 |     and to safely insert `text` inside <pre>, <textarea>, or similar elements.
 65 |     """
 66 |     return text.replace('\n', '&#10;')
 67 | 
 68 | @register.tag
 69 | @register.var
 70 | def lower(text):
 71 |     """A tag/function/filter that converts `text` to lowercase. Inner tag names and attributes are converted, too (!)."""
 72 |     return text.lower()
 73 | 
 74 | @register.tag
 75 | @register.var
 76 | def upper(text):
 77 |     """A tag/function/filter that converts `text` to uppercase. Inner tag names and attributes are converted, too (!)."""
 78 |     return text.upper()
 79 | 
 80 | @register.var
 81 | def cycle(*sequences, **params):   #stop = 'first'
 82 |     """
 83 |     A generator that cycles through each of the provided iterables IN PARALLEL and yields tuples containing
 84 |     one element from each iterable - like zip(), but the cycling runs indefinitely (if stop=False), or until
 85 |     the FIRST iterable gets exhausted (if stop='first', default), or until the LONGEST iterable gets exhausted
 86 |     (stop='longest'). When an iterable is exhausted along the way, its sequence of elements is repeated.
 87 |     This function is inspired by Django's {%cycle%} tag, yet it differs from this tag in that it doesn't require
 88 |     maintaining a hidden state, which enhances code readability.
 89 |     This function can be used, for instance, to alternate colors of rows in a table.
 90 |     """
 91 |     stop = params.pop('stop', 'first')
 92 |     if params: raise Exception('unrecognized keyword argument "%s" in cycle()' % list(params.keys())[0])
 93 |     
 94 |     if not sequences: return []
 95 |     length = None
 96 |     
 97 |     if stop == 'first':
 98 |         sequences = list(sequences)
 99 |         sequences[0] = list(sequences[0])               # convert the 1st iterable to a list, so that we can check its length
100 |         length = len(sequences[0])                      # length of the output sequence
101 |         
102 |     elif stop == 'longest':
103 |         sequences = list(map(list, sequences))          # convert each sequence to a list, so that we can check their lengths
104 |         length = max(map(len, sequences))               # ...and find the longest one
105 |     
106 |     cycles = map(itertools.cycle, sequences)            # create indefinite-loop iterators over each iterable
107 |     
108 |     return itertools.islice(zip(*cycles), length)
109 | 
110 | @register.var
111 | def changes(seq, first = True):
112 |     """
113 |     Iterates over a given sequence and yields pairs (value, change), where `value` is the next element of `seq`,
114 |     and `change` is True if the value differs from the previous one, False otherwise.
115 |     In the first tuple, `change` equals `first` (True by default).
116 |     This generator can be used in loops as a replacement for Django's {%ifchanged%} tag.
117 |     """
118 |     prev = None
119 |     for i, value in enumerate(seq):
120 |         if i == 0:
121 |             yield value, first
122 |         else:
123 |             yield value, (value != prev)
124 |         prev = value
125 | 
126 | @register.var
127 | def crop(s, length = 255, end = '...', killwords = False, maxdrop = 10, leeway = 0):
128 |     """
129 |     If `killwords` is false, the last word will be discarder, unless the resulting string
130 |     gets shorter than `maxdrop` characters only due to word preservation,
131 |     otherwise the word gets split (rather than discarded) anyway.
132 |     """
133 | 
134 |     # the implementation below was inspired by Jinja's truncate(): https://github.com/pallets/jinja/blob/main/src/jinja2/filters.py
135 |     assert isinstance(s, str), f"expected a string, got {type(s)}"
136 |     assert length >= len(end), f"expected length >= {len(end)}, got {length}"
137 |     assert leeway >= 0, f"expected leeway >= 0, got {leeway}"
138 | 
139 |     if len(s) <= length + leeway:
140 |         return s
141 |     
142 |     maxlen = length - len(end)          # maximum string length before appending the `end`
143 |     
144 |     if killwords:
145 |         return s[:maxlen] + end
146 | 
147 |     result = s[:maxlen].rsplit(" ", 1)[0]
148 |     if len(result) < maxlen - maxdrop:
149 |         result = s[:maxlen]
150 |         
151 |     return result + end
152 |     
153 | truncate = crop
154 | register.var('truncate', truncate)          # truncate() is an alias for crop(), for similarity with Jinja/Django
155 | 
156 | 
157 | ########################################################################################################################################################
158 | 
159 | """
160 | Terminal tags:
161 | - resource            -- terminal tag that marks a given block as a resource that shall be moved to HTML's <meta> section
162 | - error               -- inserts a standard error message in a place of occurrence; root document node might collect all <error> nodes and produce a combined (hidden) error message
163 | 
164 | Functions inspired by Django/Ninja:
165 | - findchange(seq)            -- similar to enumerate() but outputs a binary flag whenever a value of the sequence changes relative to the previous one;
166 |                                 replacement for Django's {%ifchanged%}
167 |   finddiff() changes()
168 |   
169 | - url ??
170 | 
171 | Filters:
172 | - groupby(key, d=None)       -- when applied to a list of dicts, returns a list of groups of records having the same value of `key`;
173 |                                 each group is a list of records; `key` can be a tuple of key names, names can refer to related objects "x.y.z";
174 |                                 replacement for Django's {%regroup%}
175 | """
176 | 
177 | 


--------------------------------------------------------------------------------
/src/hypertag/std/html.py:
--------------------------------------------------------------------------------
 1 | from hypertag.core.errors import TypeErrorEx
 2 | from hypertag.core.tag import Markup
 3 | from hypertag.std.registry import Registry
 4 | 
 5 | 
 6 | register = Registry()
 7 | 
 8 | 
 9 | ########################################################################################################################################################
10 | #####
11 | #####  UTILITIES
12 | #####
13 | 
14 | def html_escape(text):
15 |     """Escape HTML/XML special characters (& < >) in a string that's to be embedded in a text part of an HTML/XML document.
16 |     For escaping attribute values use html_attr_escape() - attributes need a different set of special characters to be escaped.
17 |     """
18 |     return text.replace('&', '&amp;').replace('>', '&gt;').replace('<', '&lt;')
19 | 
20 | 
21 | ########################################################################################################################################################
22 | #####
23 | #####  STANDARD HTML5 tags
24 | #####
25 | #####  In the future, we would like to provide a way to easily produce "polyglot" (X)HTML5 markup, which will be
26 | #####  compatible with both HTML5 and XHTML5.
27 | #####
28 | 
29 | _HTML_TAGS_VOID    = "area base br col command embed hr img input link meta param source track wbr".split()
30 | _HTML_TAGS_NONVOID = "a abbr acronym address applet article aside audio b basefont bdi bdo big blockquote body " \
31 |                      "button canvas caption center cite code colgroup data datalist dd del details dfn dialog dir " \
32 |                      "div dl dt em fieldset figcaption figure font footer form frame frameset h1 h2 h3 h4 h5 h6 " \
33 |                      "head header html i iframe ins kbd label legend li main map mark meter nav noframes noscript " \
34 |                      "object ol optgroup option output p picture pre progress q rp rt ruby s samp script section " \
35 |                      "select small span strike strong style sub summary sup svg table tbody td template textarea " \
36 |                      "tfoot th thead time title tr tt u ul var video".split()
37 | 
38 | def _create(name, void):
39 |     # register.tag(Markup(name.lower(), void, 'XHTML'))         # XHTML only permits lowercase tag names
40 |     register.tag(Markup(name.lower(), void, 'HTML'))
41 |     register.tag(Markup(name.upper(), void, 'HTML'))
42 | 
43 | 
44 | ### create tags
45 | 
46 | for tag in _HTML_TAGS_NONVOID:  _create(tag, False)
47 | for tag in _HTML_TAGS_VOID:     _create(tag, True)
48 | 
49 | 
50 | class CustomTag(Markup):
51 |     """
52 |     The %custom "NAME" tag: outputs non-standard HTML tag names, so that HTML Custom Elements (Web Components)
53 |     can be used inside target documents. Example use:
54 |     
55 |         custom "my-fancy-widget" id="widget" | ...
56 |         
57 |     which outputs:
58 |         
59 |         <my-fancy-widget id="widget">...</my-fancy-widget>
60 |        
61 |     Keyword attributes can be provided after NAME, as above.
62 |     """
63 |     name = "custom"
64 | 
65 |     def expand(self, body, attrs, kwattrs):
66 |         if not attrs: raise TypeErrorEx("the %%custom tag requires a positional attribute with a desired output name of a tag")
67 |         if len(attrs) > 1: raise TypeErrorEx("the %%custom tag accepts exactly one positional attribute, not %s" % len(attrs))
68 |         
69 |         name = attrs[0]
70 |         return self._expand(name, body, kwattrs)
71 | 
72 |         
73 | register.tag(CustomTag())
74 | 
75 | 
76 | ########################################################################################################################################################
77 | #####
78 | #####  HYPERTAG'S CUSTOM tags & functions (HTML-specific)
79 | #####
80 | 
81 | @register.tag
82 | class comment(Markup):
83 |     
84 |     name = "comment"
85 |     
86 |     def expand(self, body, attrs, kwattrs):
87 |         
88 |         if attrs or kwattrs: raise TypeErrorEx("'comment' tag does not accept attributes")
89 |         body = body.render()
90 |         
91 |         # if the block contains a headline, the closing tag is placed on the same line as body;
92 |         # a newline is added at the end, otherwise
93 |         nl = '\n' if body[:1] == '\n' else ''
94 |         return "<!--" + body + nl + "-->"
95 |     
96 |     


--------------------------------------------------------------------------------
/src/hypertag/std/registry.py:
--------------------------------------------------------------------------------
 1 | from hypertag.nifty.util import isfunction
 2 | from hypertag.core.tag import TagFunction
 3 | 
 4 | 
 5 | class Registry:
 6 |     """A registry of standard tags and variables (e.g., functions, filters) implemented by a given module."""
 7 |     
 8 |     def __init__(self):
 9 |         self.tags = {}
10 |         self.vars = {}
11 |     
12 |     def tag(self, obj):
13 |         """
14 |         Register a given object (Tag subclass, Tag instance, tag function), as a tag by adding to self.tags.
15 |         Tag subclasses are instantiated before adding to self.tags. Functions are wrapped up in TagFunction.
16 |         """
17 |         if isinstance(obj, type):           # instantiate tag classes
18 |             tag  = obj()
19 |             name = tag.name
20 |         elif isfunction(obj):
21 |             tag  = TagFunction(obj)
22 |             name = obj.__name__
23 |         else:
24 |             tag  = obj
25 |             name = obj.name
26 | 
27 |         self.tags[name] = tag
28 |         return obj
29 |     
30 |     def var(self, name, obj = None):
31 |         """Register a given object as a Hypertag variable ($)."""
32 |         
33 |         if obj is None:
34 |             # name is not provided, object stored under `name`; must assume the object is a function, so the name can be deduced
35 |             obj  = name
36 |             assert isfunction(obj)
37 |             name = obj.__name__
38 |         
39 |         self.vars[name] = obj
40 |         return obj
41 |         
42 | 
43 | 


--------------------------------------------------------------------------------
/src/hypertag/tests/__init__.py:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/mwojnars/hypertag/8cd989127c12d78f6902d4b3dc972de31a3e88d9/src/hypertag/tests/__init__.py


--------------------------------------------------------------------------------
/src/hypertag/tests/package1/__init__.py:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/mwojnars/hypertag/8cd989127c12d78f6902d4b3dc972de31a3e88d9/src/hypertag/tests/package1/__init__.py


--------------------------------------------------------------------------------
/src/hypertag/tests/package1/sample3.hy:
--------------------------------------------------------------------------------
1 | $y = 15
2 | 
3 | %G @body x=9:
4 |     | {x+y}
5 |     @body
6 |     | {x*y}
7 | 
8 | 


--------------------------------------------------------------------------------
/src/hypertag/tests/sample1.py:
--------------------------------------------------------------------------------
1 | 
2 | x = 155
3 | 
4 | def f(a):
5 |     return a * x
6 | 
7 | 


--------------------------------------------------------------------------------
/src/hypertag/tests/sample2.hy:
--------------------------------------------------------------------------------
 1 | from .package1.sample3 import %G, $y
 2 | 
 3 | $x = 5
 4 | 
 5 | %H @body y=5:
 6 |     | {x+y}
 7 |     @body
 8 |     | {x*y}
 9 | 
10 | $x = 155
11 | $z = 155 + y
12 | 


--------------------------------------------------------------------------------
/src/hypertag/tests/sample3.hy:
--------------------------------------------------------------------------------
1 | %print_data item
2 |     h2 : b | Data
3 | 


--------------------------------------------------------------------------------
/src/hypertag/tests/sample4.hy:
--------------------------------------------------------------------------------
1 | from sample2 import $x as y, %G as H
2 | 
3 | $x = y * 2
4 | 
5 | %G @body
6 |     | in G
7 |     H @body
8 | 


--------------------------------------------------------------------------------
/tests/__init__.py:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/mwojnars/hypertag/8cd989127c12d78f6902d4b3dc972de31a3e88d9/tests/__init__.py


--------------------------------------------------------------------------------
/tests/sample_usage.html:
--------------------------------------------------------------------------------
 1 | 
 2 |     <html><body>
 3 |         <h1 style="color: #00f">
 4 |             Example document with a <u>list of items</u>
 5 |         </h1>
 6 |         <ul>
 7 |             <li>item &lt;1&gt;</li>
 8 |             <li>item no. 2</li>
 9 |         </ul>
10 |         <p><a href="http://hypertag.io">See the Hypertag site for more!</a></p>
11 |     </body></html>
12 | 
13 | 


--------------------------------------------------------------------------------
/tests/sample_usage.py:
--------------------------------------------------------------------------------
 1 | # run:  python3 -m hypertag.test.sample_usage > hypertag/test/sample_usage.html
 2 | #
 3 | 
 4 | from hypertag import HyperHTML
 5 | 
 6 | script = \
 7 | """
 8 |     import $blue
 9 |     html: body:
10 |         h1 style="color: $blue"
11 |             / Example document with a <u>list of items</u>
12 |         ul
13 |             li | item <1>
14 |             li | item no. {1+1}
15 |         p : a href='http://hypertag.io' | See the Hypertag site for more!
16 | """
17 | 
18 | html = HyperHTML().render(script, blue = '#00f')
19 | print(html)
20 | 


--------------------------------------------------------------------------------
/tests/sandbox.py:
--------------------------------------------------------------------------------
  1 | from hypertag import HyperHTML, Runtime
  2 | from hypertag.core.ast import HypertagAST
  3 | from hypertag.core.runtime import RootModule
  4 | 
  5 | 
  6 | ########################################################################################################################################################
  7 | #####
  8 | #####  MAIN
  9 | #####
 10 | 
 11 | if __name__ == '__main__':
 12 | 
 13 |     DEBUG = True
 14 |     ctx = {}
 15 |     
 16 |     # text = """
 17 |     #     $g = 100
 18 |     #     %g x | xxx {x+g}
 19 |     #     %H @body a=0
 20 |     #         @ body
 21 |     #         $g = 200
 22 |     #         g (g+5)
 23 |     #         g a[0] !
 24 |     #     H [0,6]?
 25 |     #         | pies
 26 |     # """
 27 |     # text = """
 28 |     #     $ x = 0
 29 |     #     try | x $x!
 30 |     #     else:
 31 |     #         try  / x*2 = {x*2}!
 32 |     #         else / x+1 = {x+1}!
 33 |     # """
 34 |     text = """
 35 |         DIV
 36 |             < try | {True}!
 37 |             < try | {False}!
 38 |             <? | {True}!
 39 |             <? | {False}!
 40 |     """
 41 |     text = """
 42 |         if False:
 43 |             $ x = 5
 44 |         else:
 45 |             $ x = 10
 46 |         | {x}
 47 |     """
 48 |     # text = """
 49 |     #     if False
 50 |     #         $x = 1
 51 |     #         %H | $x
 52 |     #     else
 53 |     #         $x = 2
 54 |     #         %H | {x*2}
 55 |     #         $x = 3
 56 |     #     $x = 4
 57 |     #     H
 58 |     # """
 59 | 
 60 |     # text = """
 61 |     #     for i in [2,1,0]:
 62 |     #         if $i
 63 |     #             $a = 1
 64 |     #         else
 65 |     #             $b = 2
 66 |     #         | $a
 67 |     # """
 68 |     text = """
 69 |         %H x
 70 |             if $x
 71 |                 $a = 1
 72 |             else
 73 |                 $b = 2
 74 |             | $a
 75 |         H 1
 76 |         H 0
 77 |     """
 78 |     # TODO: dodać czyszczenie slotów w `state` po wykonaniu bloku, przynajmniej dla xblock_def.expand() ??
 79 |     
 80 |     text = r"""
 81 |         %G x
 82 |             p | $x
 83 |             i : b | $x
 84 |         G 0
 85 |         G 1
 86 |         div .wide
 87 |             p #par1
 88 |                 b : i | bleble
 89 |     """
 90 |     text = r"""
 91 |     ul .short-list
 92 |         li : i | Item
 93 |     
 94 |     for i in [1,2,3]:
 95 |         b class="row$i" | Row no. $i
 96 |     
 97 |     p id='paragraph1'
 98 |         b : i
 99 |             | Paragraph
100 | 
101 |     % hypertag @body x y=0
102 |         | Title $x $y
103 |         @ body
104 |         for node in list(body.walk('postorder')):
105 |             | node class $node.__class__.__name__ tag {node.tag.name if node.tag else 'None'}
106 |     hypertag 1 y=2
107 |         p
108 |             b | Paragraph
109 |         
110 |     """
111 |     text = """
112 |     % tableRow @info name price='UNKNOWN'
113 |         tr
114 |             td | $name
115 |             td | $price
116 |             td
117 |                @ info           # inline form can be used as well:  td @ info
118 |             / $info
119 |             / $info.render()
120 | 
121 |     tableRow 'Porsche' '200,000'
122 |         img src="porsche.jpg"
123 |         / If you insist on <s>air conditioning</s>, 🤔
124 |     """
125 |     text = """
126 |     %toc @document
127 |         for heading in document['h2']
128 |             $ id = heading.get('id', '')
129 |             li : a href="#{id}" @ heading.body
130 | 
131 |     %with_toc @document
132 |         | Table of Contents:
133 |         ol
134 |             toc @document
135 | 
136 |         | The document:
137 |         @document
138 |         
139 |         #
140 |             | Everything except 'h2':
141 |             @document.skip('h2')
142 |     
143 |             | Everything except 'i':
144 |             @document.skip('i')
145 | 
146 |     with_toc
147 |         h2 #first  | First heading
148 |         p  | text...
149 |         h2 #second | Second heading
150 |         p  | text...
151 |         h2 #third  | Third heading
152 |         p  | text...
153 |         
154 |         p : i | Contents...
155 |     """
156 |     
157 |     # ctx  = {'x': 10, 'y': 11, 's': "A"}
158 |     # text = """
159 |     #     context $x            # variable "x"
160 |     #     context $y as z       -- variable "y" renamed internally to "z"
161 |     #     | {x + z}
162 |     # """
163 |     # text = """
164 |     #     context $s
165 |     #     from builtins import *
166 |     #     from builtins import $ord
167 |     #     #from ~ import *
168 |     #     #import *
169 |     #     | $ord(s)
170 |     # """
171 |     # text = """
172 |     # from hypertag.tests.sample2 import %G
173 |     # %H | hypertag
174 |     # / {%div('kot', id=5)}
175 |     # / G is: {%G()} .
176 |     # / H is: {%H()} .
177 |     #
178 |     # %link @text
179 |     #     a href='#' @text
180 |     #
181 |     # link | Ala
182 |     # link
183 |     #     | kot
184 |     # b / ({%link('pies')})
185 |     #
186 |     # / {%div('plain text').upper()}
187 |     #
188 |     # %t x | {10+x}
189 |     # | { %t(x=5) }
190 |     # """
191 |     # text = """
192 |     # %f x
193 |     #     | this is x: $x
194 |     #     if x > 0:
195 |     #        f (x-1)
196 |     # f 5
197 |     # """
198 |     text = """
199 |         $c = 0
200 |         p
201 |             for x in [1,2,3]:
202 |                 | $c
203 |                 $c = x
204 |     """
205 |     text = """
206 |         $c = 0
207 |         %H
208 |             for x in [1,2,3]:
209 |                 $c = x
210 |                 | $c
211 |         H
212 |         | $c
213 |     """
214 |     
215 |     # TODO:
216 |     text = r"""
217 |         import $os
218 |         | $os
219 |     """
220 |     text = r"""
221 |         try: p | text
222 |     """
223 |     text = """
224 |         from os import $x $y
225 |     """
226 | 
227 |     root = RootModule(runtime = HyperHTML(), filename = __file__, package = __package__)
228 |     tree = HypertagAST(text, root, verbose = True)
229 |     
230 |     # print()
231 |     # print("===== AST =====")
232 |     # print(tree.ast)
233 |     # print(type(tree.ast))
234 |     # print()
235 |     print("===== After rewriting =====")
236 |     print(tree)
237 |     print()
238 |     
239 |     #
240 |     # print("===== After semantic analysis =====")
241 |     # tree.analyse()
242 |     # # print()
243 |     # # print(tree)
244 |     # print()
245 |     
246 |     print("===== After translation =====")
247 |     dom, _, _ = tree.translate(**ctx)
248 |     print('DOM:')
249 |     print(dom.tree('  '))
250 |     print()
251 | 
252 |     print("===== After rendering =====")
253 |     print(dom.render(), end = "=====\n")
254 |     # print(tree.A())
255 |     
256 |     
257 |     
258 |     # import inspect, importlib
259 |     # print(__file__)
260 |     # module = importlib.import_module('.', 'hypertag.core')  #__package__)
261 |     # print(module.__name__)
262 |     # print(module.__file__)
263 |     # print(module.__package__)
264 |     # print(inspect.getfile(module))
265 |     # print(inspect.getfile(text))
266 |     # from hypertag.django.filters import slugify
267 |     # print(slugify('Hypertag rocks'))
268 |     # __package__ = "hypertag.core"
269 |     # from .runtime import __file__, __name__
270 |     # from hypertag.core.runtime import __file__
271 |     # print(__name__)
272 |     
273 |     
274 | # TODO:
275 | # - disallow mixed inline+outline text blocks, they're confusing esp. when some lines are commented out
276 | # - czyszczenie slotów w `state` po wykonaniu bloku, przynajmniej dla xblock_def.expand() ??
277 | # - Markdown/Textile/Sass/Scss blocks
278 | # - Runtime & Loader: detection of circular imports
279 | # - Python's "bitwise not" operator (~)
280 | 


--------------------------------------------------------------------------------
/tests/sandbox_py2.py:
--------------------------------------------------------------------------------
 1 | #-*- coding: utf-8 -*-
 2 | """
 3 | Sandbox for testing Hypertag with Python 2.
 4 | For now, we stop backporting to Python 2 because Parsimonious apparently does NOT support Python 2
 5 | (issues around Unicode handling).
 6 | """
 7 | 
 8 | from __future__ import print_function
 9 | # from __future__ import unicode_literals
10 | 
11 | from hypertag.core.ast import HypertagAST
12 | 
13 | 
14 | ########################################################################################################################################################
15 | #####
16 | #####  MAIN
17 | #####
18 | 
19 | if __name__ == '__main__':
20 | 
21 |     from hypertag import HyperHTML
22 |     DEBUG = True
23 |     
24 |     print(u"test: ąłżęśćó ")
25 |     
26 |     text = u"""
27 |         | Ala ma kota
28 |     """
29 |     
30 |     tree = HypertagAST(text, HyperHTML(), stopAfter = "rewrite", verbose = True)
31 |     
32 |     # print()
33 |     # print("===== AST =====")
34 |     # print(tree.ast)
35 |     # print(type(tree.ast))
36 |     # print()
37 |     print("===== After rewriting =====")
38 |     print(tree)
39 |     print()
40 |     
41 | 
42 |     print("===== After semantic analysis =====")
43 |     tree.analyse()
44 |     # print()
45 |     # print(tree)
46 |     print()
47 |     
48 |     print("===== After translation =====")
49 |     dom, _, _ = tree.translate()
50 |     print('DOM:')
51 |     print(dom.tree('  '))
52 |     print()
53 | 
54 |     print("===== After rendering =====")
55 |     print(dom.render(), end = "=====\n")
56 |     # print(tree.A())
57 |     
58 |     # import inspect, importlib
59 |     # print(__file__)
60 |     # module = importlib.import_module('.', 'hypertag.core')  #__package__)
61 |     # print(module.__name__)
62 |     # print(module.__file__)
63 |     # print(module.__package__)
64 |     # print(inspect.getfile(module))
65 |     # print(inspect.getfile(text))
66 |     # from hypertag.django.filters import slugify
67 |     # print(slugify('Hypertag rocks'))
68 |     
69 |     
70 | # TODO:
71 | # - czyszczenie slotów w `state` po wykonaniu bloku, przynajmniej dla xblock_def.expand() ??
72 | # - selectors @body[...]
73 | # - Python's "bitwise not" operator (~)
74 | 


--------------------------------------------------------------------------------
/tox.ini:
--------------------------------------------------------------------------------
 1 | # Test, build & upload to PyPI:
 2 | # $ sudo tox -r
 3 | # $ python3 -m build
 4 | # $ python3 -m twine upload dist/*        -- use __token__ as username
 5 | 
 6 | 
 7 | [tox]
 8 | envlist = py35,py3
 9 | isolated_build = True
10 | 
11 | [testenv]
12 | # install testing framework
13 | # ... or install anything else you might need here
14 | deps = 
15 |     pytest
16 |     django
17 |     parsimonious==0.8.1
18 |     
19 | # run the tests
20 | # ... or run any other command line tool you need to run here
21 | commands = pytest -vW ignore::DeprecationWarning
22 | 


--------------------------------------------------------------------------------