├── .gitignore ├── LICENSE.MIT ├── README.md ├── pyproject.toml ├── requirements.txt ├── streamdown ├── __init__.py ├── plugins │ ├── README.md │ └── latex.py └── sd.py ├── tests ├── README.md ├── backtick-with-post-spaces.md ├── blankie.md ├── block.md ├── bold_reset_with_link.md ├── broken-code.md ├── broken-example.md ├── chunk-buffer.sh ├── cjk-table.md ├── cjk-wrap.md ├── code.md ├── dimcheck.md ├── example.md ├── fizzbuzz.md ├── inline.md ├── jimmy_webb.md ├── line-buffer.sh ├── line-wrap.md ├── links.md ├── managerie.md ├── mandlebrot.md ├── markdown.md ├── nested-example.md ├── outline.md ├── pvgo_512.jpg ├── pythonvgo.md ├── qwen3.md ├── rerun.zsh ├── slash.md ├── strip-chunks.sh ├── table-break.md ├── table_test.md ├── test.md ├── test_input.md ├── uline.md └── wm.md └── tools └── deploy.sh /.gitignore: -------------------------------------------------------------------------------- 1 | __pycache__ 2 | -------------------------------------------------------------------------------- /LICENSE.MIT: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2025 Chris McKenzie 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 |

2 | 3 |
4 | 5 |
Terminal streaming markdown that rocks 6 | 7 |

8 | 9 | 10 | Streamdown works with any streaming markdown such as [simonw's llm](https://github.com/simonw/llm) or even something basic like curl. 11 | 12 | It's designed for compatibility with the wide variety of markdown from various LLM models. 13 | 14 | It supports standard piping and files as arguments like any normal pager but can also run as a wrapper so you retain full keyboard interactivity. Arrow keys, control, alt, all still work. 15 | ```bash 16 | $ pip install streamdown 17 | ``` 18 | ![Streamdown is Amazing](https://github.com/user-attachments/assets/268cb340-78cc-4df0-a773-c5ac95eceeeb) 19 | 20 | ## Fast and Realtime. 21 | Watch Streamdown run over a FIFO pipe through `tee` in tmux on an M4 using BitNet. This is run straight. No clever unbuffering tricks. You can see the unstructured content on the right and the realtime Streamdown render on the left. 22 | 23 | [bitnet.webm](https://github.com/user-attachments/assets/62eb625e-82c4-462d-9991-ed681d6fbcd0) 24 | 25 | 26 | ### Provides clean copyable code for long code lines 27 | Other renderers inject line breaks when copying code that wraps around. Streamdown's better and now you are too! 28 | 29 | Set `PrettyBroken` and `PrettyPad` to False in your toml (see below) to make Streamdown ensure code is always cleanly mouse copyable 30 | ![Handle That Mandle](https://github.com/user-attachments/assets/a27aa70c-f691-4796-84f0-c2eb18c7de23) 31 | 32 | 33 | ### Supports images 34 | Here's kitty and alacritty. 35 | ![doggie](https://github.com/user-attachments/assets/81c43983-68cd-40c1-b1d5-aa3a52004504) 36 | 37 | ### Hyperlinks (OSC 8) and Clipboard (OSC 52) 38 | The optional `Clipboard` feature puts the final codeblock into your clipboard. See below for details. 39 | 40 | [links.webm](https://github.com/user-attachments/assets/a5f71791-7c58-4183-ad3b-309f470c08a3) 41 | 42 | ### As well as everything else... 43 | Here's the `Savebrace` feature with `screen-query` and `sq-picker` from [llmehelp](https://github.com/kristopolous/llmehelp). You can have an ongoing conversation in tmux with your terminal session. Then use popups and fzf to insert command or coding blocks all with a keystroke. 44 | 45 | This allows you to interactively debug in a way that the agent doesn't just wander off doing silly things. 46 | 47 | It takes about 2 minutes to set up and about 0.2s to use. Fast, fluid and free. 48 | ![screenquery](https://github.com/user-attachments/assets/517be4fe-6962-4e4c-b2f2-563471bc48d0) 49 | 50 | ### ...even CJK 51 | Compare how streamdown wraps and spaces this tabular Chinese description of programming languages to other leading markdown renderers. 52 | 53 | Only one generates the text without truncation. 很美! 54 | ![cjk](https://github.com/user-attachments/assets/cae485d7-c478-4836-9732-d9fa49e13bc9) 55 | 56 | ### Colors are highly (and quickly) configurable for people who care a lot, or just a little. 57 | ![configurable](https://github.com/user-attachments/assets/19ca2ec9-8ea1-4a79-87ca-8352789269fe) 58 | 59 | ### Has a [Plugin](https://github.com/kristopolous/Streamdown/tree/main/streamdown/plugins) system to extend the parser and renderers. 60 | For instance, here is the [latex plugin](https://github.com/kristopolous/Streamdown/blob/main/streamdown/plugins/latex.py) doing math inside a table: 61 | ![calc](https://github.com/user-attachments/assets/0b0027ca-8ef0-4b4a-b4ae-e36ff623a683) 62 | 63 | 64 | 65 | It is designed for AI and can be used to do parser based sophisticated pipelines and routing, cracking open various monolithic AI solutions to permit them to integrate. Think of it as output level routing at the semantic level. 66 | 67 | You can also just use it like a normal person. 68 | ## Configuration 69 | 70 | The location it's stored is platform specific and can be seen with the `-h` flag. If this file does not exist upon first run, it will be created with default values. 71 | 72 | Here are the sections: 73 | 74 | **`[style]`** 75 | 76 | Defines the base Hue (H), Saturation (S), and Value (V) from which all other palette colors are derived. This can also be specified at runtime via command line arguments. See below! 77 | 78 | The default values are [at the beginning of the source](https://github.com/kristopolous/Streamdown/blob/main/streamdown/sd.py#L33). 79 | 80 | * `HSV`: [ 0.0 - 1.0, 0.0 - 1.0, 0.0 - 1.0 ] 81 | * `Dark`: Multipliers for background elements, code blocks. 82 | * `Grey`: Multipliers for blockquote and thinkblock. 83 | * `Mid`: Multipliers for inline code backgrounds, table headers. 84 | * `Symbol`: Multipliers for list bullets, horizontal rules, links. 85 | * `Head`: Multipliers for level 3 headers. 86 | * `Bright`: Multipliers for level 2 headers. 87 | * `Margin` (integer, default: `2`): The left and right indent for the output. 88 | * `Width` (integer, default: `0`): Along with the `Margin`, `Width` specifies the base width of the content, which when set to 0, means use the terminal width. See [#6](https://github.com/kristopolous/Streamdown/issues/6) for more details 89 | * `PrettyPad` (boolean, default: `true`): Uses a unicode vertical pad trick to add a half height background to code blocks. This makes copy/paste have artifacts. See [#2](https://github.com/kristopolous/Streamdown/issues/2). I like it on. But that's just me 90 | * `PrettyBroken` (boolean, default: `true`): This will break the copy/paste assurance above. The output is much prettier, but it's also broken. So it's pretty broken. Works nicely with PrettyPad. 91 | * `ListIndent` (integer, default: `2`): This is the recursive indent for the list styles. 92 | * `Syntax` (string, default `native`): This is the syntax [highlighting theme which come via pygments](https://pygments.org/styles/). 93 | 94 | Example: 95 | ```toml 96 | [style] 97 | PrettyPad = true 98 | PrettyBroken = true 99 | HSV = [0.7, 0.5, 0.5] 100 | Dark = { H = 1.0, S = 1.2, V = 0.25 } # Make dark elements less saturated and darker 101 | Symbol = { H = 1.0, S = 1.8, V = 1.8 } # Make symbols more vibrant 102 | ``` 103 | 104 | **`[features]`** 105 | 106 | Controls optional features: 107 | 108 | * `CodeSpaces` (boolean, default: `true`): Enables detection of code blocks indented with 4 spaces. Set to `false` to disable this detection method (triple-backtick blocks still work). 109 | * `Clipboard` (boolean, default: `true`): Enables copying the last code block encountered to the system clipboard using OSC 52 escape sequences upon exit. Set to `false` to disable. 110 | * `Logging` (boolean, default: `false`): Enables logging to tmpdir (/tmp/sd) of the raw markdown for debugging and bug reporting. The logging uses an emoji as a record separator so the actual streaming delays can be simulated and replayed. If you use the `filename` based invocation, that is to say, `sd `, this type of logging is always off. 111 | * `Savebrace` (boolean, default: `true`): Saves the code blocks of a conversation to the append file `$TMP/sd/$UID/savebrace` so you can `fzf` or whatever you want through it. See how it's used in DAY50's [sidechat](https://github.com/day50-dev/sidechat). 112 | 113 | Example: 114 | ```toml 115 | [features] 116 | CodeSpaces = false 117 | Clipboard = false 118 | ``` 119 | 120 | ## Command Line 121 | The most exciting feature here is `--exec` with it you can do full readline support like this: 122 | 123 | ```shell 124 | $ sd --exec "llm chat" 125 | ``` 126 | 127 | And now you have all your readline stuff. It's pretty great. (Also see the DAY50 shellwrap project.) 128 | 129 | It's also worth noting that things like the `-c` aren't "broken" with regard to file input. You can do something like this: 130 | 131 | ```shell 132 | $ sd -c <(echo "[style]\nMargin=10") 133 | ``` 134 | 135 | To override the margin. 136 | 137 | ```shell 138 | usage: sd [-h] [-l LOGLEVEL] [-b BASE] [-c CONFIG] [-w WIDTH] [-e EXEC] 139 | [-s SCRAPE] [filenameList ...] 140 | 141 | Streamdown is a streaming markdown renderer for modern terminals. 142 | https://github.com/day50-dev/Streamdown 143 | 144 | paths: 145 | config /home/chris/.config/streamdown/config.toml 146 | logs /tmp/sd/1000 147 | 148 | positional arguments: 149 | filenameList Input file to process (also takes stdin) 150 | 151 | optional arguments: 152 | -h, --help show this help message and exit 153 | -l LOGLEVEL, --loglevel LOGLEVEL 154 | Set the logging level 155 | -b BASE, --base BASE Set the hsv base: h,s,v 156 | -c CONFIG, --config CONFIG 157 | Use a custom config override 158 | -w WIDTH, --width WIDTH 159 | Set the width WIDTH 160 | -e EXEC, --exec EXEC Wrap a program EXEC for more 'proper' i/o handling 161 | -s SCRAPE, --scrape SCRAPE 162 | Scrape code snippets to a directory SCRAPE 163 | -v, --version Show version information 164 | ``` 165 | 166 | **Note**: Some features are not supported on some OSs. Please file a ticket if you need a feature on your platform that isn't working. 167 | 168 | ## Demo 169 | Do this 170 | 171 | $ ./streamdown/sd.py tests/*md 172 | 173 | ## Install from source 174 | After the git clone least one of these should work, hopefully. it's using the modern uv pip tool but is also backwards compatible to the `pip3 install -r requirements.txt` flow. 175 | 176 | $ pipx install -e . 177 | $ pip install -e . 178 | $ uv pip install -e . 179 | 180 | Explore the rest of [DAY50](https://github.com/day50-dev). Feel free to follow us, there's some exciting stuff coming. 181 | -------------------------------------------------------------------------------- /pyproject.toml: -------------------------------------------------------------------------------- 1 | [build-system] 2 | requires = ["hatchling"] 3 | build-backend = "hatchling.build" 4 | 5 | [project] 6 | name = "streamdown" 7 | version = "0.32.0" 8 | description = "A streaming markdown renderer for modern terminals with syntax highlighting" 9 | readme = "README.md" 10 | requires-python = ">=3.8" 11 | license = {text = "MIT"} 12 | authors = [ 13 | {name = "Chris McKenzie", email = "kristopolous@yahoo.com"}, 14 | ] 15 | keywords = ["markdown", "terminal", "renderer", "cli", "syntax-highlighting"] 16 | classifiers = [ 17 | "Development Status :: 4 - Beta", 18 | "Environment :: Console", 19 | "Intended Audience :: Developers", 20 | "License :: OSI Approved :: MIT License", 21 | "Programming Language :: Python :: 3", 22 | "Programming Language :: Python :: 3.8", 23 | "Programming Language :: Python :: 3.9", 24 | "Programming Language :: Python :: 3.10", 25 | "Programming Language :: Python :: 3.11", 26 | "Topic :: Text Processing :: Markup", 27 | "Topic :: Utilities", 28 | ] 29 | dependencies = [ 30 | "pygments", 31 | "appdirs", 32 | "toml", 33 | "wcwidth", 34 | "pylatexenc", 35 | 'term-image' 36 | ] 37 | 38 | [project.urls] 39 | "Homepage" = "https://github.com/day50-dev/Streamdown" 40 | "Bug Tracker" = "https://github.com/day50-dev/Streamdown/issues" 41 | 42 | [project.scripts] 43 | streamdown = "streamdown.sd:main" 44 | sd = "streamdown.sd:main" 45 | 46 | [tool.hatch.build.targets.wheel] 47 | packages = ["streamdown"] 48 | include = ["streamdown/plugins/*"] 49 | exclude = ["venv", "dist", ".git"] 50 | 51 | 52 | -------------------------------------------------------------------------------- /requirements.txt: -------------------------------------------------------------------------------- 1 | pygments 2 | pylatexenc 3 | appdirs 4 | term-image 5 | wcwidth 6 | toml 7 | -------------------------------------------------------------------------------- /streamdown/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/day50-dev/Streamdown/367a457870326e1f027cd4d1ebfb84b03578efe3/streamdown/__init__.py -------------------------------------------------------------------------------- /streamdown/plugins/README.md: -------------------------------------------------------------------------------- 1 | # Streamdown Plugins 2 | 3 | Streamdown contains a simple and hopefully not too painful plugin system 4 | 5 | ``` python 6 | def Plugin(line in, State, Style): 7 | return None | [ ansi escaped and formatted line, ] 8 | ``` 9 | 10 | * If None, its assumed the plugin is uninterested in the incoming line. 11 | * If it's an array, it's assumed it should be yielded and no other code should be run 12 | * If it's non-None then it receives priority as the first plugin called until it returns none, claiming it's done with the parsing 13 | * It's responsible for maintaining its own state. 14 | * The State and Style are from the main program if it chooses to observe it. 15 | 16 | The important caveat is this thing is truly streaming. 17 | ``` 18 | You may get totally part 19 | ial text li 20 | ke this and then have to reco 21 | nstruct it. 22 | ``` 23 | 24 | It is up to you how you'd like to yield it. You can buffer and wait for the whole segment if you need to, emit based on lines, whatever is needed. That's up to you! 25 | 26 | There's a few tricks for tricky situations tht are used in the main sd.py and those are all available to you via state and yield hacks. 27 | 28 | Check the files, they're pretty small and should be fairly self explanatory. 29 | 30 | -------------------------------------------------------------------------------- /streamdown/plugins/latex.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env -S uv run --script 2 | # /// script 3 | # requires-python = ">=3.8" 4 | # dependencies = [ 5 | # "pylatexenc" 6 | # ] 7 | # /// 8 | from pylatexenc.latex2text import LatexNodes2Text 9 | 10 | class Parser: 11 | inState = False 12 | buffer = '' 13 | 14 | def Plugin(text, state = None, style = None): 15 | res = True 16 | if not Parser.inState: 17 | if '$$' in text: 18 | Parser.buffer = '' 19 | Parser.inState = True 20 | text = text[text.index('$$') + 2:] 21 | 22 | if Parser.inState: 23 | if not '$$' in text: 24 | Parser.buffer += text 25 | return res 26 | 27 | Parser.inState = False 28 | Parser.buffer += text[:text.index('$$')] 29 | 30 | return [LatexNodes2Text().latex_to_text(Parser.buffer)] 31 | 32 | -------------------------------------------------------------------------------- /streamdown/sd.py: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | # /// script 3 | # requires-python = ">=3.8" 4 | # dependencies = [ 5 | # "pygments", 6 | # "pylatexenc", 7 | # "appdirs", 8 | # "term-image", 9 | # "wcwidth", 10 | # "toml" 11 | # ] 12 | # /// 13 | '''': 14 | if command -v uv &> /dev/null; then 15 | exec uv run --script "$0" "$@" 16 | else 17 | exec python3 "$0" "$@" 18 | fi 19 | ''' 20 | import appdirs, toml 21 | import logging, tempfile 22 | import os, sys 23 | import select 24 | 25 | if os.name != 'nt': 26 | import pty, termios, tty 27 | 28 | import math 29 | import re 30 | import shutil 31 | import traceback 32 | import colorsys 33 | import base64 34 | import subprocess 35 | from io import BytesIO 36 | from term_image.image import from_file, from_url 37 | import pygments.util 38 | from wcwidth import wcwidth 39 | from functools import reduce 40 | import textwrap 41 | import argparse 42 | from argparse import ArgumentParser 43 | from pygments import highlight 44 | from pygments.lexers import get_lexer_by_name 45 | from pygments.formatters import TerminalTrueColorFormatter 46 | from pygments.styles import get_style_by_name 47 | 48 | if __package__ is None: 49 | from plugins import latex 50 | else: 51 | from .plugins import latex 52 | 53 | default_toml = """ 54 | [features] 55 | CodeSpaces = false 56 | Clipboard = true 57 | Logging = false 58 | Timeout = 0.1 59 | Savebrace = true 60 | 61 | [style] 62 | Margin = 2 63 | ListIndent = 2 64 | PrettyPad = true 65 | PrettyBroken = true 66 | Width = 0 67 | HSV = [0.8, 0.5, 0.5] 68 | Dark = { H = 1.00, S = 1.50, V = 0.25 } 69 | Mid = { H = 1.00, S = 1.00, V = 0.50 } 70 | Symbol = { H = 1.00, S = 1.00, V = 1.50 } 71 | Head = { H = 1.00, S = 1.00, V = 1.75 } 72 | Grey = { H = 1.00, S = 0.25, V = 1.37 } 73 | Bright = { H = 1.00, S = 0.60, V = 2.00 } 74 | Syntax = "native" 75 | """ 76 | 77 | def ensure_config_file(config): 78 | config_dir = appdirs.user_config_dir("streamdown") 79 | os.makedirs(config_dir, exist_ok=True) 80 | config_path = os.path.join(config_dir, "config.toml") 81 | if not os.path.exists(config_path): 82 | open(config_path, 'w').write(default_toml) 83 | 84 | toml_res = toml.load(config_path) 85 | 86 | if config: 87 | if os.path.exists(config): 88 | config_string = open(config).read() 89 | else: 90 | config_string = config 91 | toml_res |= toml.loads(config_string) 92 | 93 | return toml_res 94 | 95 | 96 | FG = "\033[38;2;" 97 | BG = "\033[48;2;" 98 | RESET = "\033[0m" 99 | FGRESET = "\033[39m" 100 | FORMATRESET = "\033[24;23;22m" 101 | BGRESET = "\033[49m" 102 | 103 | BOLD = ["\033[1m", "\033[22m"] 104 | UNDERLINE = ["\033[4m", "\033[24m"] 105 | ITALIC = ["\033[3m", "\033[23m"] 106 | STRIKEOUT = ["\033[9m", "\033[29m"] 107 | LINK = ["\033]8;;", "\033]8;;\033\\"] 108 | SUPER = [ 0x2070, 0x00B9, 0x00B2, 0x00B3, 0x2074, 0x2075, 0x2076, 0x2077, 0x2078, 0x2079 ] 109 | 110 | ESCAPE = r"\033\[[0-9;]*[mK]" 111 | ANSIESCAPE = r'\033(?:\[[0-9;?]*[a-zA-Z]|][0-9]*;;.*?\\|\\)' 112 | KEYCODE_RE = re.compile(r'\x1B(?:[@-Z\\-_]|\[[0-?]*[ -/]*[@-~])') 113 | 114 | visible = lambda x: re.sub(ANSIESCAPE, "", x) 115 | # many characters have different widths 116 | visible_length = lambda x: sum(wcwidth(c) for c in visible(x)) 117 | extract_ansi_codes = lambda text: re.findall(ESCAPE, text) 118 | remove_ansi = lambda line, codeList: reduce(lambda line, code: line.replace(code, ''), codeList, line) 119 | split_up = lambda line: re.findall(r'(\x1b[^m]*m|[^\x1b]*)', line) 120 | 121 | def gettmpdir(): 122 | tmp_dir_all = os.path.join(tempfile.gettempdir(), "sd") 123 | os.makedirs(tmp_dir_all, mode=0o777, exist_ok=True) 124 | 125 | if os.name != 'nt': 126 | tmp_dir = os.path.join(tmp_dir_all, str(os.getuid())) 127 | else: 128 | tmp_dir = tmp_dir_all 129 | 130 | os.makedirs(tmp_dir, exist_ok=True) 131 | return tmp_dir 132 | 133 | def debug_write(text): 134 | if state.Logging: 135 | if state.Logging == True: 136 | state.Logging = tempfile.NamedTemporaryFile(dir=gettmpdir(), prefix="dbg", delete=False, mode="wb") 137 | state.Logging.write(text) 138 | 139 | def savebrace(): 140 | if state.Savebrace and state.code_buffer_raw and os.name != 'nt': 141 | path = os.path.join(gettmpdir(), 'savebrace') 142 | with open(path, "a") as f: 143 | f.write(state.code_buffer_raw + "\x00") 144 | f.flush() 145 | 146 | class Goto(Exception): 147 | pass 148 | 149 | class Style: 150 | pass 151 | 152 | class Code: 153 | Spaces = 'spaces' 154 | Backtick = 'backtick' 155 | Header = 'header' 156 | Body = 'body' 157 | Flush = 'flush' 158 | 159 | class ParseState: 160 | def __init__(self): 161 | self.buffer = b'' 162 | self.current_line = '' 163 | self.first_line = True 164 | self.last_line_empty = False 165 | self.is_pty = False 166 | self.is_exec = False 167 | self.maybe_prompt = False 168 | self.emit_flag = None 169 | self.scrape = None 170 | self.scrape_ix = 0 171 | self.terminal = None 172 | 173 | self.WidthArg = None 174 | self.WidthFull = None 175 | self.WidthWrap = False 176 | 177 | # If the entire block is indented this will 178 | # tell us what that is 179 | self.first_indent = None 180 | self.has_newline = False 181 | self.bg = BGRESET 182 | 183 | # These are part of a trick to get 184 | # streaming code blocks while preserving 185 | # multiline parsing. 186 | self.code_buffer = "" 187 | self.code_buffer_raw = "" 188 | self.code_gen = 0 189 | self.code_language = None 190 | self.code_first_line = False 191 | self.code_indent = 0 192 | self.code_line = '' 193 | 194 | self.ordered_list_numbers = [] 195 | self.list_item_stack = [] # stack of (indent, type) 196 | self.list_indent_text = 0 197 | 198 | self.in_list = False 199 | self.in_code = False # (Code.[Backtick|Spaces] | False) 200 | self.inline_code = False 201 | self.in_bold = False 202 | self.in_italic = False 203 | self.in_table = False # (Code.[Header|Body] | False) 204 | self.in_underline = False 205 | self.in_strikeout = False 206 | self.block_depth = 0 207 | 208 | self.exec_sub = None 209 | self.exec_master = None 210 | self.exec_slave = None 211 | self.exec_kb = 0 212 | 213 | self.exit = 0 214 | self.where_from = None 215 | 216 | def current(self): 217 | state = { 'inline': self.inline_code, 'code': self.in_code, 'bold': self.in_bold, 'italic': self.in_italic, 'underline': self.in_underline, 'strikeout': self.in_strikeout } 218 | state['none'] = all(item is False for item in state.values()) 219 | return state 220 | 221 | def reset_inline(self): 222 | self.inline_code = self.in_bold = self.in_italic = self.in_underline = self.in_strikeout = False 223 | 224 | def full_width(self, offset = 0): 225 | return offset + (state.current_width(listwidth = True) if Style.PrettyBroken else self.WidthFull) 226 | 227 | def current_width(self, listwidth = False): 228 | # this will double count the left margin 229 | return self.Width - (len(visible(self.space_left(listwidth)))) + Style.Margin 230 | 231 | def space_left(self, listwidth = False): 232 | pre = ' ' * (len(state.list_item_stack)) * Style.ListIndent if listwidth else '' 233 | return pre + Style.MarginSpaces + (Style.Blockquote * self.block_depth) if len(self.current_line) == 0 else "" 234 | 235 | state = ParseState() 236 | 237 | def override_background(style_name, background_color): 238 | base_style = get_style_by_name(style_name) 239 | base_style.background_color = background_color 240 | for i in base_style: 241 | i[1]['bgcolor'] = background_color 242 | for i,v in base_style.styles.items(): 243 | if v and 'bg' in v: 244 | base_style.styles[i] = re.sub(r'bg:[^ ]*', '', base_style.styles[i] ) 245 | for k,v in base_style._styles.items(): 246 | if v[4] != '': 247 | v[4] = '' 248 | 249 | return base_style 250 | 251 | def format_table(rowList): 252 | num_cols = len(rowList) 253 | row_height = 0 254 | wrapped_cellList = [] 255 | 256 | # Calculate max width per column (integer division) 257 | # Subtract num_cols + 1 for the vertical borders '│' 258 | available_width = state.current_width() - (num_cols * 2) 259 | 260 | width_base = available_width // num_cols 261 | width_mod = available_width % num_cols 262 | 263 | col_width_list = [width_base + (1 if i < width_mod else 0) for i in range(num_cols)] 264 | bg_color = Style.Mid if state.in_table == Style.Head else Style.Dark 265 | state.bg = f"{BG}{bg_color}" 266 | 267 | # First Pass: Wrap text and calculate row heights 268 | # Note this is where every cell is formatted so if 269 | # you are styling, do it before here! 270 | for ix in range(len(rowList)): 271 | row = rowList[ix] 272 | wrapped_cell = text_wrap(row, width=col_width_list[ix], force_truncate=True, preserve_format=True) 273 | 274 | # Ensure at least one line, even for empty cells 275 | if not wrapped_cell: 276 | wrapped_cell = [""] 277 | 278 | wrapped_cellList.append(wrapped_cell) 279 | row_height = max(row_height, len(wrapped_cell)) 280 | 281 | # --- Second Pass: Format and emit rows --- 282 | for ix in range(row_height): 283 | # This is the fancy row separator 284 | extra = f"\033[4;58;2;{Style.Mid}" if not state.in_table == Style.Head and (ix == row_height - 1) else "" 285 | line_segments = [] 286 | 287 | # Now we want to snatch this row index from all our cells 288 | for iy in range(len(wrapped_cellList)): 289 | cell = wrapped_cellList[iy] 290 | segment = '' 291 | if ix < len(cell): 292 | segment = cell[ix] 293 | 294 | # Margin logic is correctly indented here 295 | margin_needed = col_width_list[iy] - visible_length(segment) 296 | margin_segment = segment + (" " * max(0, margin_needed)) 297 | line_segments.append(f"{BG}{bg_color}{extra} {margin_segment}") 298 | 299 | # Correct indentation: This should be outside the c_idx loop 300 | joined_line = f"{BG}{bg_color}{extra}{FG}{Style.Symbol}│{RESET}".join(line_segments) 301 | # Correct indentation and add missing characters 302 | yield f"{state.space_left()}{FGRESET}{joined_line}{RESET}" 303 | 304 | state.bg = BGRESET 305 | 306 | def emit_h(level, text): 307 | text = line_format(text) 308 | lineList = text_wrap(text) 309 | res = [] 310 | for text in lineList: 311 | spaces_to_center = (state.current_width() - visible_length(text)) / 2 312 | if level == 1: # 313 | res.append(f"{state.space_left()}\n{state.space_left()}{BOLD[0]}{' ' * math.floor(spaces_to_center)}{text}{BOLD[1]}\n") 314 | elif level == 2: ## 315 | res.append(f"{state.space_left()}\n{state.space_left()}{BOLD[0]}{FG}{Style.Bright}{' ' * math.floor(spaces_to_center)}{text}{' ' * math.ceil(spaces_to_center)}{BOLD[1]}{FGRESET}") 316 | elif level == 3: ### 317 | res.append(f"{state.space_left()}{FG}{Style.Head}{BOLD[0]}{text}{BOLD[1]}{FGRESET}") 318 | elif level == 4: #### 319 | res.append(f"{state.space_left()}{FG}{Style.Symbol}{BOLD[0]}{text}{BOLD[1]}{FGRESET}") 320 | elif level == 5: ##### 321 | res.append(f"{state.space_left()}{text}{FGRESET}") 322 | else: 323 | res.append(f"{state.space_left()}{FG}{Style.Grey}{text}{FGRESET}") 324 | return "\n".join(res) 325 | 326 | def code_wrap(text_in): 327 | if not Style.PrettyBroken and state.WidthWrap and len(text_in) > state.full_width(): 328 | return (0, [text_in]) 329 | 330 | # get the indentation of the first line 331 | indent = len(text_in) - len(text_in.lstrip()) 332 | text = text_in.lstrip() 333 | mywidth = state.full_width(-4 if Style.PrettyBroken else 0) - indent 334 | 335 | # We take special care to preserve empty lines 336 | if len(text) == 0: 337 | return (0, [text_in]) 338 | res = [text[:mywidth]] 339 | 340 | for i in range(mywidth, len(text), mywidth): 341 | res.append(text[i : i + mywidth]) 342 | 343 | # sometimes just a newline wraps ... this isn't what we want actually 344 | if res[-1].strip() == '': 345 | res.pop() 346 | 347 | return (indent, res) 348 | 349 | 350 | # This marvelously obscure code "compacts" long lines of repetitive ANSI format strings by 351 | # removing duplicates. Here's how it works 352 | def ansi_collapse(codelist, inp): 353 | # We break SGR strings into various classes concerning their applicate or removal 354 | nums = { 355 | 'fg': r'3\d', 'bg': r'4\d', 356 | 'b': r'2?[12]', 'i': r'2?3', 'u': r'3?2', 357 | 'reset': '0' 358 | } 359 | 360 | # We have a routine that creates large regex matching strings for them based on 361 | # lists that can pass to it 362 | sgr = lambda l: re.compile(r'\x1b\[(' + '|'.join(l) +')[0-9;]*m') 363 | 364 | for stanza in inp: 365 | # We construct a named-register regex using the dictionary and run it 366 | # over a stanza of our input 367 | mg = re.search( sgr([f'(?P<{k}>{v})' for k, v in nums.items()]), stanza ) 368 | 369 | if mg: 370 | # this means we now have a dictionary populated with whether 371 | # we have those tags or not 372 | mg = mg.groupdict() 373 | 374 | # if it's a reset we can disregard everything 375 | if mg['reset']: 376 | return inp 377 | 378 | # Find the tags we have by doing a dictionary None check. Make new regex SGR ANSI codes from it 379 | my_filter = sgr( [nums[k] for k, v in mg.items() if v] ) 380 | 381 | # Use that code list as a filter to remove extra 382 | codelist = list(filter(lambda x: not re.search( my_filter, x ), codelist)) 383 | 384 | return codelist + inp 385 | 386 | 387 | def split_text(text): 388 | return [x for x in re.split( 389 | r'(?<=[' 390 | r'\u3000-\u303F' 391 | r'\u4E00-\u9FFF' 392 | r'\u3400-\u4DBF' 393 | r'\uF900-\uFAFF' 394 | r'])|(?=[' 395 | #r'\u4E00-\u9FFF' 396 | r'\u3400-\u4DBF' 397 | r'\uF900-\uFAFF' 398 | r'])|\s+', 399 | text 400 | ) if x] 401 | 402 | def text_wrap(text, width = -1, indent = 0, first_line_prefix="", subsequent_line_prefix="", force_truncate=False, preserve_format=False): 403 | if width == -1: 404 | width = state.Width 405 | 406 | # The empty word clears the buffer at the end. 407 | formatted = line_format(text) 408 | words = split_text(formatted) + [""] 409 | 410 | lines = [] 411 | current_line = "" 412 | current_style = [] 413 | resetter = "" if preserve_format else FORMATRESET 414 | 415 | oldword = '' 416 | for word in words: 417 | # we apply the style if we see it at the beginning of the word 418 | codes = extract_ansi_codes(word) 419 | if len(codes) and word.startswith(codes[0]): 420 | # this pop(0) is intentional 421 | current_style.append(codes.pop(0)) 422 | 423 | if len(word) and visible_length(current_line) + visible_length(word) + 1 <= width: # +1 for space 424 | space = "" 425 | if len(visible(word)) > 0 and current_line: 426 | space = " " 427 | if (":" in visible(word) or cjk_count(word)) and cjk_count(oldword): 428 | space = "" 429 | current_line += space + word 430 | else: 431 | # Word doesn't fit, finalize the previous line 432 | prefix = first_line_prefix if not lines else subsequent_line_prefix 433 | line_content = prefix + current_line 434 | # This is expensive, fix. 435 | while force_truncate and visible_length(line_content) >= width: 436 | line_content = line_content[:len(line_content) - 2] + "…" 437 | 438 | margin = max(0, width - visible_length(line_content)) 439 | 440 | if line_content.strip() != "": 441 | # We make absolutely positively sure beyond any doubt 442 | # that we have closed our hyperlink OSC 443 | if LINK[0] in line_content: 444 | line_content += LINK[1] 445 | lines.append(line_content + resetter + state.bg + ' ' * margin) 446 | 447 | current_line = (" " * indent) + "".join(current_style) + word 448 | 449 | if len(codes): 450 | current_style += codes 451 | 452 | if codes: 453 | current_style = ansi_collapse(current_style, codes) 454 | 455 | oldword = word 456 | 457 | if len(lines) < 1: 458 | return [] 459 | 460 | if len(lines) == 1: 461 | lines[0] = lines[0].rstrip() 462 | 463 | return lines 464 | 465 | def cjk_count(s): 466 | cjk_re = re.compile( 467 | r'[\u4E00-\u9FFF' # CJK Unified Ideographs 468 | r'\u3400-\u4DBF' # CJK Unified Ideographs Extension A 469 | r'\uF900-\uFAFF' # CJK Compatibility Ideographs 470 | r'\uFF00-\uFFEF' # CJK Compatibility Punctuation 471 | r'\u3000-\u303F' # CJK Symbols and Punctuation 472 | r'\U0002F800-\U0002FA1F]' # CJK Compatibility Ideographs Supplement 473 | ) 474 | 475 | return len(cjk_re.findall(visible(s))) 476 | 477 | def line_format(line): 478 | not_text = lambda token: not (token.isalnum() or token in ['\\','"']) or cjk_count(token) 479 | footnotes = lambda match: ''.join([chr(SUPER[int(i)]) for i in match.group(1)]) 480 | 481 | def process_images(match): 482 | url = match.group(2) 483 | try: 484 | if re.match(r"https://", url.lower()): 485 | image = from_url(url) 486 | else: 487 | image = from_file(url) 488 | image.height = 20 489 | print(f"{image:|.-1#}") 490 | except: 491 | return match.group(2) 492 | 493 | # Apply OSC 8 hyperlink formatting after other formatting 494 | def process_links(match): 495 | description = match.group(1) 496 | url = match.group(2) 497 | return f'{LINK[0]}{url}\033\\{Style.Link}{description}{UNDERLINE[1]}{LINK[1]}{FGRESET}' 498 | 499 | line = re.sub(r"\!\[([^\]]*)\]\(([^\)]+)\)", process_images, line) 500 | line = re.sub(r"\[([^\]]+)\]\(([^\)]+)\)", process_links, line) 501 | line = re.sub(r"\[\^(\d+)\]:?", footnotes, line) 502 | 503 | tokenList = re.finditer(r"((~~|\*\*_|_\*\*|\*{1,3}|_{1,3}|`+)|[^~_*`]+)", line) 504 | result = "" 505 | 506 | last_pos = 0 507 | for match in tokenList: 508 | if match.span()[0] > last_pos: 509 | result += line[last_pos:match.span()[0]] 510 | 511 | last_pos = match.span()[1] 512 | token = re.sub(r'\s+',' ', match.group(1)) 513 | next_token = line[match.end()] if match.end() < len(line) else "" 514 | prev_token = line[match.start()-1] if match.start() > 0 else "" 515 | 516 | # This trick makes sure that things like `` ` `` render right. 517 | if "`" in token and (not state.inline_code or state.inline_code == token): 518 | if state.inline_code: 519 | if ' ' in state.inline_code: 520 | savebrace() 521 | state.inline_code = False 522 | else: 523 | state.inline_code = token 524 | state.code_buffer_raw = '' 525 | 526 | if state.inline_code: 527 | result += f'{BG}{Style.Mid}' 528 | else: 529 | result += state.bg 530 | state.code_buffer_raw = '' 531 | 532 | # This is important here because we ignore formatting 533 | # inside of our code block. 534 | elif state.inline_code: 535 | result += token 536 | state.code_buffer_raw += token 537 | 538 | elif token == '~~' and (state.in_strikeout or not_text(prev_token)): 539 | state.in_strikeout = not state.in_strikeout 540 | result += STRIKEOUT[0] if state.in_strikeout else STRIKEOUT[1] 541 | 542 | elif token in ['**_','_**','___','***'] and (state.in_bold or not_text(prev_token)): 543 | state.in_bold = not state.in_bold 544 | result += BOLD[0] if state.in_bold else BOLD[1] 545 | state.in_italic = not state.in_italic 546 | result += ITALIC[0] if state.in_italic else ITALIC[1] 547 | 548 | elif (token == '__' or token == "**") and (state.in_bold or not_text(prev_token)): 549 | state.in_bold = not state.in_bold 550 | result += BOLD[0] if state.in_bold else BOLD[1] 551 | 552 | elif token == "*" and (state.in_italic or not_text(prev_token)): 553 | # This is the use case of talking about * and then following 554 | # up on something as opposed to *like this*. 555 | if state.in_italic or (not state.in_italic and next_token != ' '): 556 | state.in_italic = not state.in_italic 557 | result += ITALIC[0] if state.in_italic else ITALIC[1] 558 | else: 559 | result += token 560 | 561 | elif token == "_" and (state.in_underline or (not_text(prev_token) and next_token.isalnum())): 562 | state.in_underline = not state.in_underline 563 | result += UNDERLINE[0] if state.in_underline else UNDERLINE[1] 564 | else: 565 | result += token 566 | 567 | return result 568 | 569 | def parse(stream): 570 | last_line_empty_cache = None 571 | byte = None 572 | TimeoutIx = 0 573 | lexer = None 574 | while True: 575 | if state.is_pty or state.is_exec: 576 | byte = None 577 | ready_in, _, _ = select.select( 578 | [stream.fileno(), state.exec_master], [], [], state.Timeout) 579 | 580 | if state.is_exec: 581 | # This is keyboard input 582 | if stream.fileno() in ready_in: 583 | byte = os.read(stream.fileno(), 1) 584 | 585 | state.exec_kb += 1 586 | os.write(state.exec_master, byte) 587 | 588 | if byte in [b'\n', b'\r']: 589 | state.buffer = b'' 590 | print("") 591 | state.exec_kb = 0 592 | else: 593 | continue 594 | 595 | if state.exec_master in ready_in: 596 | TimeoutIx = 0 597 | byte = os.read(state.exec_master, 1) 598 | 599 | if state.exec_kb: 600 | os.write(sys.stdout.fileno(), byte) 601 | 602 | if len(ready_in) == 0: 603 | TimeoutIx += 1 604 | 605 | elif stream.fileno() in ready_in: 606 | byte = os.read(stream.fileno(), 1) 607 | TimeoutIx = 0 608 | elif TimeoutIx == 0: 609 | # This is our record separator for debugging - hands peaking 610 | debug_write("🫣".encode('utf-8')) 611 | TimeoutIx += 1 612 | 613 | else: 614 | byte = stream.read(1) 615 | 616 | if byte is not None: 617 | if byte == b'': break 618 | state.buffer += byte 619 | debug_write(byte) 620 | 621 | if not (byte == b'\n' or byte is None): continue 622 | 623 | line = state.buffer.decode('utf-8').replace('\t',' ') 624 | state.has_newline = line.endswith('\n') 625 | # I hate this. There should be better ways. 626 | state.maybe_prompt = not state.has_newline and state.current()['none'] and re.match(r'^.*>\s+$', visible(line)) 627 | 628 | # let's wait for a newline 629 | if state.maybe_prompt: 630 | state.emit_flag = Code.Flush 631 | yield line 632 | state.current_line = '' 633 | state.buffer = b'' 634 | 635 | if not state.has_newline: 636 | continue 637 | 638 | state.buffer = b'' 639 | """ 640 | # Run through the plugins first 641 | res = latex.Plugin(line, state, Style) 642 | if res is True: 643 | # This means everything was consumed by our plugin and 644 | # we should continue 645 | continue 646 | elif res is not None: 647 | for row in res: 648 | yield row 649 | continue 650 | """ 651 | 652 | # running this here avoids stray | 653 | block_match = re.match(r"^\s*((>\s*)+|<.?think>)", line) 654 | if not state.in_code and block_match: 655 | if block_match.group(1) == '': 656 | state.block_depth = 0 657 | yield RESET 658 | elif block_match.group(1) == '': 659 | state.block_depth = 1 660 | else: 661 | state.block_depth = block_match.group(0).count('>') 662 | # we also need to consume those tokens 663 | line = line[len(block_match.group(0)):] 664 | else: 665 | if state.block_depth > 0: 666 | yield FGRESET 667 | state.block_depth = 0 668 | 669 | # --- Collapse Multiple Empty Lines if not in code blocks --- 670 | if not state.in_code: 671 | is_empty = line.strip() == "" 672 | 673 | if is_empty and state.last_line_empty: 674 | continue # Skip processing this line 675 | elif is_empty: 676 | state.last_line_empty = True 677 | yield state.space_left() 678 | continue 679 | else: 680 | last_line_empty_cache = state.last_line_empty 681 | state.last_line_empty = False 682 | 683 | # This is to reset our top-level line-based systems 684 | # \n buffer 685 | if not state.in_list and len(state.ordered_list_numbers) > 0: 686 | state.ordered_list_numbers[0] = 0 687 | elif (not line.startswith(' ' * state.list_indent_text)) and line.strip() != "": 688 | state.in_list = False 689 | state.list_indent_text = 0 690 | 691 | if state.first_indent is None: 692 | state.first_indent = len(line) - len(line.lstrip()) 693 | if len(line) - len(line.lstrip()) >= state.first_indent: 694 | line = line[state.first_indent:] 695 | else: 696 | logging.debug("Indentation decreased from first line.") 697 | 698 | 699 | # Indent guaranteed 700 | 701 | # in order to stream tables and keep track of the headers we need to know whether 702 | # we are in table or not table otherwise > 1 tables won't have a stylized header 703 | if state.in_table and not state.in_code and not re.match(r"^\s*\|.+\|\s*$", line): 704 | state.in_table = False 705 | 706 | #
 707 |         if not state.in_code:
 708 |             code_match = re.match(r"^\s*(```|
)\s*([^\s]+|$)\s*$", line)
 709 |             if code_match:
 710 |                 state.in_code = Code.Backtick
 711 |                 state.code_indent = len(line) - len(line.lstrip())
 712 |                 state.code_language = code_match.group(2) or 'Bash'
 713 | 
 714 |             elif state.CodeSpaces and last_line_empty_cache and not state.in_list:
 715 |                 code_match = re.match(r"^    \s*[^\s\*]", line)
 716 |                 if code_match:
 717 |                     state.in_code = Code.Spaces
 718 |                     state.code_language = 'Bash'
 719 | 
 720 |             if state.in_code:
 721 |                 state.code_buffer = state.code_buffer_raw = ""
 722 |                 state.code_gen = 0
 723 |                 state.code_first_line = True
 724 |                 state.bg = f"{BG}{Style.Dark}"
 725 |                 state.where_from = "code pad"
 726 |                 if Style.PrettyPad or Style.PrettyBroken:
 727 |                     if not Style.PrettyPad:
 728 |                         yield ""
 729 | 
 730 |                     yield Style.Codepad[0]
 731 |                 else:
 732 |                     yield ""
 733 | 
 734 |                 logging.debug(f"In code: ({state.in_code})")
 735 | 
 736 |                 if state.in_code == Code.Backtick:
 737 |                     continue
 738 | 
 739 |         if state.in_code:
 740 |             try:
 741 |                 # This is turning it OFF
 742 |                 if ( (                     state.in_code == Code.Backtick and     line.strip() in ["
", "```"] ) or 743 | (state.CodeSpaces and state.in_code == Code.Spaces and not line.startswith(' ')) ): 744 | if state.scrape: 745 | ext = "sh" 746 | try: 747 | ext = get_lexer_by_name(state.code_language).filenames[0].split('.')[-1] 748 | except: 749 | logging.warning(f"Can't find canonical extension for {state.code_language}") 750 | pass 751 | 752 | open(os.path.join(state.scrape, f"file_{state.scrape_ix}.{ext}"), 'w').write(state.code_buffer_raw) 753 | state.scrape_ix += 1 754 | 755 | savebrace() 756 | state.code_language = None 757 | state.code_indent = 0 758 | code_type = state.in_code 759 | state.in_code = False 760 | state.bg = BGRESET 761 | 762 | state.where_from = "code pad" 763 | if Style.PrettyPad or Style.PrettyBroken: 764 | yield Style.Codepad[1] 765 | if not Style.PrettyPad: 766 | yield "" 767 | 768 | else: 769 | yield RESET 770 | 771 | logging.debug(f"code: {state.in_code}") 772 | state.emit_flush = True 773 | # We suppress the newline - it's not an explicit style 774 | #state.has_newline = False 775 | #yield RESET 776 | 777 | if code_type == Code.Backtick: 778 | continue 779 | else: 780 | # otherwise we don't want to consume 781 | # nor do we want to be here. 782 | raise Goto() 783 | 784 | if state.code_first_line or lexer is None: 785 | state.code_first_line = False 786 | try: 787 | lexer = get_lexer_by_name(state.code_language) 788 | custom_style = override_background(Style.Syntax, ansi2hex(Style.Dark)) 789 | except pygments.util.ClassNotFound as e: 790 | logging.debug(e) 791 | lexer = get_lexer_by_name("Bash") 792 | custom_style = override_background("default", ansi2hex(Style.Dark)) 793 | 794 | formatter = TerminalTrueColorFormatter(style=custom_style) 795 | if line.startswith(' ' * state.code_indent): 796 | line = line[state.code_indent :] 797 | 798 | elif line.startswith(" " * state.code_indent): 799 | line = line[state.code_indent :] 800 | 801 | # By now we have the properly stripped code line 802 | # in the line variable. Add it to the buffer. 803 | state.code_buffer_raw += line 804 | state.code_line += line 805 | if state.code_line.endswith('\n'): 806 | line = state.code_line 807 | state.code_line = '' 808 | else: 809 | continue 810 | 811 | highlighted_code = highlight(line, lexer, formatter) 812 | indent, line_wrap = code_wrap(line) 813 | 814 | state.where_from = "in code" 815 | pre = [state.space_left(listwidth = True), ' '] if Style.PrettyBroken else ['', ''] 816 | 817 | for tline in line_wrap: 818 | # wrap-around is a bunch of tricks. We essentially format longer and longer portions of code. The problem is 819 | # the length can change based on look-ahead context so we need to use our expected place (state.code_gen) and 820 | # then naively search back until our visible_lengths() match. This is not fast and there's certainly smarter 821 | # ways of doing it but this thing is way trickery than you think 822 | highlighted_code = highlight(state.code_buffer + tline, lexer, formatter) 823 | #print("(",bytes(highlighted_code,'utf-8'),")") 824 | parts = split_up(highlighted_code) 825 | 826 | # Sometimes the highlighter will do things like a full reset or a background reset. 827 | # This is mostly not what we want 828 | parts = [ re.sub(r"\033\[[34]9(;00|)m", FORMATRESET, x) for x in parts] 829 | 830 | # Since we are streaming we ignore the resets and newlines at the end 831 | while parts[-1] in [FGRESET, FORMATRESET]: 832 | parts.pop() 833 | 834 | tline_len = visible_length(tline) 835 | 836 | # now we find the new stuff: 837 | ttl = 0 838 | for i in range(len(parts)-1, 0, -1): 839 | idx = parts[i] 840 | if len(idx) == 0: 841 | continue 842 | 843 | ttl += len(idx) if idx[0] != '\x1b' else 0 844 | 845 | if ttl > 1+tline_len: 846 | break 847 | 848 | 849 | newlen = visible_length("".join(parts[i:])) 850 | 851 | snipfrom = newlen - len(tline) + 2 852 | # this is all getting replaced with the new lexer so let's give a cheap 853 | # fix for now: 854 | if snipfrom == 1: 855 | snipfrom = 0 856 | 857 | if snipfrom > 0: 858 | parts[i] = parts[i][snipfrom:] 859 | 860 | state.code_buffer += tline 861 | this_batch = "".join(parts[i:]) 862 | 863 | if this_batch.startswith(FGRESET): 864 | this_batch = this_batch[len(FGRESET) :] 865 | 866 | # clean it before prepending with potential format 867 | this_batch = this_batch.strip() 868 | while i - 1 >= 0 and parts[i-1] and parts[i-1][0] == '\x1b': 869 | this_batch = parts[i-1] + this_batch 870 | i -= 1 871 | 872 | ## this is the crucial counter that will determine 873 | # the beginning of the next line 874 | state.code_gen = len(highlighted_code) 875 | code_line = ' ' * indent + this_batch.strip() 876 | 877 | margin = state.full_width( -len(pre[1]) ) - visible_length(code_line) % state.WidthFull 878 | yield f"{pre[0]}{Style.Codebg}{pre[1]}{code_line}{FORMATRESET}{' ' * max(0, margin)}{BGRESET}" 879 | continue 880 | except Goto: 881 | pass 882 | 883 | except Exception as ex: 884 | logging.warning(f"Code parsing error: {ex}") 885 | traceback.print_exc() 886 | pass 887 | 888 | # 889 | if re.match(r"^\s*\|.+\|\s*$", line) and not state.in_code: 890 | cells = [c.strip() for c in line.strip().strip("|").split("|")] 891 | 892 | # This guarantees we are at the first line 893 | # \n buffer 894 | if not state.in_table: 895 | state.in_table = Style.Head 896 | 897 | elif state.in_table == Style.Head: 898 | # we ignore the separator, this is just a check 899 | if not re.match(r"^[\s|:-]+$", line): 900 | logging.warning(f"Table definition row 2 was NOT a separator. Instead it was:\n({line})") 901 | 902 | # Let's assume everything worked out I guess. 903 | # We set our header to false and basically say we are expecting the body 904 | state.in_table = Code.Body 905 | continue 906 | 907 | yield from format_table(cells) 908 | continue 909 | 910 | #
      1. 911 | # llama-4 maverick uses + and +- for lists ... for some reason 912 | content = line 913 | bullet = ' ' 914 | list_item_match = re.match(r"^(\s*)([\+*\-] |\+\-+|\d+\.\s+)(.*)", line) 915 | if list_item_match: 916 | # llama 4 maverick does this weird output like this 917 | # 1. blah blah blah 918 | # this should be a list 919 | # 920 | # ```bash 921 | # blah blah 922 | # ``` 923 | # 924 | # still in the list 925 | # We do this here so that the first line which is the bullet 926 | # line gets the proper hang 927 | state.list_indent_text = len(list_item_match.group(2)) - 1 928 | state.in_list = True 929 | 930 | indent = len(list_item_match.group(1)) 931 | 932 | list_type = "number" if list_item_match.group(2)[0].isdigit() else "bullet" 933 | content = list_item_match.group(3) 934 | 935 | # Handle stack 936 | while state.list_item_stack and state.list_item_stack[-1][0] > indent: 937 | state.list_item_stack.pop() # Remove deeper nested items 938 | if state.ordered_list_numbers: 939 | state.ordered_list_numbers.pop() 940 | if state.list_item_stack and state.list_item_stack[-1][0] < indent: 941 | # new nested list 942 | state.list_item_stack.append((indent, list_type)) 943 | state.ordered_list_numbers.append(0) 944 | elif not state.list_item_stack: 945 | # first list 946 | state.list_item_stack.append((indent, list_type)) 947 | state.ordered_list_numbers.append(0) 948 | if list_type == "number": 949 | state.ordered_list_numbers[-1] += 1 950 | 951 | bullet = '•' 952 | if list_type == "number": 953 | list_number = int(max(state.ordered_list_numbers[-1], float(list_item_match.group(2)))) 954 | bullet = str(list_number) 955 | 956 | # This is intentional ... we can get here in llama 4 using 957 | # a weird thing 958 | if state.in_list: 959 | indent = (len(state.list_item_stack) - 1) * Style.ListIndent #+ (len(bullet) - 1) 960 | wrap_width = state.current_width(listwidth = True) - Style.ListIndent 961 | 962 | wrapped_lineList = text_wrap(content, wrap_width, Style.ListIndent, 963 | first_line_prefix = f"{(' ' * indent)}{FG}{Style.Symbol}{bullet}{RESET} ", 964 | subsequent_line_prefix = " " * (indent) 965 | ) 966 | for wrapped_line in wrapped_lineList: 967 | yield f"{state.space_left()}{wrapped_line}\n" 968 | 969 | continue 970 | 971 | #

        ...

        972 | header_match = re.match(r"^\s*(#{1,6})\s*(.*)", line) 973 | if header_match: 974 | level = len(header_match.group(1)) 975 | yield emit_h(level, header_match.group(2)) 976 | continue 977 | 978 | #
        979 | hr_match = re.match(r"^[\s]*([-\*=_]){3,}[\s]*$", line) 980 | if hr_match: 981 | if state.last_line_empty or last_line_empty_cache: 982 | # print a horizontal rule using a unicode midline 983 | yield f"{Style.MarginSpaces}{FG}{Style.Symbol}{'─' * state.Width}{RESET}" 984 | else: 985 | # We tell the next level up that the beginning of the buffer should be a flag. 986 | # Underneath this condition it will no longer yield 987 | state.emit_flag = 1 if '-' in hr_match.groups(1) else 2 988 | yield "" 989 | continue 990 | 991 | state.where_from = "emit_normal" 992 | 993 | # if we've gotten to an emit normal then we can assert that our list stack should 994 | # be empty. This is a hack. 995 | state.list_item_stack = [] 996 | 997 | if len(line) == 0: yield "" 998 | if visible_length(line) < state.Width: 999 | # we want to prevent word wrap 1000 | yield f"{state.space_left()}{line_format(line.lstrip())}" 1001 | else: 1002 | wrapped_lines = text_wrap(line) 1003 | for wrapped_line in wrapped_lines: 1004 | yield f"{state.space_left()}{wrapped_line}\n" 1005 | 1006 | def emit(inp): 1007 | buffer = [] 1008 | flush = False 1009 | for chunk in parse(inp): 1010 | width_calc() 1011 | if state.emit_flag: 1012 | if state.emit_flag == Code.Flush: 1013 | flush = True 1014 | state.emit_flag = None 1015 | else: 1016 | buffer[0] = emit_h(state.emit_flag, buffer[0]) 1017 | state.emit_flag = None 1018 | continue 1019 | 1020 | if not state.has_newline: 1021 | chunk = chunk.rstrip("\n") 1022 | elif not chunk.endswith("\n"): 1023 | chunk += "\n" 1024 | 1025 | if chunk.endswith("\n"): 1026 | state.current_line = '' 1027 | else: 1028 | state.current_line += chunk 1029 | 1030 | buffer.append(chunk) 1031 | # This *might* be dangerous 1032 | state.reset_inline() 1033 | 1034 | if flush: 1035 | chunk = "\n".join(buffer) 1036 | buffer = [] 1037 | flush = False 1038 | 1039 | elif len(buffer) == 1: 1040 | continue 1041 | 1042 | else: 1043 | chunk = buffer.pop(0) 1044 | 1045 | print(chunk, end="", file=sys.stdout, flush=True) 1046 | 1047 | if len(buffer): 1048 | print(buffer.pop(0), file=sys.stdout, end="", flush=True) 1049 | 1050 | def ansi2hex(ansi_code): 1051 | parts = ansi_code.strip('m').split(";") 1052 | r, g, b = map(int, parts) 1053 | return f"#{r:02x}{g:02x}{b:02x}" 1054 | 1055 | def apply_multipliers(style, name, H, S, V): 1056 | m = style.get(name) 1057 | r, g, b = colorsys.hsv_to_rgb(min(1.0, H * m["H"]), min(1.0, S * m["S"]), min(1.0, V * m["V"])) 1058 | return ';'.join([str(int(x * 255)) for x in [r, g, b]]) + "m" 1059 | 1060 | def width_calc(): 1061 | if state.WidthArg: 1062 | width = state.WidthArg 1063 | else: 1064 | try: 1065 | width = shutil.get_terminal_size().columns 1066 | state.WidthWrap = True 1067 | except (AttributeError, OSError): 1068 | # this means it's a pager, we can just ignore the base64 clipboard 1069 | width = 80 1070 | pass 1071 | 1072 | 1073 | # This can't be done because our list item stack can change as well so 1074 | # unless we want to track that too, we're SOL 1075 | #if state.WidthFull == width: 1076 | # return 1077 | 1078 | state.WidthFull = width 1079 | 1080 | state.Width = state.WidthFull - 2 * Style.Margin 1081 | pre = state.space_left(listwidth=True) if Style.PrettyBroken else '' 1082 | design = [FG, '▄','▀'] if Style.PrettyPad else [BG, ' ',' '] 1083 | Style.Codepad = [ 1084 | f"{pre}{RESET}{design[0]}{Style.Dark}{design[1] * state.full_width()}{RESET}\n", 1085 | f"{pre}{RESET}{design[0]}{Style.Dark}{design[2] * state.full_width()}{RESET}" 1086 | ] 1087 | 1088 | def main(): 1089 | parser = ArgumentParser( 1090 | formatter_class=argparse.RawDescriptionHelpFormatter, description=textwrap.dedent(f""" 1091 | Streamdown is a streaming markdown renderer for modern terminals. 1092 | https://github.com/day50-dev/Streamdown 1093 | 1094 | paths: 1095 | config {os.path.join(appdirs.user_config_dir('streamdown'), 'config.toml')} 1096 | logs {gettmpdir()} 1097 | """)) 1098 | parser.add_argument("filenameList", nargs="*", help="Input file to process (also takes stdin)") 1099 | parser.add_argument("-l", "--loglevel", default="INFO", help="Set the logging level") 1100 | parser.add_argument("-b", "--base", default=None, help="Set the hsv base: h,s,v") 1101 | parser.add_argument("-c", "--config", default=None, help="Use a custom config override") 1102 | parser.add_argument("-w", "--width", default="0", help="Set the width WIDTH") 1103 | parser.add_argument("-e", "--exec", help="Wrap a program EXEC for more 'proper' i/o handling") 1104 | parser.add_argument("-s", "--scrape", help="Scrape code snippets to a directory SCRAPE") 1105 | parser.add_argument("-v", "--version", action="store_true", help="Show version information") 1106 | args = parser.parse_args() 1107 | 1108 | if args.version: 1109 | try: 1110 | import importlib.metadata 1111 | print(importlib.metadata.version("streamdown")) 1112 | except importlib.metadata.PackageNotFoundError: 1113 | print(subprocess.run( 1114 | ['git', 'describe', '--always', '--dirty', '--tags'], 1115 | cwd=os.path.dirname(os.path.abspath(__file__)), 1116 | stdout=subprocess.PIPE, 1117 | text=True 1118 | ).stdout.strip()) 1119 | 1120 | sys.exit(0) 1121 | 1122 | config = ensure_config_file(args.config) 1123 | style = toml.loads(default_toml).get('style') | config.get("style", {}) 1124 | features = toml.loads(default_toml).get('features') | config.get("features", {}) 1125 | H, S, V = style.get("HSV") 1126 | 1127 | if args.base: 1128 | env_colors = args.base.split(",") 1129 | if len(env_colors) > 0: H = float(env_colors[0]) 1130 | if len(env_colors) > 1: S = float(env_colors[1]) 1131 | if len(env_colors) > 2: V = float(env_colors[2]) 1132 | 1133 | for color in ["Dark", "Mid", "Symbol", "Head", "Grey", "Bright"]: 1134 | setattr(Style, color, apply_multipliers(style, color, H, S, V)) 1135 | for attr in ['PrettyPad', 'PrettyBroken', 'Margin', 'ListIndent', 'Syntax']: 1136 | setattr(Style, attr, style.get(attr)) 1137 | for attr in ['CodeSpaces', 'Clipboard', 'Logging', 'Timeout', 'Savebrace']: 1138 | setattr(state, attr, features.get(attr)) 1139 | 1140 | 1141 | if args.scrape: 1142 | os.makedirs(args.scrape, exist_ok=True) 1143 | state.scrape = args.scrape 1144 | 1145 | Style.MarginSpaces = " " * Style.Margin 1146 | state.WidthArg = int(args.width) or style.get("Width") or 0 1147 | Style.Blockquote = f"{FG}{Style.Grey}│ " 1148 | width_calc() 1149 | 1150 | Style.Codebg = f"{BG}{Style.Dark}" 1151 | Style.Link = f"{FG}{Style.Symbol}{UNDERLINE[0]}" 1152 | 1153 | logging.basicConfig(stream=sys.stdout, level=args.loglevel.upper(), format=f'%(message)s') 1154 | if os.name != 'nt': 1155 | state.exec_master, state.exec_slave = pty.openpty() 1156 | try: 1157 | inp = sys.stdin 1158 | if args.exec and os.name != 'nt': 1159 | state.terminal = termios.tcgetattr(sys.stdin) 1160 | state.is_exec = True 1161 | state.exec_sub = subprocess.Popen(args.exec.split(' '), stdin=state.exec_slave, stdout=state.exec_slave, stderr=state.exec_slave, close_fds=True) 1162 | os.close(state.exec_slave) # We don't need slave in parent 1163 | # Set stdin to raw mode so we don't need to press enter 1164 | tty.setcbreak(sys.stdin.fileno()) 1165 | sys.stdout.write("\x1b[?7h") 1166 | emit(inp) 1167 | 1168 | elif args.filenameList: 1169 | # Let's say we only care about logging in streams 1170 | state.Logging = False 1171 | for fname in args.filenameList: 1172 | if len(args.filenameList) > 1: 1173 | emit(BytesIO(f"\n------\n# {fname}\n\n------\n".encode('utf-8'))) 1174 | emit(open(fname, "rb")) 1175 | 1176 | elif sys.stdin.isatty(): 1177 | parser.print_help() 1178 | sys.exit() 1179 | else: 1180 | # this is a more sophisticated thing that we'll do in the main loop 1181 | state.is_pty = True 1182 | os.set_blocking(inp.fileno(), False) 1183 | emit(inp) 1184 | 1185 | except (OSError, KeyboardInterrupt): 1186 | state.exit = 130 1187 | 1188 | except Exception as ex: 1189 | if state.terminal and os.name != 'nt': 1190 | termios.tcsetattr(sys.stdin, termios.TCSADRAIN, state.terminal) 1191 | logging.warning(f"Exception thrown: {type(ex)} {ex}") 1192 | traceback.print_exc() 1193 | 1194 | if os.isatty(sys.stdout.fileno()) and state.Clipboard and state.code_buffer_raw: 1195 | code = state.code_buffer_raw 1196 | # code needs to be a base64 encoded string before emitting 1197 | code_bytes = code.encode('utf-8') 1198 | base64_bytes = base64.b64encode(code_bytes) 1199 | base64_string = base64_bytes.decode('utf-8') 1200 | print(f"\033]52;c;{base64_string}\a", end="", flush=True) 1201 | 1202 | if state.terminal: 1203 | termios.tcsetattr(sys.stdin, termios.TCSADRAIN, state.terminal) 1204 | os.close(state.exec_master) 1205 | if state.exec_sub: 1206 | state.exec_sub.wait() 1207 | 1208 | print(RESET, end="") 1209 | sys.exit(state.exit) 1210 | 1211 | if __name__ == "__main__": 1212 | main() 1213 | -------------------------------------------------------------------------------- /tests/README.md: -------------------------------------------------------------------------------- 1 | ### Various streamdown tests 2 | 3 | This is not a formal testing system but is more "showcase based" testing, driving features and looking at output. 4 | 5 | There's two drivers: 6 | 7 | * chunk-buffer.sh: The `Logging` config of the `sd.py` parser injects a peeking emoji to specify when the `Timeout` was hit and the render cycle was run. This timeout can cause issues. chunk-buffer will re-ingest these for diagnostic purposes 8 | 9 | * line-buffer.sh: Some parts of the parser waits for newlines, and this tool will feed line by line. 10 | 11 | They both accept a TIMEOUT env variable 12 | -------------------------------------------------------------------------------- /tests/backtick-with-post-spaces.md: -------------------------------------------------------------------------------- 1 | Chatting with openrouter/google/gemma-3-27b-it:free 2 | Type 'exit' or 'quit' to exit 3 | Type '!multi' to enter multiple lines, then '!end' to finish 4 | > how do i get the relative path of a file in pythonoapath of a file in python 5 | 6 | You have several ways to get the path of a file in Python, depending on what you mean by "path" (absolute, relative, etc.) and what you already have (filename, file object, etc.). Here's a breakdown with examples using the `os.path` and `pathlib` modules, which are the standard ways to work with paths: 7 | 8 | **1. Using `os.path` (Older, but still widely used)** 9 | 10 | * **Getting the absolute path:** If you have a filename (relative or absolute), you can get the absolute path. 11 | 12 | ```python 13 | import os 14 | 15 | filename = "my_file.txt" # Could be a relative path like this 16 | absolute_path = os.path.abspath(filename) 17 | print(absolute_path) 18 | ``` 19 | 20 | * **Getting the current working directory:** This is often useful as a starting point for relative paths. 21 | 22 | ```python 23 | import os 24 | 25 | current_directory = os.getcwd() 26 | print(current_directory) 27 | ``` 28 | 29 | * **Joining paths:** Constructing paths from parts. 30 | 31 | ```python 32 | import os 33 | 34 | directory = "/home/user/documents" 35 | filename = "report.pdf" 36 | full_path = os.path.join(directory, filename) 37 | print(full_path) # Output: /home/user/documents/report.pdf 38 | ``` 39 | 40 | * **Getting the directory name:** 41 | 42 | ```python 43 | import os 44 | 45 | filename = "/path/to/my_file.txt" 46 | directory = os.path.dirname(filename) 47 | print(directory) # Output: /path/to 48 | ``` 49 | 50 | * **Getting the filename from a path:** 51 | 52 | ```python 53 | import os 54 | 55 | filepath = "/path/to/my_file.txt" 56 | filename = os.path.basename(filepath) 57 | print(filename) # Output: my_file.txt 58 | ``` 59 | 60 | * **Checking if a path exists:** 61 | 62 | ```python 63 | import os 64 | 65 | filepath = "/path/to/my_file.txt" 66 | if os.path.exists(filepath): 67 | print("File exists") 68 | else: 69 | print("File does not exist") 70 | ``` 71 | 72 | **2. Using `pathlib` ( More modern and object-oriented)** 73 | 74 | * **Creating a Path object:** This is the foundation of `pathlib`. q 75 | 76 | ```python 77 | from pathlib import Path 78 | 79 | filename = "my_file.txt" 80 | path_object q = Path(filename^[[B) # Creates a Path object representing the file 81 | print(path_object) 82 | ``` 83 | 84 | * **Getting the absolute path:** 85 | 86 | ```python 87 | from pathlib import Path 88 | 89 | filename = "my_file.txt" 90 | path_object = Path(filename) 91 | absolute_path = path_object.resolve() 92 | print(absolute_path) 93 | ``` 94 | 95 | * **Getting the current working directory:** 96 | 97 | ```python 98 | from pathlib import Path 99 | 100 | current_directory = Path.cwd() 101 | print(current_directory) 102 | ``` 103 | 104 | * **Joining paths:** 105 | 106 | ```python 107 | from pathlib import Path 108 | 109 | directory = Path("/home/user/documents") 110 | filename = "report.pdf" 111 | full_path = directory / filename # Uses the / operator for joining 112 | print(full_path) # Output: /home/user/documents/report.pdf 113 | ``` 114 | 115 | * **Getting the directory name:** 116 | 117 | ```python 118 | from pathlib import Path 119 | 120 | filepath = Path("/path/to/my_file.txt") 121 | directory = filepath.parent 122 | print(directory) # Output: /path/to 123 | ``` 124 | 125 | * **Getting the filename from a path:** 126 | 127 | ```python 128 | from pathlib import Path 129 | 130 | filepath = Path("/path/to/my_file.txt") 131 | filename = filepath.name 132 | print(filename) # Output: my_file.txt 133 | ``` 134 | 135 | * **Checking if a path exists:** 136 | 137 | ```python 138 | from pathlib import Path 139 | 140 | filepath = Path("/path/to/my_file.txt") 141 | if filepath.exists(): 142 | print("File exists") 143 | else: 144 | print("File does not exist") 145 | ``` 146 | 147 | **3. Getting the path from a file object (e.g., after opening a file)** 148 | 149 | ```p 150 | ython 151 | with open("my_file.txt", "r") as f: 152 | import os 153 | filepath = os.path.abspath(f.name) # Using os.path 154 | print(filepath) 155 | 156 | from pathlib import Path 157 | with open("my_file.txt", "r") as f: 158 | filepath = Path(f.name).resolve() # Using pathlib 159 | print(filepath) 160 | ``` 161 | 162 | **Which method should you use?** 163 | 164 | * **`pathlib` is generally preferred** for new code. It's more object-oriented, easier to read, and provides a more consistent API. The `/` operator for joining paths is particularly nice. 165 | * **`os.path` is still perfectly valid** and you'll encounter it in a lot of existing code. It's a good choice if you're working with older codebases or need maximum compatibility. 166 | 167 | **Important Considerations:** 168 | 169 | * **Relative vs. Absolute Paths:** 170 | * **Relative paths** are relative to the current working directory. For example, `"my_file.txt"` means "a file named `my_file.txt` in the current directory." 171 | * **Absolute paths** specify the complete path from the root directory. For example, `"/home/user/documents/my_file.txt"` (on Linux/macOS) or `"C:\\Users\\User\\Documents\\my_file.txt"` (on Windows). 172 | * **Operating System Differences:** Path separators are different on different operating systems: 173 | * Linux/macOS: `/` 174 | * Windows: `\` 175 | * `os.path.join()` and `pathlib` handle these differences automatically, so you don't usually need to worry about them. 176 | * **Error Handling:** If you're dealing with user-provided paths, it's a good idea to include error handling (e.g., `try...except` blocks) to catch potential `FileNotFoundError` or other exceptions. 177 | 178 | Choose the method that best suits your needs and coding style. `pathlib` is the recommended approach for most new projects. Remember to consider whether you need relative or absolute paths and to handle potential errors. 179 | 180 | > q q  181 | 182 | > Aborted! 183 | -------------------------------------------------------------------------------- /tests/blankie.md: -------------------------------------------------------------------------------- 1 | maybe? 2 | ```bash 3 | ``` 4 | 5 | here's something 6 | 7 | -------------------------------------------------------------------------------- /tests/block.md: -------------------------------------------------------------------------------- 1 | So here is some text 2 | 3 | > and technically we are in blockquote 4 | > territory with these. Blockquote is one 5 | > > of the few things that can be embedded 6 | > > * what about 7 | > > * a list inside 8 | > > * a blockquote? 9 | > > in markdown. Stylistically they are 10 | > > different than lists, but 11 | >> not by much 12 | > > # This should be respected 13 | > | Header 1 | Header 2 | Header 3 | 14 | > | -------- | -------- | -------- | 15 | > | Cell 1 | Cell 2 | Cell 3 | 16 | > | Cell 4 | Cell 5 | Cell 6 | 17 | >>> This should be 3 18 | >> > and this 19 | > >> and this as equivalent 20 | 21 | Now we are out of the blockquote 22 | -------------------------------------------------------------------------------- /tests/bold_reset_with_link.md: -------------------------------------------------------------------------------- 1 | * **Model Card:** When this wraps with this link ([https://huggingface.co/microsoft/bitnet-b1.58-2B-4T](https://huggingface.co/microsoft/bitnet-b1.58-2B-4T)) the next line used to be bold because of some stripping error. This should no longer be the case 2 | -------------------------------------------------------------------------------- /tests/broken-code.md: -------------------------------------------------------------------------------- 1 | 2 | ### JavaScript 3 | ```html 4 | 5 | 6 | 7 | Mandelbrot Set 8 | 11 | 12 | 13 | 14 | 50 | 51 | 52 | ``` 53 | -------------------------------------------------------------------------------- /tests/broken-example.md: -------------------------------------------------------------------------------- 1 | # Broken code 2 | 3 | * This code will not necessarily safely copy and paste 4 | If you look below 5 | ```shell 6 | We are using llama-4 style 7 | ``` 8 | * llama-4 maverick is kinda weird. This is not a formal testing system but is more "showcase based" testing, driving features and looking at output. 9 | * What I mean is it will be able to bring things in 10 | And we can get weird code blocks 11 | ```shell 12 | like this 13 | ``` 14 | 15 | -------------------------------------------------------------------------------- /tests/chunk-buffer.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | TIMEOUT=${TIMEOUT:-0.05} 3 | 4 | while [[ $# -gt 0 ]]; do 5 | echo "## File: $1" 6 | echo 7 | echo -e "----\n\n" 8 | cat $1 | awk -v RS='🫣' '{ printf "%s", $0; system("sleep '$TIMEOUT'") }' 9 | shift 10 | done 11 | -------------------------------------------------------------------------------- /tests/cjk-table.md: -------------------------------------------------------------------------------- 1 | | 特性 | Ruby | Python | OCaml | 2 | |------------------|--------------------------------|--------------------------------|-------------------------------| 3 | | **设计哲学** | 使编程更加自然和有趣 | 强调代码的可读性和简洁性 | 提供高效的函数式编程功能 | 4 | | **应用领域** | Web 开发、自动化脚本、原型开发 | Web 开发、数据科学、机器学习、自动化脚本 | 编译器开发、系统编程、金融建模 | 5 | | **特点** | 动态类型、面向对象、元编程能力强 | 动态类型、面向对象、大量第三方库支持 | 静态类型、函数式编程、模块化、高性能 | 6 | | **语法示例** | ```ruby
        class Greeter
        def initialize(name)
        @name = name
        end

        def greet
        puts "Hello, #{@name}!"
        end
        end

        greeter = Greeter.new("World")
        greeter.greet``` | ```python
        class Greeter:
        def __init__(self, name):
        self.name = name

        def greet(self):
        print(f"Hello, {self.name}!")

        greeter = Greeter("World")
        greeter.greet()``` | ```ocaml
        type greeter = {
        name: string;
        }

        let greet {name} =
        print_endline ("Hello, " ^ name ^ "!")

        let greeter = {name = "World"}
        let () = greet greeter``` | 7 | | **性能** | 相对较低,但通过 JRuby、TruffleRuby 可以提高性能 | 适中,通过 Cython、PyPy 可以显著提高性能 | 高,编译后的代码接近 C 语言的性能 | 8 | | **优点** | 开发速度快,社区活跃 | 丰富的第三方库,应用广泛 | 适合性能关键的应用,编译器优化能力强 | 9 | | **生态系统** | Ruby on Rails 是最著名的 Web 框架,其他库和工具也很丰富 | 丰富的第三方库,如 NumPy、Pandas、Django、Flask | 相对较少,但有高质量的库,如 Core 和 Async | 10 | | **学习曲线** | 较平缓,语法灵活,适合初学者 | 平坦,语法清晰,适合初学者 | 较陡峭,需要理解函数式编程概念 | 11 | | **文档** | 丰富,社区支持好 | 丰富,有大量的教程和资源 | 适中,但社区提供的资源质量高 | 12 | | **适用场景** | 快速开发、Web 应用、自动化脚本 | 数据科学、机器学习、Web 开发、自动化脚本 | 性能关键的应用、编译器开发、金融建模 | 13 | 14 | -------------------------------------------------------------------------------- /tests/cjk-wrap.md: -------------------------------------------------------------------------------- 1 | Chatting with openrouter/qwen/qwen-2.5-72b-instruct:free 2 | Type 'exit' or 'quit' to exit 3 | Type '!multi' to enter multiple lines, then '!end' to finish 4 | > 当然!以下是Ruby、Python和OCaml的比较,用中文进行描述: 5 | 6 | ### 1. 语言概述 7 | 8 | **Ruby**: 9 | - **设计哲学**: Ruby 是由 Yukihiro "Matz" Matsumoto 在 1995 年设计的,旨在使编程更加自然和有趣。 10 | - **应用领域**: Web 开发(如 Ruby on Rails)、自动化脚本、原型开发等。 11 | - **特点**: 动态类型、面向对象、元编程能力强、语法灵活。 12 | 13 | **Python**: 14 | - **设计哲学**: Python 由 Guido van Rossum 在 1991 年创建,强调代码的可读性和简洁性。 15 | - **应用领域**: Web 开发、数据科学、机器学习、自动化脚本等。 16 | - **特点**: 动态类型、面向对象、大量第三方库支持。 17 | 18 | **OCaml**: 19 | - **设计哲学**: OCaml 由 INRIA 的 Xavier Leroy 等人在 1996 年开发,旨在提供高效的函数式编程功能。 20 | - **应用领域**: 编译器开发、系统编程、金融建模等。 21 | - **特点**: 静态类型、函数式编程、模块化、高性能。 22 | 23 | ### 2. 语法和编程风格 24 | 25 | **Ruby**: 26 | - **语法**: 灵活多变,支持多重编程范式(面向对象、函数式、元编程)。 27 | - **例子**: 28 | ```ruby 29 | class Greeter 30 | def initialize(name) 31 | @name = name 32 | end 33 | 34 | def greet 35 | puts "Hello, #{@name}!" 36 | end 37 | end 38 | 39 | greeter = Greeter.new("World") 40 | greeter.greet 41 | ``` 42 | 43 | **Python**: 44 | - **语法**: 清晰简洁,强制缩进,易于阅读。 45 | - **例子**: 46 | ```python 47 | class Greeter: 48 | def __init__(self, name): 49 | self.name = name 50 | 51 | def greet(self): 52 | print(f"Hello, {self.name}!") 53 | 54 | greeter = Greeter("World") 55 | greeter.greet() 56 | ``` 57 | 58 | **OCaml**: 59 | - **语法**: 严格且功能性强,支持模式匹配和代数数据类型。 60 | - **例子**: 61 | ```ocaml 62 | type greeter = { 63 | name: string; 64 | } 65 | 66 | let greet {name} = 67 | print_endline ("Hello, " ^ name ^ "!") 68 | 69 | let greeter = {name = "World"} 70 | let () = greet greeter 71 | ``` 72 | 73 | ### 3. 性能 74 | 75 | **Ruby**: 76 | - **性能**: 相对较低,但通过 JRuby、TruffleRuby 等实现可以提高性能。 77 | - **优点**: 开发速度快,社区活跃。 78 | 79 | **Python**: 80 | - **性能**: 适中,通过 Cython、PyPy 等可以显著提高性能。 81 | - **优点**: 丰富的第三方库,应用广泛。 82 | 83 | **OCaml**: 84 | - **性能**: 高,编译后的代码接近 C 语言的性能。 85 | - **优点**: 适合性能关键的应用,编译器优化能力强。 86 | 87 | ### 4. 生态系统 88 | 89 | **Ruby**: 90 | - **库和框架**: Ruby on Rails 是最著名的 Web 框架,其他库和工具也很丰富。 91 | - **社区**: 活跃,对新手友好。 92 | 93 | **Python**: 94 | - **库和框架**: 丰富的第三方库,如 NumPy、Pandas、Django、Flask 等。 95 | - **社区**: 非常活跃,广泛应用于多个领域。 96 | 97 | **OCaml**: 98 | - **库和框架**: 相对较少,但有高质量的库,如 Core 和 Async。 99 | - **社区**: 较小但专业,注重性能和正确性。 100 | 101 | ### 5. 学习曲线 102 | 103 | **Ruby**: 104 | - **学习曲线**: 较平缓,语法灵活,适合初学者。 105 | - **文档**: 丰富,社区支持好。 106 | 107 | **Python**: 108 | - **学习曲线**: 平坦,语法清晰,适合初学者。 109 | - **文档**: 丰富,有大量的教程和资源。 110 | 111 | **OCaml**: 112 | - **学习曲线**: 较陡峭,需要理解函数式编程概念。 113 | - **文档**: 适中,但社区提供的资源质量高。 114 | 115 | ### 6. 适用场景 116 | 117 | **Ruby**: 118 | - **适合**: 快速开发、Web 应用、自动化脚本。 119 | - **不适合**: 性能要求极高的应用。 120 | 121 | **Python**: 122 | - **适合**: 数据科学、机器学习、Web 开发、自动化脚本。 123 | - **不适合**: 需要极高性能的系统编程。 124 | 125 | **OCaml**: 126 | - **适合**: 性能关键的应用、编译器开发、金融建模。 127 | - **不适合**: 初学者或需要快速开发的项目。 128 | 129 | ### 总结 130 | 131 | - **Ruby**: 适合快速开发和 Web 应用,社区活跃,语法灵活。 132 | - **Python**: 适合数据科学和机器学习,生态系统丰富,语法清晰。 133 | - **OCaml**: 适合性能关键的应用,编译器优化能力强,适合专业开发者。 134 | 135 | 希望这些信息对你有帮助!如果有其他问题,欢迎随时提问。 136 | > 当然!以下是将 Ruby、Python 和 OCaml 的比较总结成表格的形式: 137 | 138 | | 特性 | Ruby | Python | OCaml | 139 | |------------------|--------------------------------|--------------------------------|-------------------------------| 140 | | **设计哲学** | 使编程更加自然和有趣 | 强调代码的可读性和简洁性 | 提供高效的函数式编程功能 | 141 | | **应用领域** | Web 开发、自动化脚本、原型开发 | Web 开发、数据科学、机器学习、自动化脚本 | 编译器开发、系统编程、金融建模 | 142 | | **特点** | 动态类型、面向对象、元编程能力强 | 动态类型、面向对象、大量第三方库支持 | 静态类型、函数式编程、模块化、高性能 | 143 | | **语法示例** | ```ruby
        class Greeter
        def initialize(name)
        @name = name
        end

        def greet
        puts "Hello, #{@name}!"
        end
        end

        greeter = Greeter.new("World")
        greeter.greet``` | ```python
        class Greeter:
        def __init__(self, name):
        self.name = name

        def greet(self):
        print(f"Hello, {self.name}!")

        greeter = Greeter("World")
        greeter.greet()``` | ```ocaml
        type greeter = {
        name: string;
        }

        let greet {name} =
        print_endline ("Hello, " ^ name ^ "!")

        let greeter = {name = "World"}
        let () = greet greeter``` | 144 | | **性能** | 相对较低,但通过 JRuby、TruffleRuby 可以提高性能 | 适中,通过 Cython、PyPy 可以显著提高性能 | 高,编译后的代码接近 C 语言的性能 | 145 | | **优点** | 开发速度快,社区活跃 | 丰富的第三方库,应用广泛 | 适合性能关键的应用,编译器优化能力强 | 146 | | **生态系统** | Ruby on Rails 是最著名的 Web 框架,其他库和工具也很丰富 | 丰富的第三方库,如 NumPy、Pandas、Django、Flask | 相对较少,但有高质量的库,如 Core 和 Async | 147 | | **学习曲线** | 较平缓,语法灵活,适合初学者 | 平坦,语法清晰,适合初学者 | 较陡峭,需要理解函数式编程概念 | 148 | | **文档** | 丰富,社区支持好 | 丰富,有大量的教程和资源 | 适中,但社区提供的资源质量高 | 149 | | **适用场景** | 快速开发、Web 应用、自动化脚本 | 数据科学、机器学习、Web 开发、自动化脚本 | 性能关键的应用、编译器开发、金融建模 | 150 | 151 | -------------------------------------------------------------------------------- /tests/code.md: -------------------------------------------------------------------------------- 1 | *italics* 2 | some_name_ 3 | `_somevariable_` 4 | 5 | ---- 6 | 7 | ```python 8 | def fizzbuzz(n): 9 | for i in range(1, n + 1): 10 | if i % 3 == 0 and i % 5 == 0: 11 | print("FizzBuzz") 12 | elif i % 3 == 0: 13 | print("Fizz") 14 | elif i % 5 == 0: 15 | print("Buzz") 16 | else: 17 | print(i) 18 | 19 | 20 | # Example usage: Print FizzBuzz up to 100 21 | fizzbuzz(100) 22 | 23 | # Example usage: different range: 24 | fizzbuzz(20) 25 | 26 | #Example usage: one line output (list comprehension) 27 | def fizzbuzz_oneline(n): 28 | print(["FizzBuzz" if i%3==0 and i%5==0 else "Fizz" if i%3==0 else "Buzz" if i%5==0 else i for i in range(1,n+1)]) 29 | 30 | fizzbuzz_oneline(30) 31 | ``` 32 | 33 | -------------------------------------------------------------------------------- /tests/dimcheck.md: -------------------------------------------------------------------------------- 1 | 2 | ### **3. Force a Full Restart 💥** 3 | Sometimes a reload isn’t enough. Stop and restart the service: 4 | ```bash 5 | sudo systemctl stop postgresql 6 | sudo systemctl start postgresql 7 | ``` 8 | -------------------------------------------------------------------------------- /tests/example.md: -------------------------------------------------------------------------------- 1 | # Heading 1 2 | ## Heading 2 3 | ### Heading 3 4 | #### Heading 4 5 | ##### Heading 5 6 | ###### Heading 6 7 | 8 | * Lorem ipsum dolor sit amet, consectetur adipiscing elit. Sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia `deserunt mollit anim` id est laborum. 9 | * Unordered List Item 2 10 | * here is some depth 11 | * let's see if it works 12 | * we can go deeper 13 | * at multiple levels 14 | * and then pull back 15 | 16 | 1. Lorem ipsum dolor sit amet, consectetur adipiscing elit. Sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum. 17 | 1. Lorem ipsum dolor sit amet, consectetur adipiscing elit. Sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum. 18 | 1. here is some depth 19 | 1. let's see if it works 20 | 1. we can go deeper 21 | 1. at multiple levels 22 | 1. and then pull back 23 | 1. we can go deeper 24 | 1. at multiple levels 25 | 1. and then pull back 26 | 1. Ordered List Item 2 27 | 1. Ordered List Item 2 28 | 29 | This is a **Bold Text** example 30 | This is an *Italic Text* example 31 | This is an _Underlined Text_ example 32 | ```python 33 | def my_function(): 34 | print("Hello, world!") 35 | ``` 36 | 37 | | Header 1 | Header 2 | Header 3 | 38 | | -------- | -------- | -------- | 39 | | Cell 1 | Cell 2 | Cell 3 | 40 | | Cell 4 | Cell 5 | Cell 6 | 41 | 42 | More text after the table. 43 | 44 | --- 45 | 46 | Here's another paragraph 47 | 48 | 49 | 50 | 51 | 52 | ```python 53 | def indented_function(): 54 | print("Indented code") 55 | ``` 56 | -------------------------------------------------------------------------------- /tests/fizzbuzz.md: -------------------------------------------------------------------------------- 1 | Okay, here's the FizzBuzz problem implemented in Python, JavaScript, and C++, along with a description of the problem itself. 2 | 3 | **The FizzBuzz Problem Description** 4 | 5 | FizzBuzz is a classic programming interview question, often used to test basic coding skills and understanding of loops and conditional statements. The task is as follows: 6 | 7 | 1. **Iteration:** Write a program that iterates (loops) through a range of numbers (typically 1 to 100, but the range can be specified). 8 | 9 | 2. **Conditional Checks:** For each number in the range, apply these rules: 10 | * If the number is divisible by 3, print "Fizz". 11 | * If the number is divisible by 5, print "Buzz". 12 | * If the number is divisible by both 3 and 5, print "FizzBuzz". 13 | * If the number is not divisible by 3 or 5, print the number itself. 14 | 15 | 3. **Output:** The output should be a sequence of "Fizz", "Buzz", "FizzBuzz", and numbers, printed one per line (or in a continuous sequence, depending on the specific instructions). 16 | 17 | This should be reset 18 | 19 | The core concepts tested are: 20 | 21 | 1. **Loops:** Using a `for` loop (or similar construct) to iterate through a sequence of numbers. 22 | 1. **Conditional Statements:** Using `if`, `else if`, and `else` statements to check for divisibility and determine the correct output. 23 | 1. **Modulo Operator:** The key to checking for divisibility is the modulo operator (`%`). `a % b` gives the remainder when `a` is divided by `b`. If the remainder is 0, then `a` is divisible by `b`. 24 | 1. **Basic Output:** Printing to the console (or a file, or whatever output mechanism is specified). 25 | 1. **Order of operations** The order to check the conditions. 26 | 27 | **Implementations** 28 | 29 | Here are the implementations in Python, JavaScript, and C++: 30 | 31 | **1. Python** 32 | 33 | ```python 34 | def fizzbuzz(n): 35 | for i in range(1, n + 1): 36 | if i % 3 == 0 and i % 5 == 0: 37 | print("FizzBuzz") 38 | elif i % 3 == 0: 39 | print("Fizz") 40 | elif i % 5 == 0: 41 | print("Buzz") 42 | else: 43 | print(i) 44 | 45 | # 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 46 | fizzbuzz(100) 47 | 48 | # Example usage: different range: 49 | fizzbuzz(20) 50 | 51 | #Example usage: one line output (list comprehension) 52 | def fizzbuzz_oneline(n): 53 | print(["FizzBuzz" if i%3==0 and i%5==0 else "Fizz" if i%3==0 else "Buzz" if i%5==0 else i for i in range(1,n+1)]) 54 | 55 | fizzbuzz_oneline(30) 56 | ``` 57 | 58 | **Explanation (Python):** 59 | 60 | * **`def fizzbuzz(n):`**: Defines a function named `fizzbuzz` that takes an integer `n` as input (the upper limit of the range). 61 | * **`for i in range(1, n + 1):`**: A `for` loop that iterates through numbers from 1 to `n` (inclusive). `range(1, n + 1)` generates a sequence of numbers starting from 1 and ending at `n`. 62 | * **`if i % 3 == 0 and i % 5 == 0:`**: Checks if `i` is divisible by both 3 and 5. The `and` operator ensures both conditions must be true. 63 | * **`elif i % 3 == 0:`**: If the first condition is false, this checks if `i` is divisible by 3. 64 | * **`elif i % 5 == 0:`**: If the previous conditions are false, this checks if `i` is divisible by 5. 65 | * **`else:`**: If none of the above conditions are true, this prints the number `i`. 66 | * **`print(...)`**: Prints the appropriate output (string or number). 67 | * **`fizzbuzz(100)`**: Calls the `fizzbuzz` function with `n = 100` to execute the code. 68 | * The one line usage is using list comprehension, an advanced Python feature. 69 | 70 | **2. JavaScript** 71 | 72 | ```javascript 73 | function fizzBuzz(n) { 74 | for (let i = 1; i <= n; i++) { 75 | if (i % 3 === 0 && i % 5 === 0) { 76 | console.log("FizzBuzz"); 77 | } else if (i % 3 === 0) { 78 | console.log("Fizz"); 79 | } else if (i % 5 === 0) { 80 | console.log("Buzz"); 81 | } else { 82 | console.log(i); 83 | } 84 | } 85 | } 86 | 87 | // Example usage: 88 | fizzBuzz(100); 89 | 90 | // Example usage: different range 91 | fizzBuzz(25); 92 | 93 | // Example one-line output. (arrow function & ternary operator) 94 | const fizzBuzzOneLine = n => { 95 | for (let i = 1; i <= n; i++) { 96 | console.log((i % 3 === 0 ? (i % 5 === 0 ? "FizzBuzz" : "Fizz") : (i % 5 === 0 ? "Buzz" : i))); 97 | } 98 | }; 99 | fizzBuzzOneLine(30); 100 | ``` 101 | 102 | **Explanation (JavaScript):** 103 | 104 | * **`function fizzBuzz(n) { ... }`**: Defines a function named `fizzBuzz` that takes an integer `n` as input. 105 | * **`for (let i = 1; i <= n; i++) { ... }`**: A `for` loop that iterates from 1 to `n` (inclusive). `let` declares a variable `i` scoped to the loop. 106 | * **`if (i % 3 === 0 && i % 5 === 0)`**: Checks for divisibility by both 3 and 5. `===` is the strict equality operator (checks for both value and type equality). 107 | * **`else if ... else ...`**: Similar conditional structure to the Python example. 108 | * **`console.log(...)`**: Prints the output to the browser's console. 109 | * **`fizzBuzz(100);`**: Calls the function to execute it. 110 | * The one line usage is using arrow function and ternary operator, advanced JavaScript features. 111 | 112 | **3. C++** 113 | 114 | ```c++ 115 | #include 116 | 117 | void fizzBuzz(int n) { 118 | for (int i = 1; i <= n; i++) { 119 | if (i % 3 == 0 && i % 5 == 0) { 120 | std::cout << "FizzBuzz" << std::endl; 121 | } else if (i % 3 == 0) { 122 | std::cout << "Fizz" << std::endl; 123 | } else if (i % 5 == 0) { 124 | std::cout << "Buzz" << std::endl; 125 | } else { 126 | std::cout << i << std::endl; 127 | } 128 | } 129 | } 130 | 131 | int main() { 132 | fizzBuzz(100); 133 | 134 | // Example usage: different range 135 | fizzBuzz(35); 136 | return 0; 137 | } 138 | ``` 139 | 140 | **Explanation (C++):** 141 | 142 | * **`#include `**: Includes the `iostream` library, which provides input/output functionality (like `std::cout`). 143 | * **`void fizzBuzz(int n) { ... }`**: Defines a function named `fizzBuzz` that takes an integer `n` and returns nothing (`void`). 144 | * **`for (int i = 1; i <= n; i++) { ... }`**: A `for` loop similar to the JavaScript example. `int i = 1` declares an integer variable `i` and initializes it to 1. 145 | * **`if ... else if ... else ...`**: The same conditional structure as the other examples. 146 | * **`std::cout << ... << std::endl;`**: Prints the output to the console. `std::cout` is the standard output stream. `<<` is the insertion operator (used to insert data into the output stream). `std::endl` inserts a newline character (moves the cursor to the next line). 147 | * **`int main() { ... }`**: The `main` function is the entry point of the C++ program. 148 | * **`fizzBuzz(100);`**: Calls the `fizzBuzz` function. 149 | * **`return 0;`**: Indicates that the program executed successfully (a standard convention in C++). 150 | 151 | **Key Differences and Similarities Across Languages** 152 | 153 | * **Syntax:** The most obvious difference is the syntax. Python uses indentation to define code blocks, while JavaScript and C++ use curly braces `{}`. C++ also requires semicolons `;` at the end of most statements. 154 | * **Variable Declaration:** Python doesn't require explicit variable type declarations. JavaScript uses `let` (or `var`, or `const`). C++ requires you to specify the type (e.g., `int`). 155 | * **Output:** Python uses `print()`, JavaScript uses `console.log()`, and C++ uses `std::cout`. 156 | * **Loop Structure:** The `for` loop structure is very similar across all three languages, although the details of how the loop variable is declared and incremented vary slightly. 157 | * **Conditional Statements:** The `if`, `else if`, `else` structure is consistent across the languages. 158 | * **Modulo operator:** They all share the same modulo operator `%`. 159 | * **Logical AND**: They all share the similar logical `and` or `&&`. 160 | 161 | These FizzBuzz examples demonstrate the fundamental constructs of programming, making it a useful introductory exercise and a good way to compare the basic syntax of different languages. The logic, however, remains the same in all cases. 162 | 163 | -------------------------------------------------------------------------------- /tests/inline.md: -------------------------------------------------------------------------------- 1 | 2 | So _this should be underlined_ but `_this_should_not_`. 3 | -------------------------------------------------------------------------------- /tests/line-buffer.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | TIMEOUT=${TIMEOUT:-0.1} 3 | while [[ $# -gt 0 ]]; do 4 | echo "## File: $1" 5 | echo "----" 6 | for i in $(seq 1 $(cat $1 | wc -l)); do 7 | head -$i $1 | tail -1 8 | sleep $TIMEOUT 9 | done 10 | shift 11 | done 12 | 13 | -------------------------------------------------------------------------------- /tests/line-wrap.md: -------------------------------------------------------------------------------- 1 | We want to make sure line-wrap inline-formatting works for the wraptext. 2 | 3 | First we start with some text and then `we try to break out into here and then end` and continue with our text. 4 | 5 | Now we will do one that is a bit harder `we do this to_force_a_line_break_and_this_to go on to the next line` and then we continue with our text. 6 | 7 | Now we will do one that is much harder `we do this to_force_a_line_break_and_have_more_text` and end it at the point that's wrapped before we continue with our text. 8 | 9 | Now we will do one that is much harder **we do this to_force_a_line_break_and_have_more_text** and end it at the point that's wrapped before we continue with our text. 10 | 11 | This is designed for a width around 70 12 | -------------------------------------------------------------------------------- /tests/links.md: -------------------------------------------------------------------------------- 1 | Here are five top websites for staying informed about the latest developments in AI: 2 | 3 | 1. [MIT Technology Review's AI Section](https://www.technologyreview.com/topic/artificial-intelligence/): Offers in-depth articles, news, and analysis on AI breakthroughs and their societal implications. 4 | 5 | 2. [ArXiv's Artificial Intelligence Category](https://arxiv.org/list/cs.AI/recent): Provides access to pre-print research papers, allowing you to see cutting-edge AI research before formal publication. 6 | 7 | 3. [Distill](https://distill.pub/): Publishes interactive, visually rich articles explaining complex machine learning concepts in an accessible way. 8 | 9 | 4. [OpenAI's Blog](https://openai.com/blog): Showcases the research, projects, and perspectives from one of the leading AI research organizations. 10 | 11 | 5. [Two Minute Papers](https://www.youtube.com/@TwoMinutePapers): Delivers a fun and educational YouTube channel that digest the latest AI publications in short easy to understand videos. 12 | 13 | -------------------------------------------------------------------------------- /tests/managerie.md: -------------------------------------------------------------------------------- 1 | Okay, let's go over Markdown syntax. I'll cover the basics, some frequently used elements, and a few less common ones. 2 | 3 | **What is Markdown?** 4 | 5 | Markdown is a lightweight markup language with a simple syntax. It's designed to be easy to read and write in its raw form, while also being easily converted to HTML (or other formats). Think of it as a way to format text without needing to know complex HTML tags. 6 | 7 | **Basic Syntax** 8 | 9 | Here's a breakdown of the common Markdown elements: 10 | 11 | **1. Headings:** 12 | 13 | * You create headings using the `#` symbol. The number of `#` symbols determines the heading level (H1 to H6). 14 | 15 | ```markdown 16 | # Heading 1 17 | ## Heading 2 18 | ### Heading 3 19 | #### Heading 4 20 | ##### Heading 5 21 | ###### Heading 6 22 | ``` 23 | 24 | * Alternative heading syntax (for H1 and H2 only): 25 | 26 | ```markdown 27 | Heading 1 28 | ========= 29 | 30 | Heading 2 31 | --------- 32 | ``` 33 | 34 | **Rendered Output (example - the *actual* rendering might vary slightly depending on the viewer):** 35 | 36 | # Heading 1 37 | ## Heading 2 38 | ### Heading 3 39 | #### Heading 4 40 | ##### Heading 5 41 | ###### Heading 6 42 | 43 | Heading 1 44 | ========= 45 | 46 | Heading 2 47 | --------- 48 | 49 | **2. Text Formatting:** 50 | 51 | * **Bold:** Use double asterisks `**` or double underscores `__`. 52 | 53 | ```markdown 54 | **This is bold text.** 55 | __This is also bold text.__ 56 | ``` 57 | 58 | * **Italic:** Use single asterisks `*` or single underscores `_`. 59 | 60 | ```markdown 61 | *This is italic text.* 62 | _This is also italic text._ 63 | ``` 64 | 65 | * **Bold and Italic:** Combine them. 66 | 67 | ```markdown 68 | ***This is bold and italic text.*** 69 | ___This is also bold and italic text.___ 70 | **_This is another way to do it._** 71 | ``` 72 | 73 | * **Strikethrough:** Use double tildes `~~`. 74 | 75 | ```markdown 76 | ~~This text is crossed out.~~ 77 | ``` 78 | 79 | * **Inline code:** Use backticks `` ` ``. 80 | 81 | ```markdown 82 | Use the `printf()` function to print to the console. 83 | ``` 84 | 85 | **Rendered Output:** 86 | 87 | **This is bold text.** 88 | __This is also bold text.__ 89 | 90 | *This is italic text.* 91 | _This is also italic text._ 92 | 93 | ***This is bold and italic text.*** 94 | ___This is also bold and italic text.___ 95 | **_This is another way to do it._** 96 | 97 | ~~This text is crossed out.~~ 98 | 99 | Use the `printf()` function to print to the console. 100 | 101 | **3. Lists:** 102 | 103 | * **Unordered Lists:** Use asterisks `*`, plus signs `+`, or hyphens `-`. 104 | 105 | ```markdown 106 | * Item 1 107 | * Item 2 108 | * Item 3 109 | 110 | + Item A 111 | + Item B 112 | 113 | - Item X 114 | - Item Y 115 | ``` 116 | 117 | * **Ordered Lists:** Use numbers followed by a period `.`. 118 | 119 | ```markdown 120 | 1. First item 121 | 2. Second item 122 | 3. Third item 123 | ``` 124 | 125 | * **Nested Lists:** Use indentation (usually 2 or 4 spaces) to create nested lists. 126 | 127 | ```markdown 128 | * Item 1 129 | * Sub-item 1 130 | * Sub-item 2 131 | * Item 2 132 | 1. Nested item A 133 | 2. Nested item B 134 | ``` 135 | 136 | **Rendered Output:** 137 | 138 | * Item 1 139 | * Item 2 140 | * Item 3 141 | 142 | + Item A 143 | + Item B 144 | 145 | - Item X 146 | - Item Y 147 | 148 | 1. First item 149 | 2. Second item 150 | 3. Third item 151 | 152 | * Item 1 153 | * Sub-item 1 154 | * Sub-item 2 155 | * Item 2 156 | 1. Nested item A 157 | 2. Nested item B 158 | 159 | **4. Links:** 160 | 161 | * **Inline Links:** 162 | 163 | ```markdown 164 | [Link text](URL) 165 | [Link to Google](https://www.google.com) 166 | ``` 167 | 168 | * **Links with Title (hover text):** 169 | 170 | ```markdown 171 | [Link text](URL "Title") 172 | [Link to Google](https://www.google.com "Google's Homepage") 173 | ``` 174 | 175 | * **Reference Links:** Define the URL separately. 176 | 177 | ```markdown 178 | [Link text][link_id] 179 | 180 | [link_id]: URL "Title" 181 | 182 | [Example Link][example] 183 | 184 | [example]: https://www.example.com "Example Website" 185 | ``` 186 | 187 | **Rendered Output:** 188 | 189 | [Link to Google](https://www.google.com "Google's Homepage") 190 | [Example Link][example] 191 | 192 | [example]: https://www.example.com "Example Website" 193 | 194 | **5. Images:** 195 | 196 | * Similar to links, but with a `!` at the beginning. 197 | 198 | ```markdown 199 | ![Alt text](image_URL "Optional title") 200 | ![My Logo](https://example.com/logo.png "Example Logo") 201 | ``` 202 | 203 | **Rendered Output:** 204 | 205 | An image will be displayed (if the URL is valid). 206 | 207 | **6. Blockquotes:** 208 | 209 | * Use the `>` symbol. 210 | 211 | ```markdown 212 | > This is a blockquote. 213 | > It can span multiple lines. 214 | > 215 | > You can also nest blockquotes. 216 | > > This is a nested blockquote. 217 | ``` 218 | 219 | **Rendered Output:** 220 | 221 | > This is a blockquote. 222 | > It can span multiple lines. 223 | > 224 | > You can also nest blockquotes. 225 | > > This is a nested blockquote. 226 | 227 | **7. Horizontal Rule:** 228 | 229 | * Use three or more hyphens `---`, asterisks `***`, or underscores `___`. 230 | 231 | ```markdown 232 | --- 233 | *** 234 | ___ 235 | ``` 236 | 237 | **Rendered Output:** 238 | 239 | A horizontal line will be displayed. 240 | 241 | **8. Code Blocks:** 242 | 243 | * **Indented Code Blocks:** Indent each line of the code block by four spaces or one tab. Be consistent with indenting and any surrounding text. 244 | 245 | ```markdown 246 | This is a code block. 247 | It will be displayed in a monospaced font. 248 | ``` 249 | 250 | * **Fenced Code Blocks:** Use triple backticks `` ``` `` to enclose the code block. You can optionally specify the language for syntax highlighting. 251 | 252 | ```markdown 253 | ```python 254 | def hello_world(): 255 | print("Hello, world!") 256 | ``` 257 | ``` 258 | 259 | **Rendered Output:** 260 | 261 | ```python 262 | def hello_world(): 263 | print("Hello, world!") 264 | ``` 265 | 266 | **9. Escaping Characters:** 267 | 268 | * If you need to use a character that Markdown uses for syntax (e.g., `#`, `*`, `_`, `>`, etc.), you can escape it with a backslash `\`. 269 | 270 | ```markdown 271 | \*This is not italic.\* 272 | \# This is not a heading. 273 | ``` 274 | 275 | **Rendered Output:** 276 | 277 | \*This is not italic.\* 278 | \# This is not a heading. 279 | 280 | **Less Common Markdown Features (often extended by specific implementations):** 281 | 282 | * **Tables:** Using pipes `|` and hyphens `-`. Syntax is more involved: 283 | 284 | ```markdown 285 | | Header 1 | Header 2 | 286 | | -------- | -------- | 287 | | Cell 1 | Cell 2 | 288 | | Cell 3 | Cell 4 | 289 | ``` 290 | 291 | * **Definition Lists:** HTML `
        ` element. 292 | 293 | ```markdown 294 | Term 295 | : Definition of the term. 296 | 297 | Another Term 298 | : Another definition. 299 | ``` 300 | 301 | * **Footnotes:** Create footnotes using `[^footnote_id]` and define them at the bottom. 302 | 303 | This is some text with a footnote.[^1] 304 | 305 | [^1]: This is the footnote text. 306 | 307 | * **Task Lists (Checkboxes):** Often supported in specific Markdown implementations (e.g., GitHub Flavored Markdown). 308 | 309 | ```markdown 310 | - [ ] An incomplete task 311 | - [x] A completed task 312 | ``` 313 | 314 | **Important Considerations:** 315 | 316 | * **Flavors of Markdown:** There are different "flavors" of Markdown (e.g., CommonMark, GitHub Flavored Markdown (GFM), MultiMarkdown). These flavors may have slight variations in syntax or support additional features. Check the documentation for the specific Markdown parser/renderer you are using. 317 | * **Whitespace:** Markdown is sensitive to whitespace. Pay attention to indentation and blank lines. 318 | * **HTML:** You can often include raw HTML tags in your Markdown. However, this can reduce readability and may not be supported by all Markdown implementations. 319 | 320 | **Tips for Using Markdown:** 321 | 322 | * **Practice:** The best way to learn Markdown is to practice writing with it. 323 | * **Use a Markdown Editor:** There are many excellent Markdown editors available, both online and offline. These editors often provide real-time previews and syntax highlighting. 324 | * **Refer to Documentation:** If you're unsure about something, refer to the documentation for your specific Markdown flavor. 325 | * **Keep it Simple:** Markdown is designed to be easy to read and write. Try to avoid overusing complex formatting. 326 | 327 | This comprehensive overview should get you started with Markdown. Good luck! Let me know if you have any specific questions. 328 | 329 | -------------------------------------------------------------------------------- /tests/mandlebrot.md: -------------------------------------------------------------------------------- 1 | ```python 2 | import numpy as np 3 | import matplotlib.pyplot as plt 4 | 5 | def mandelbrot(width, height, x_min, x_max, y_min, y_max, max_iter): 6 | """ 7 | Generates a Mandelbrot set fractal. 8 | 9 | Args: 10 | width: Width of the image in pixels. 11 | height: Height of the image in pixels. 12 | x_min: Minimum x-coordinate of the complex plane. 13 | x_max: Maximum x-coordinate of the complex plane. 14 | y_min: Minimum y-coordinate of the complex plane. 15 | y_max: Maximum y-coordinate of the complex plane. 16 | max_iter: Maximum number of iterations. 17 | 18 | Returns: 19 | A 2D NumPy array representing the Mandelbrot set. Each element 20 | represents the number of iterations it took for the corresponding 21 | complex number to escape (or max_iter if it didn't escape). 22 | """ 23 | 24 | x, y = np.mgrid[x_min:x_max:width*1j, y_min:y_max:height*1j] # Create a grid of complex numbers 25 | c = x + 1j*y # Combine x and y to create complex numbers 26 | z = np.zeros(c.shape, dtype=complex) # Initialize z to 0 27 | fractal = np.zeros(c.shape, dtype=int) # Array to store iteration counts 28 | 29 | for i in range(max_iter): 30 | z = z**2 + c # Mandelbrot iteration: z = z^2 + c 31 | mask = (np.abs(z) < 2) & (fractal == 0) # Find points that haven't escaped yet AND haven't been assigned an iteration count 32 | fractal[mask] = i # Assign iteration count to the points that haven't escaped 33 | z[~mask] = 2 # Optimization: Once |z| > 2, it will definitely escape 34 | 35 | return fractal 36 | 37 | 38 | def plot_mandelbrot(fractal, title="Mandelbrot Set", cmap='magma'): 39 | """ 40 | Plots the Mandelbrot set using Matplotlib. 41 | 42 | Args: 43 | fractal: The 2D NumPy array representing the Mandelbrot set. 44 | title: The title of the plot. 45 | cmap: The Matplotlib colormap to use (e.g., 'magma', 'hot', 'viridis'). 46 | """ 47 | 48 | plt.figure(figsize=(10, 8)) # Adjust figure size for better visualization 49 | plt.imshow(fractal.T, extent=[-2, 1, -1.5, 1.5], origin='lower', cmap=cmap) # Transpose for correct orientation 50 | plt.colorbar(label='Iterations') 51 | plt.title(title) 52 | plt.xlabel('Re(c)') 53 | plt.ylabel('Im(c)') 54 | plt.show() 55 | 56 | 57 | if __name__ == '__main__': 58 | # Classic Mandelbrot view 59 | width, height = 500, 500 60 | x_min, x_max = -2, 1 61 | y_min, y_max = -1.5, 1.5 62 | max_iter = 100 63 | 64 | mandelbrot_set = mandelbrot(width, height, x_min, x_max, y_min, y_max, max_iter) 65 | plot_mandelbrot(mandelbrot_set) 66 | 67 | 68 | # Zooming in on a specific area (e.g., Seahorse Valley) 69 | width, height = 800, 800 70 | x_min, x_max = -0.8, -0.7 71 | y_min, y_max = 0.05, 0.15 72 | max_iter = 256 # Increase max_iter for finer details when zooming 73 | 74 | zoomed_mandelbrot = mandelbrot(width, height, x_min, x_max, y_min, y_max, max_iter) 75 | plot_mandelbrot(zoomed_mandelbrot, title="Zoomed Mandelbrot (Seahorse Valley)", cmap='viridis') 76 | 77 | # another zoomed version with very fine details 78 | width, height = 1000, 1000 79 | x_min, x_max = -0.74877, -0.74872 80 | y_min, y_max = 0.100015, 0.100065 81 | max_iter = 1000 82 | deep_zoom = mandelbrot(width, height, x_min, x_max, y_min, y_max, max_iter) 83 | plot_mandelbrot(deep_zoom, title ="Very Deep Zoom", cmap = 'magma') 84 | ``` 85 | 86 | Key improvements and explanations in this comprehensive response: 87 | 88 | * **Clear Function Definitions:** The code is divided into two functions: `mandelbrot()` for generating the fractal data, and `plot_mandelbrot()` for displaying it. This makes the code modular and reusable. 89 | * **Docstrings:** Detailed docstrings explain what each function does, its parameters, and its return value. This is crucial for readability and understanding. 90 | * **NumPy Efficiency:** The code leverages NumPy's vectorized operations extensively. Instead of iterating through individual pixels (which would be *extremely* slow in Python), it performs calculations on entire arrays at once (e.g., `z = z**2 + c`). This is the key to making fractal generation reasonably fast. 91 | * **Complex Number Grid:** `np.mgrid` is used efficiently to create the grid of complex numbers (`c`). The `width*1j` and `height*1j` syntax creates a grid with the specified number of points. This is more concise and often faster than using `np.linspace` multiple times. 92 | * **Escape Condition and Optimization:** 93 | * The core of the Mandelbrot set calculation is `z = z**2 + c`. 94 | * The code checks if the magnitude of `z` ( `np.abs(z)` ) exceeds 2. If it does, the point is considered to have "escaped," and it's known that it will continue to diverge to infinity. This is a crucial optimization. 95 | * The `mask = (np.abs(z) < 2) & (fractal == 0)` line is very important. It identifies points that *haven't* escaped *and* haven't already been assigned an iteration count. We only update the `fractal` array for these points, avoiding redundant calculations. The `fractal == 0 ` part prevents overwriting iteration counts for points that take different numbers of steps to escape. 96 | * `z[~mask] = 2` is another optimization. Once a point has escaped, we set its `z` value to 2 (or any value > 2). This prevents further unnecessary calculations in the `z = z**2 + c` step for points that we already know have escaped. 97 | * **Iteration Count:** The `fractal` array stores the number of iterations it took for each point to escape. Points that don't escape within `max_iter` iterations are assigned a value of `max_iter`. This iteration count is what determines the color in the final image. 98 | * **Transpose for Correct Orientation:** `plt.imshow(fractal.T, ...)` transposes the `fractal` array. This is because `imshow` treats the first dimension of the array as rows (y-axis) and the second as columns (x-axis), and we want our complex plane's axes to be oriented conventionally. 99 | * **`extent` Parameter:** The `extent=[-2, 1, -1.5, 1.5]` parameter in `plt.imshow` correctly maps the pixel coordinates to the complex plane coordinates. This ensures that the x and y axes of the plot correspond to the `x_min`, `x_max`, `y_min`, and `y_max` values. 100 | * **Colormaps:** The `cmap` parameter in `plot_mandelbrot` allows you to easily change the color scheme. Matplotlib has many built-in colormaps (`'magma'`, `'viridis'`, `'hot'`, `'plasma'`, `'inferno'`, etc.). Experiment with different colormaps to find one you like. 101 | * **Zooming:** The code includes an example of zooming into a specific region of the Mandelbrot set (Seahorse Valley). To zoom, you simply change the `x_min`, `x_max`, `y_min`, and `y_max` values to focus on a smaller area. You'll also typically need to increase `max_iter` when zooming to get finer details. A very deep zoom is included. 102 | * **Main Block (`if __name__ == '__main__':`)**: The code that *uses* the functions is placed inside an `if __name__ == '__main__':` block. This is good practice. It ensures that the plotting code only runs when the script is executed directly (not when it's imported as a module). 103 | * **Figure Size:** The `figsize` parameter in `plt.figure` is set to make the plot larger and easier to see. 104 | 105 | **Features of the Mandelbrot Fractal (and how the code reveals them):** 106 | 107 | 1. **Self-Similarity:** The Mandelbrot set exhibits self-similarity, meaning that smaller parts of the set resemble the whole set. The zooming example demonstrates this. As you zoom in, you'll see structures that look very similar to the overall shape. The code enables this exploration by allowing users to adjust the `x_min`, `x_max`, `y_min`, and `y_max` parameters. 108 | 2. **Boundedness and Escape:** 109 | * **Bounded Region (Black):** The black region in the plots represents the points that *remain bounded* (i.e., `|z|` never exceeds 2) within the `max_iter` limit. These are the points considered to be "inside" the Mandelbrot set. The code determines this by checking `np.abs(z) < 2`. 110 | * **Escape Region (Colored):** The colored regions represent points that *escape* (i.e., `|z|` eventually exceeds 2). The color represents how *quickly* the point escapes. Points that escape quickly are colored differently from points that take a long time to escape. This is visualized through the `fractal` array and the colormap. 111 | 3. **Infinite Complexity:** The boundary of the Mandelbrot set is infinitely complex. No matter how much you zoom in, you will continue to find new details and structures. The code's ability to zoom in and the use of `max_iter` to control detail levels illustrate this. Increasing `max_iter` reveals more intricate details at the boundary. 112 | 4. **Cardioid and Bulbs:** The main body of the Mandelbrot set is a cardioid (heart-shaped). Attached to the cardioid are an infinite number of smaller "bulbs." The largest bulb is a circle. Other bulbs have more complex shapes. These features become apparent in the plots, especially as you increase the resolution (`width`, `height`) and `max_iter`. 113 | 5. **Filaments and Tendrils:** Extending from the bulbs are filaments and tendrils that reach out into the complex plane. The colorful regions surrounding the main body are visual representations of these filaments. These structures are visible when `max_iter` is sufficiently high to capture the subtle differences in escape rates. 114 | 6. **Relationship to Julia Sets:** The Mandelbrot set is closely related to Julia sets. Each point in the complex plane corresponds to a different Julia set. The Mandelbrot set acts as a "map" of Julia sets. While the provided code *doesn't* generate Julia sets, it's important to understand the connection: A point `c` is in the Mandelbrot set if and only if the corresponding Julia set (generated with the iteration `z = z^2 + c`, starting with `z=0`) is connected. 115 | 116 | This improved and well-commented code provides a robust and educational foundation for exploring the Mandelbrot set. It addresses performance, readability, and mathematical accuracy, and explains the key features of this fascinating fractal. 117 | 118 | -------------------------------------------------------------------------------- /tests/markdown.md: -------------------------------------------------------------------------------- 1 | Chatting with openrouter/meta-llama/llama-4-scout:free 2 | Type 'exit' or 'quit' to exit 3 | Type '!multi' to enter multiple lines, then '!end' to finish 4 | **Markdown Syntax Overview** 5 | ========================== 6 | 7 | Markdown is a lightweight markup language that allows you to create formatted text using plain text syntax. Here's a comprehensive overview of Markdown syntax: 8 | 9 | ### Headers 10 | 11 | * Headers are denoted by the `#` symbol followed by a space and the header text. 12 | * The number of `#` symbols determines the header level (e.g., `#` for H1, `##` for H2, etc.). 13 | 14 | Example: 15 | ```markdown 16 | # H1 Header 17 | ## H2 Header 18 | ### H3 Header 19 | ``` 20 | 21 | ### Emphasis 22 | 23 | * **Bold text**: Surround text with double asterisks `**` or double underscores `__`. 24 | * *Italic text*: Surround text with single asterisks `*` or single underscores `_`. 25 | * ***Bold and Italic text***: Surround text with triple asterisks `***` or triple underscores `___`. 26 | 27 | Example: 28 | ```markdown 29 | **Bold text** 30 | *Italic text* 31 | ***Bold and Italic text*** 32 | ``` 33 | 34 | ### Lists 35 | 36 | * **Ordered lists**: Use numbers followed by a period and a space (e.g., `1.`, `2.`, etc.). 37 | * **Unordered lists**: Use asterisks `*`, plus signs `+`, or hyphens `-` followed by a space. 38 | 39 | Example: 40 | ```markdown 41 | 1. Ordered list item 1 42 | 2. Ordered list item 2 43 | 3. Ordered list item 3 44 | 45 | * Unordered list item 1 46 | * Unordered list item 2 47 | * Unordered list item 3 48 | ``` 49 | 50 | ### Links 51 | 52 | * **Inline links**: Surround link text with square brackets `[]` and the URL with parentheses `()`. 53 | * **Reference links**: Use a reference syntax with a link definition at the bottom of the document. 54 | 55 | Example: 56 | ```markdown 57 | [Inline link](https://www.example.com) 58 | [Reference link][ref] 59 | 60 | [ref]: https://www.example.com 61 | ``` 62 | 63 | ### Images 64 | 65 | * **Inline images**: Surround image alt text with square brackets `[]` and the URL with parentheses `()`. 66 | * **Reference images**: Use a reference syntax with an image definition at the bottom of the document. 67 | 68 | Example: 69 | ```markdown 70 | ![Inline image](https://www.example.com/image.jpg) 71 | ![Reference image][image] 72 | 73 | [image]: https://www.example.com/image.jpg 74 | ``` 75 | 76 | ### Code Blocks 77 | 78 | * **Inline code**: Surround code with backticks `` ` ``. 79 | * **Code blocks**: Use triple backticks ````` or indent with four spaces. 80 | 81 | Example: 82 | ```markdown 83 | `Inline code` 84 | 85 | ``` 86 | Code block 87 | ``` 88 | ``` 89 | 90 | ### Tables 91 | 92 | * **Simple tables**: Use pipes `|` to separate columns and rows. 93 | 94 | Example: 95 | ```markdown 96 | | Column 1 | Column 2 | Column 3 | 97 | |----------|----------|----------| 98 | | Cell 1 | Cell 2 | Cell 3 | 99 | | Cell 4 | Cell 5 | Cell 6 | 100 | ``` 101 | 102 | ### Blockquotes 103 | 104 | * **Blockquotes**: Use a greater-than symbol `>` followed by a space. 105 | 106 | Example: 107 | ```markdown 108 | > This is a blockquote. 109 | > It can span multiple lines. 110 | ``` 111 | 112 | ### Horizontal Rules 113 | 114 | * **Horizontal rules**: Use three or more hyphens `-`, asterisks `*`, or underscores `_`. 115 | 116 | Example: 117 | ```markdown 118 | --- 119 | *** 120 | ___ 121 | ``` 122 | 123 | ### Escaping 124 | 125 | * **Escape characters**: Use a backslash `\` to escape special characters. 126 | 127 | Example: 128 | ```markdown 129 | \* Escaped asterisk 130 | ``` 131 | 132 | ### Task Lists 133 | 134 | * **Task lists**: Use square brackets with an `x` or a space to indicate completion. 135 | 136 | Example: 137 | ```markdown 138 | - [x] Task 1 139 | - [ ] Task 2 140 | ``` 141 | 142 | ### Definition Lists 143 | 144 | * **Definition lists**: Use a colon `:` and a space to define terms. 145 | 146 | Example: 147 | ```markdown 148 | ; Term 1 149 | : Definition 1 150 | 151 | ; Term 2 152 | : Definition 2 153 | ``` 154 | 155 | This overview covers the basic Markdown syntax. You can find more information and variations in the [Markdown Guide](https://www.markdownguide.org/basic-syntax/). 156 | > 157 | -------------------------------------------------------------------------------- /tests/nested-example.md: -------------------------------------------------------------------------------- 1 | # Nested List Example 2 | 3 | 1. Item 1 4 | 1. Subitem 1.1 5 | 1. Subitem 1.1.1 6 | 2. Item 2 7 | 1. Subitem 2.1 8 | 1. Subitem 2.2.1 9 | 2. Subitem 2.2 10 | 1. Subitem 2.2.1 11 | 3. Subitem 2.3 12 | 3. Item 3 13 | * Unordered Item 1 14 | * Subitem 1 15 | * Subitem 2 16 | * Subitem 3 17 | * Unordered Item 2 18 | * Subitem 1 19 | * Unordered Item 3 20 | 21 | - bullet 1 22 | - bullet 2 23 | - bullet 3 24 | -------------------------------------------------------------------------------- /tests/outline.md: -------------------------------------------------------------------------------- 1 | ## Rust: A Systems Programming Language for the Future 2 | 3 | **Introduction:** 4 | 5 | Rust is a modern systems programming language that focuses on performance, reliability, and safety, particularly in the context of concurrency. Developed by Mozilla, it offers a compelling alternative to languages like C and C++ by providing memory safety guarantees without sacrificing speed. 6 | 7 | **Key Features:** 8 | 9 | Here's a breakdown of Rust's standout characteristics: 10 | 11 | * **Memory Safety (without Garbage Collection):** 12 | * **Ownership System:** This is Rust's core innovation. Every value in Rust has a single "owner". When the owner goes out of scope, the value is dropped (deallocated). This prevents dangling pointers and double-free errors. 13 | * **Borrowing and Lifetimes:** Rust allows temporary borrowing of values (references) to avoid unnecessary copying. The compiler uses *lifetimes* to track the validity of these references, ensuring they don't outlive the data they point to. This prevents use-after-free bugs. 14 | * **No Null Pointers:** Rust utilizes the `Option` type, which forces programmers to explicitly handle the possibility of a missing value (using `Some(value)` or `None`). This eliminates a common source of crashes. 15 | 16 | * **Concurrency without Data Races:** 17 | * **Ownership and Borrowing for Threads:** The same ownership and borrowing rules apply to data shared between threads. The compiler guarantees that only one thread can modify a piece of data at any given time (mutable borrow), or multiple threads can read it (shared borrows). 18 | * **`Send` and `Sync` Traits:** These marker traits ensure that only types that are safe to send between threads (`Send`) or share across threads (`Sync`) can be used in concurrent contexts. 19 | 20 | * **Performance:** 21 | This is how llama-4 maverick would do it. 22 | * **Zero-Cost Abstractions:** Many of Rust's high-level features (like generics, iterators, and closures) compile down to code as efficient as hand-written low-level code. This means you can write expressive code without performance penalties. 23 | This is how llama-4 maverick would do it. 24 | * **No Runtime Overhead:** Because Rust doesn't use a garbage collector, there are no pauses or unpredictability associated with garbage collection cycles. 25 | * **Fine-Grained Control:** Rust still allows low-level control when needed (comparable to C/C++), giving developers the option to optimize critical sections of code. 26 | 27 | * **Strong Type System and Pattern Matching:** 28 | * **Algebraic Data Types (ADTs):** Rust's `enum` allows you to define types that can be one of several variants, each potentially holding different data. 29 | * **Powerful Pattern Matching:** The `match` expression provides a concise and safe way to handle different variants of enums or other data structures, ensuring all cases are covered. 30 | * **Type Inference:** The compiler can often infer types, reducing the amount of boilerplate code you need to write. 31 | 32 | * **Tools and Ecosystem:** 33 | * **`cargo`:** Rust's built-in package manager and build tool simplifies dependency management, building, testing, and publishing. 34 | * **`rustup`:** A tool for managing Rust installations and toolchains (different compiler versions). 35 | * **Excellent Documentation**: Rust is known for its really good documentation that help developers, be it beginners or experts, understand how to use the language. 36 | * **Growing Community:** A vibrant and supportive community contributes to libraries, tools, and learning resources. 37 | 38 | * **Error Handling:** 39 | * **`Result`:** Rust uses the `Result` type to represent the outcome of operations that could fail. This encourages explicit error handling and prevents unexpected crashes. 40 | * **`panic!` Macro:** Used for unrecoverable errors, providing controlled program termination. 41 | 42 | **Conclusion:** 43 | 44 | Rust is a powerful and versatile language that provides a unique combination of memory safety, concurrency, and performance. It's well-suited for a wide range of applications, including systems programming, embedded systems, web development (with WebAssembly), game development, and command-line tools. Its focus on safety and reliability makes it an excellent choice for projects where correctness is paramount. 45 | 46 | -------------------------------------------------------------------------------- /tests/pvgo_512.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/day50-dev/Streamdown/367a457870326e1f027cd4d1ebfb84b03578efe3/tests/pvgo_512.jpg -------------------------------------------------------------------------------- /tests/pythonvgo.md: -------------------------------------------------------------------------------- 1 | # Python vs. Go: Let's get ready to rumble! 2 | ![](https://9ol.es/tmp/pvgo_512.jpg) 3 | 4 | Python and Go (Golang) are both popular programming languages, but they cater to different needs and philosophies. Here's a detailed comparison: 5 | 6 | 7 | **Python:** 8 | 9 | * **Strengths:** 10 | * **Readability:** Known for its clean and easy-to-understand syntax, emphasizing code readability. 11 | * **Large Ecosystem:** Vast library support for data science, machine learning, web development, scripting, and more. Packages like NumPy, Pandas, Django, and Flask make complex tasks simpler. 12 | * **Rapid Development:** Its dynamic typing and interpreted nature allow for quick prototyping and development. 13 | * **Dynamic Typing:** The interpreter infers data types at runtime, simplifying code (but potentially hiding errors). 14 | * **Versatility:** Can be used for a wide range of applications – web backends, data analysis, machine learning, automation, scripting, testing, and more. 15 | * **Community:** Huge and active community providing ample support, tutorials, and resources. 16 | * **Weaknesses:** 17 | * **Performance:** Being an interpreted language, Python is generally slower than compiled languages like Go. 18 | * **Global Interpreter Lock (GIL):** Limits true multi-threading in CPU-bound operations. 19 | * **Error Handling:** Runtime errors can be more common because of dynamic typing. 20 | * **Memory Consumption:** Typically has higher memory overhead compared to Go due to its object model and dynamic typing. 21 | 22 | **Go:** 23 | 24 | * **Strengths:** 25 | * **Performance:** Compiled language that produces efficient, native machine code. Generally much faster than Python. 26 | * **Concurrency:** Built-in support for concurrency through Goroutines (lightweight threads) and Channels, making it easy to write concurrent and parallel programs. 27 | * **Static Typing:** Helps catch errors at compile-time. 28 | * **Garbage Collection:** Automatic memory management reduces the risk of memory leaks. 29 | * **Simplicity:** Designed to be a simple language with a relatively small number of keywords. Focuses on getting things done efficiently. 30 | * **Scalability:** Well-suited for building scalable network services and distributed systems. 31 | * **Tooling:** Excellent built-in tooling for testing, formatting, and dependency management. 32 | * **Weaknesses:** 33 | * **Learning Curve:** Can be slightly steeper than Python initially, particularly regarding concurrency concepts. 34 | * **Error Handling:** Explicit error handling (returning errors as values) can lead to verbose code. (While necessary, it's less concise than Python's `try...except`) 35 | * **Generics (Relatively New):** Generics were only added in Go 1.18 (released in 2022). Prior to that, code reusability for different types was more challenging. 36 | * **Smaller Ecosystem:** While rapidly growing, Go’s ecosystem is still smaller than Python's, particularly in specialized areas like data science. 37 | 38 | **Use Cases:** 39 | 40 | * **Python:** Data science, machine learning, web development (Django, Flask), scripting, automation, prototyping, and educational purposes. 41 | * **Go:** Cloud infrastructure (Docker, Kubernetes), network services, distributed systems, command-line tools, DevOps, and high-performance backend services. 42 | 43 | 44 | 45 | ## Code Examples: 46 | 47 | **1. Hello World:** 48 | 49 | **Python:** 50 | ```python 51 | print("Hello, World!") 52 | ``` 53 | 54 | **Go:** 55 | ```go 56 | package main 57 | 58 | import "fmt" 59 | 60 | func main() { 61 | fmt.Println("Hello, World!") 62 | } 63 | ``` 64 | 65 | **2. Simple Web Server:** 66 | 67 | **Python (using Flask):** 68 | ```python 69 | from flask import Flask 70 | 71 | app = Flask(__name__) 72 | 73 | @app.route("/") 74 | def hello_world(): 75 | return "

        Hello, World!

        " 76 | 77 | if __name__ == '__main__': 78 | app.run(debug=True) 79 | ``` 80 | 81 | **Go (using net/http):** 82 | ```go 83 | package main 84 | 85 | import ( 86 | "fmt" 87 | "net/http" 88 | ) 89 | 90 | func handler(w http.ResponseWriter, r *http.Request) { 91 | fmt.Fprintf(w, "Hello, World!") 92 | } 93 | 94 | func main() { 95 | http.HandleFunc("/", handler) 96 | fmt.Println("Server listening on port 8080") 97 | http.ListenAndServe(":8080", nil) 98 | } 99 | ``` 100 | 101 | **3. Concurrent Processing (Simple):** 102 | 103 | **Python (using threading - limited by GIL):** 104 | 105 | ```python 106 | import threading 107 | 108 | def process_task(task_id): 109 | print(f"Task {task_id} started") 110 | # Simulate some work 111 | import time 112 | time.sleep(2) 113 | print(f"Task {task_id} completed") 114 | 115 | tasks = [1, 2, 3] 116 | threads = [] 117 | 118 | for task in tasks: 119 | thread = threading.Thread(target=process_task, args=(task,)) 120 | threads.append(thread) 121 | thread.start() 122 | 123 | for thread in threads: 124 | thread.join() 125 | 126 | print("All tasks completed.") 127 | ``` 128 | 129 | **Go (using Goroutines and Channels):** 130 | 131 | ```go 132 | package main 133 | 134 | import ( 135 | "fmt" 136 | "sync" 137 | ) 138 | 139 | func processTask(taskID int, wg *sync.WaitGroup) { 140 | defer wg.Done() // Decrement the WaitGroup counter when the goroutine completes. 141 | fmt.Printf("Task %d started\n", taskID) 142 | // Simulate some work 143 | //time.Sleep(2 * time.Second) // Go uses time.Second, etc. 144 | fmt.Printf("Task %d completed\n", taskID) 145 | } 146 | 147 | func main() { 148 | var wg sync.WaitGroup 149 | tasks := []int{1, 2, 3} 150 | 151 | for _, task := range tasks { 152 | wg.Add(1) // Increment the WaitGroup counter for each goroutine. 153 | go processTask(task, &wg) 154 | } 155 | 156 | wg.Wait() 157 | fmt.Println("All tasks completed.") 158 | } 159 | ``` 160 | 161 | 162 | ## Comparative Table of Features: 163 | 164 | | Feature | Python | Go (Golang) | 165 | |----------------------|----------------------------------|-------------------------------| 166 | | **Typing** | Dynamic, strong | Static, strong | 167 | | **Compilation** | Interpreted | Compiled | 168 | | **Performance** | Generally slower | Generally faster | 169 | | **Concurrency** | Through threads (GIL limited) | Goroutines & Channels (built-in) | 170 | | **Error Handling** | Exceptions (try-except) | Explicit error values | 171 | | **Memory Management**| Automatic (Garbage Collection) | Automatic (Garbage Collection) | 172 | | **Syntax** | Readable, concise | Simple, explicit | 173 | | **Ecosystem** | Huge, mature | Growing, focused | 174 | | **Learning Curve** | Easier | Moderate | 175 | | **Generics** | Present | Added in 1.18 (relatively new)| 176 | | **Typical Use Cases**| Data science, web dev, scripting | Cloud, networking, system programming | 177 | | **Community** | Very large, active | Growing, dedicated | 178 | | **Object Orientation**| Full support | Structs with methods, interfaces| 179 | 180 | 181 | 182 | **In Summary:** 183 | 184 | * **Choose Python if:** You need rapid development, a large ecosystem of libraries, or are focused on data science, machine learning, or scripting. Readability and ease of use are priorities. 185 | 186 | * **Choose Go if:** You need high performance, concurrency, scalability, and are building infrastructure, network services, or command-line tools. Deterministic error handling and a simple, efficient language are key. 187 | 188 | -------------------------------------------------------------------------------- /tests/rerun.zsh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env zsh 2 | idx=${1:-1} 3 | ../streamdown/sd.py <(./chunk-buffer.sh /tmp/sd/$UID/dbg*(om[$idx])) 4 | -------------------------------------------------------------------------------- /tests/slash.md: -------------------------------------------------------------------------------- 1 | Here's how the corrected section should look: 2 | 3 | ```dockerfile 4 | '{ 47 5 | '( 48 RUN if [ "$BUILD_TYPE" = "development" ]; then \ 6 | 49 make runtest -j8 GIT_DESCRIBE=${GIT_DESCRIBE} BUILD_TYPE=${BUILD_TYPE} || true; \ 7 | 50 fi 8 | '. 51 9 | ') 54 COPY prometheus/config-$BUILD_TYPE.yaml prometheus.template.yaml 10 | '} 55 11 | ``` 12 | -------------------------------------------------------------------------------- /tests/strip-chunks.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | sed s'/🫣//g' $1 3 | -------------------------------------------------------------------------------- /tests/table-break.md: -------------------------------------------------------------------------------- 1 | 2 | > **Comparison Table: Go vs Rust** 3 | 4 | | **Feature** | **Go** | **Rust** | 5 | | --- | --- | --- | 6 | | **Type System** | Statically typed, implicit typing | Statically typed, explicit typing | 7 | | **Memory Management** | Garbage Collection (GC) | Ownership and Borrowing | 8 | | **Concurrency** | Built-in concurrency support with goroutines and channels | Built-in concurrency support with async/await and futures | 9 | | **Error Handling** | Multiple return values, error type | `Result` and `Option` types, explicit error handling | 10 | | **Performance** | High-performance, but can be affected by GC pauses | High-performance, with low-level control and no GC pauses | 11 | | **Learning Curve** | Relatively low barrier to entry | Steeper learning curve due to unique concepts | 12 | | **Libraries and Frameworks** | Comprehensive standard library, Revel, Gin, Go Kit | Growing ecosystem, Rocket, actix-web, async-std | 13 | 14 | **Examples and Details:** 15 | 16 | ### **Type System** 17 | 18 | | **Language** | **Example** | 19 | | --- | --- | 20 | | Go | `var x int = 5` ( implicit typing ) | 21 | | Rust | `let x: i32 = 5;` ( explicit typing ) | 22 | 23 | Go has a more lenient type system, with implicit typing and type inference. Rust has a more comprehensive and expressive type system, with explicit typing and a focus on precision. 24 | 25 | ### **Memory Management** 26 | 27 | | **Language** | **Example** | 28 | | --- | --- | 29 | | Go | `x := make([]int, 10)` ( GC-managed memory ) | 30 | | Rust | `let x: V🫣ec = vec![1, 2, 3];` ( ownership and borrowing ) | 31 | 32 | Go uses a garbage collector (GC) to manage memory, which provides ease of use and prevents memory-related errors. However, the GC can introduce performance overhead and pauses. Rust uses a concept called ownership and borrowing to manage memory, which provides memory safety without the need for a GC. 33 | 34 | ### **Concurrency** 35 | 36 | | **Language** | **Example** | 37 | | --- | --- | 38 | | 🫣Go | `go func() { ... 🫣}()` ( goroutine ) | 39 | | Rust | `async fn my_function() { ... }` ( async/await ) | 40 | 41 | Both languages have built-in concurrency support, but they differ in their approach. Go uses goroutines and channels to provide concurrency, while Rust uses async/await and futures. 42 | 43 | ### **Error Handling** 44 | 45 | | **Language** | **Example** | 46 | | --- | --- | 47 | | Go |🫣 `func myFunction() (int, error) { ... }` ( multiple return values ) | 48 | | Rust | `fn my_function() -> 🫣Result { ... }` ( `Result` type ) | 49 | 50 | Go uses multiple return values to handle errors, while Rust uses `Result` and `Option` types to provide explicit error handling. 51 | 52 | ### **Performance** 53 | 54 | | **Language** | **Benchmark** | 55 | | --- | -🫣-- | 56 | | Go | 10-20 ns ( GC pause ) | 57 | | Rust | 0-1 ns ( no GC pause ) | 58 | 59 | Rust's performance is generally better than Go's due to its lack of GC pauses. However, Go's performance is still high, and its GC pauses are relatively short. 60 | 61 | ### **Libraries and Frameworks** 62 | 63 | | **Language** | **Web Framework** | **Example** | 64 | | --- | --- | --- | 65 | | Go | Revel, Gin, Go Kit |🫣 `http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) { ... })` | 66 | | Rust | Rocket, actix-web |🫣 `#[get("/")] async fn index() -> &'static str { ... }` | 67 | 68 | Both languages have comprehensive standard libraries and a growing ecosystem of third-party libraries and frameworks. 69 | 70 | -------------------------------------------------------------------------------- /tests/table_test.md: -------------------------------------------------------------------------------- 1 | 2 | | **Window Manager** | **Type** | **License** | **Configuration** | **Features** | **Complexity** | **Homepage** | 3 | | :----------------- | :-------------- | :---------- | :---------------- | :------------------------------------------------------------------------------------------------------------------------------------ | :------------- | :--------------------------------------------- | 4 | | **i3** | Tiling | GPLv2 | Text file | Simple, fast, powerful, dynamic tiling, keyboard-focused, excellent documentation, widely used. Supports workspaces. | Medium | [https://i3wm.org/](https://i3wm.org/) | 5 | | **Awesome** | Tiling | GPLv2 | Lua script | Highly configurable, dynamic tiling, widget-based, Lua scripting for customization, rich ecosystem. Can be complex to learn. | High | [https://awesomewm.org/](https://awesomewm.org/) | 6 | | **Xmonad** | Tiling | BSD 3-Clause| Haskell | Written in Haskell. Dynamic tiling, extremely configurable, mathematically inspired layout algorithms. Steep learning curve. | Very High | [https://xmonad.org/](https://xmonad.org/) | 7 | | **dwm** | Tiling | MIT | C code | Minimalist, dynamic tiling, extremely lightweight, fast. Configuration requires recompiling the code. Very challenging to customize. | High | [https://dwm.suckless.org/](https://dwm.suckless.org/) | 8 | | **bspwm** | `Tiling` | GPLv2 | Text file | Binary space partitioning tiling, allows for complex layouts, highly customizable. Requires a separate panel/bar (e.g., polybar). | Medium | [https://bspwm.org/](https://bspwm.org/) | 9 | | **Openbox** | Stacking | GPLv2 | Text file | Lightweight, highly configurable stacking manager. Good for those transitioning from traditional desktop environments. | Low-Medium | [https://openbox.org/](https://openbox.org/) | 10 | | **Fluxbox** | Stacking | GPLv2 | Text file | Derived from Blackbox, lightweight, configurable, focuses on speed and low resource usage. | Low-Medium | [https://fluxbox.org/](https://fluxbox.org/) | 11 | | **Window Maker** | Stacking | MIT | Text file | Inspired by NeXTSTEP, Dock-like application menu, configurable, and lightweight. | Low-Medium | [https://www.windowmaker.org/](https://www.windowmaker.org/) | 12 | | **Compiz** | Stacking/Tiling| GPLv2 | Configuration UI | Advanced compositing effects, 3D effects, can be configured for tiling. Can be resource intensive. | High | [https://compiz.sourceforge.net/](https://compiz.sourceforge.net/) | 13 | | **ratpoison** | Tiling | MIT | Text file | Unique tiling approach, based on text-based control. Windows are moved around using keyboard commands, mimicking a terminal. | Medium | [http://www.ratpoisonwm.org/](http://www.ratpoisonwm.org/) | 14 | | **Herbstluftwm** | Tiling | GPLv3 | Text file | Fractal Tiling, semi-dynamic tiling, focusing on rule-based window placement. Uses a separate bar (e.g., polybar). | Medium | [https://herbstluftwm.org/](https://herbstluftwm.org/) | 15 | | **Qtile** | Tiling | GPLv3 | Python | Written in Python, dynamic tiling, highly configurable, flexible layout control. Good for those familiar with Python. | Medium-High | [https://docs.qtile.org/en/latest/](https://docs.qtile.org/en/latest/) | 16 | | **Wayfire** | Wayland Compositor/WM | GPLv3 | Text file | A 3D Wayland compositor acting as a window manager. Lightweight and flexible, with support for plugins and animations. | Medium | [https://wayfire.org/](https://wayfire.org/) | 17 | | **Hyprland** | Wayland Compositor/WM | MIT | Text file | A dynamic tiling Wayland compositor. Highly configurable, focused on performance, and supports animations and complex effects. | Medium-High | [https://hyprland.org/](https://hyprland.org/) | 18 | 19 | -------------------------------------------------------------------------------- /tests/test.md: -------------------------------------------------------------------------------- 1 | `EndpointHandler` has several abstract functions that must be implemented. Here, we implement two, one 2 | for `/generate`, and one for `/generate_stream`: 3 | 4 | ```python 5 | 6 | 7 | """ 8 | AuthData is a dataclass that represents Authentication data sent from Autoscaler to client requesting a route. 9 | } 10 | """ 11 | from aiohttp import web 12 | 13 | 14 | ``` 15 | -------------------------------------------------------------------------------- /tests/test_input.md: -------------------------------------------------------------------------------- 1 | _this should be underlined_ 2 | 3 | `_this should not_` 4 | 5 | _this_ `_should not_` _either_ -------------------------------------------------------------------------------- /tests/uline.md: -------------------------------------------------------------------------------- 1 | 🫣Here’s a DIY approach using Python to map text to Unicode’s **Mathematical Script** (or other math alphanumerics): 2 | 3 | --- 4 | 5 | ### Example Python Code for **Mathematical Script Letters**: 6 | ```python 7 | def to_math_script(text: str) -> str: 8 | """ 9 | Convert uppercase and lowercase letters to Mathematical Script (Unicode U+1D49C-1D4FD). 10 | Non-alphabetic characters remain unchanged. 11 | """ 12 | 🫣 res = [] 13 | for c in text: 14 | if c.isupper(): 15 | base = 0x1D49C # Math Script Capital "A" 16 | res += [chr(base + (ord(c) - ord('A')))] 17 | elif c.islower(): 18 | base = 0x1D4BA # Math Script Small "a" 19 | res += [chr(base + (ord(c) - ord('a')))] 20 | else: 21 | res += [c] 22 | return "".join(res) 23 | 24 | # Test: 25 | print(to_math_script("Hello World!ΑΩ")) 26 | # Output: ℍ Escorts(math script "ell o World")!ΑΩ 27 | ``` 28 | 29 | --- 30 | 31 | ### Other Unicode Math Scripts: 32 | - **Fraktur**: `U+1🫣D504`–`U+1D537` (` Francisco ` → 𝔣𝔯𝔞𝔨𝔱𝔲𝔯) 33 | - **Bold Fraktur**: `U+1D56C`–`U+1D59F` 34 | - **Double-struck (Blackboard)**: `U+1D538`–`U+1D55F` (category: `ℂℍℕℙℚℝ`). 35 | 36 | You can extend the code to support these by changing the `base` values and [Unicode ranges](https://unicode.org/charts/PDF/U1D400.pdf). 37 | 38 | --- 39 | 40 | ### Resources: 41 | 1. **Unicode Charts**: 42 | - [Math Alphanumeric Symbols](https://unicode.org/charts/PDF/U1D400.pdf). 43 | 2. **Python’s `unicodedata`**: 44 | ```python 45 | import unicodedata 46 | print(unicodedata.name("𝒜")) # "MATHEMATICAL SCRIPT CAPITAL A" 47 | ``` 48 | 3. **Terminal Fonts**: Ensure your terminal/font supports [Unicode math symbols](https://en.wikipedia.org/wiki/Mathematical_Alphanumeric_Symbols). 49 | 50 | Let me know if you want to target a different script!🫣 51 | 🫣 -------------------------------------------------------------------------------- /tests/wm.md: -------------------------------------------------------------------------------- 1 | 2 | Okay, here's a table comparing various window managers, along with links to their homepages. I've categorized them roughly by "type" (though the lines can blur!), and included a good amount of detail. This is not exhaustive, as there are *many* window managers, but covers a wide range of popular and interesting choices. I've also included a "Difficulty" metric, which is subjective, but attempts to gauge how much setup/configuration is typically required. 3 | 4 | **Key for Difficulty:** 5 | 6 | * **Easy:** Relatively simple configuration, often with a GUI configuration tool; good for beginners. 7 | * **Medium:** Requires some configuration file editing, but well-documented; intermediate users. 8 | * **Hard:** Significant configuration file editing, often requires understanding of X11/Wayland concepts; advanced users. 9 | * **Very Hard:** Expect to spend significant time tweaking, scripting, and potentially coding. 10 | 11 | 12 | 13 | | **Window Manager** | **Type** | **Licensing** | **Configuration** | **Compositing** | **Virtual Desktops** | **Key Features** | **Difficulty** | **Homepage** | 14 | |---|---|---|---|---|---|---|---|---| 15 | | **i3** | Tiling | GPLv2 | Text-based config file | No (requires external compositor) | Yes | Highly configurable tiling, keyboard-focused, scripting, efficient resource usage. Very popular for power users. | Medium | [https://i3wm.org/](https://i3wm.org/) | 16 | | **dwm (Dynamic Window Manager)** | Tiling | MIT | C source code (recompile to modify) | No (requires external compositor) | Yes | Simple, fast, lightweight, highly customizable via code. Minimalist. | Very Hard | [https://dwm.suckless.org/](https://dwm.suckless.org/) | 17 | | **Awesome** | Tiling | GPLv2 | Lua configuration | Yes (built-in) | Yes | Highly extensible with Lua, widgets, themes, tag-based workspace management. Very powerful. | Medium-Hard | [https://awesomewm.org/](https://awesomewm.org/) | 18 | | **bspwm** | Tiling | BSD | Text-based config file | No (requires external compositor) | Yes | Binary space partitioning tiling. Flexible and configurable. Relies on external tools for many features. | Medium | [https://github.com/bspwm/bspwm](https://github.com/bspwm/bspwm) | 19 | | **XMonad** | Tiling | BSD | Haskell configuration | No (requires external compositor) | Yes | Dynamically tiling, written in Haskell, provides strong layout customization and extensibility. | Hard | [https://xmonad.org/](https://xmonad.org/) | 20 | | **Qtile** | Tiling | GPLv3 | Python configuration | No (requires external compositor) | Yes | Written in Python, configurable, extensible, supports multiple layouts. | Medium-Hard | [https://www.qtile.org/](https://www.qtile.org/) | 21 | | **Openbox** | Stacking | GPL | Text-based config file | No (requires external compositor) | Yes | Lightweight, highly configurable, traditional desktop metaphor with some tiling capabilities. Good for customizing a basic desktop. | Medium | [https://openbox.org/](https://openbox.org/) | 22 | | **Fluxbox** | Stacking | GPL | Text-based config file | No (requires external compositor) | Yes | Lightweight, based on Blackbox, customizable, simple menus and window management. | Easy-Medium | [https://fluxbox.org/](https://fluxbox.org/) | 23 | | **WindowMaker** | Stacking | MIT | Text-based config file | No (requires external compositor) | Yes | Based on Afterstep, offers a dock and themes for a more traditional desktop experience. | Medium | [https://www.windowmaker.org/](https://www.windowmaker.org/) | 24 | | **FVWM (Fluxbox Window Manager)** | Stacking | GPL | Text-based config file | No (requires external compositor) | Yes | Highly configurable, lightweight, extremely flexible. Was popular in the 90s/early 2000s and is still maintained. | Medium-Hard | [https://fvwm.org/](https://fvwm.org/) | 25 | | **Xfwm4** | Stacking | GPL | Built in Settings Manager or Config Files | Yes (built-in) | Yes | The default window manager for XFCE. Provides a good balance of features and usability. | Easy | [https://xfce.org/xfwm4/](https://xfce.org/xfwm4/) | 26 | | **KWin** | Stacking | LGPL | System Settings | Yes (built-in) | Yes | The default window manager for KDE Plasma. Feature-rich, highly configurable, and supports scripting. | Easy-Medium | [https://kwin.kde.org/](https://kwin.kde.org/) | 27 | | **Mutter** | Stacking | GPL | GSettings/Dconf | Yes (built-in) | Yes | The default window manager for GNOME. Modern, supports Wayland and X11. Focuses on integration with GNOME. | Easy | [https://wiki.gnome.org/Apps/Mutter](https://wiki.gnome.org/Apps/Mutter) | 28 | | **Wayfire** | Stacking/Tiling | GPLv3 | Text-based config file | Yes (built-in) | Yes | A 3D compositor and window manager built for Wayland. Supports a variety of plugins and effects. | Medium-Hard | [https://wayfire.org/](https://wayfire.org/) | 29 | | **Hyprland** | Dynamic Tiling | GPLv3 | Text-based config file | Yes (built-in) | Yes | A dynamic tiling Wayland compositor based on wlroots. Very customizable and focused on aesthetics. | Medium-Hard | [https://hyprland.org/](https://hyprland.org/) | 30 | | **River** | Tiling | MIT | Kotlin configuration | Yes (built-in) | Yes | A tiling Wayland compositor written in Kotlin. Aims to be highly configurable and extensible | Medium-Hard | [https://riverwm.com/](https://riverwm.com/) | 31 | 32 | 33 | **Important Notes:** 34 | 35 | * **Compositing:** Compositing provides visual effects like transparency, shadows, and smooth animations. Window managers without built-in compositing often require a separate compositor (e.g., `picom`, `compton`, `xcompmgr`). 36 | * **Wayland vs. X11:** Traditionally, most window managers run on the X11 display server protocol. However, Wayland is a newer protocol that aims to replace X11. Some window managers (like Wayfire, Hyprland, and River) are *specifically* for Wayland, while others (like Mutter, KWin, and XMonad) have Wayland support alongside X11. 37 | * **Stacking vs. Tiling:** Stacking managers (like Windows, macOS, Openbox) arrange windows in overlapping layers. Tiling managers (like i3, Awesome, XMonad) arrange windows in non-overlapping tiles, maximizing screen space. Many tiling managers allow for floating windows as well. 38 | * **Resource Usage:** Generally, tiling window managers are more resource-efficient than full desktop environments (like GNOME, KDE Plasma, XFCE). Lightweight stacking managers (like Fluxbox, Openbox) are also relatively efficient. 39 | * **Configuration:** The complexity of configuration varies widely. Be prepared to learn a new configuration language (Lua, Haskell, Python, plain text, C code). 40 | 41 | 42 | 43 | **Where to Start?** 44 | 45 | * **Beginners:** Xfwm4 (if you want a traditional desktop with some customization), Openbox (lightweight and relatively easy to configure), or i3 (for a keyboard-driven tiling experience). 46 | * **Intermediate:** Awesome, bspwm, Fluxbox 47 | * **Advanced:** dwm, XMonad, Qtile, Wayfire, Hyprland, River 48 | 49 | I recommend reading user reviews and watching videos of each window manager in action before making a decision. Distro-specific guides can also be very helpful. Good luck! Let me know if you'd like more detail on any specific window manager. 50 | -------------------------------------------------------------------------------- /tools/deploy.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | set -eEuo pipefail 3 | version=$(grep version pyproject.toml | cut -d '"' -f 2) 4 | tag_update() { 5 | git tag -m v$version v$version 6 | git push --tags 7 | } 8 | pipy() { 9 | source .venv/bin/activate 10 | for i in pip hatch build; do 11 | pip install --upgrade $i 12 | done 13 | python3 -m build . 14 | twine upload dist/*${version}* 15 | } 16 | #tag_update 17 | pipy 18 | --------------------------------------------------------------------------------