├── screenshots ├── pmacs_script.png ├── vedsel_script.png ├── pmacs_script_terminal.png ├── editor_readme_x2_buffers.png ├── browser_story_comment_links.png ├── editor_sked_edsel_directory.png ├── browser_story_comment_links_sl.png ├── editor_sked_edsel_directory_sl.png ├── browser_story_comment_frontpage.png └── browser_story_comment_frontpage_sl.png ├── editors ├── test │ ├── README.md │ ├── lines10.txt │ ├── lines30.txt │ └── lines40.txt ├── demo │ ├── README.md │ └── sked_fragments.py ├── dm.py ├── pm.py ├── vpm.py ├── bugs.md ├── breakpt.py └── breakpt.md ├── doc ├── README.md ├── platforms.md ├── modules.md ├── gracle_excerpts.txt ├── term.txt ├── term.md ├── other.md └── rationale.md ├── piety ├── apm.py ├── atimer_script.py ├── v2pmacs_script.py ├── vedsel_script.md ├── pmacs_script.py ├── edsel_script.py ├── atimer_script.txt ├── vpmacs_script.py ├── vedsel_script.py ├── apmacs.py ├── piety.py ├── edsel_script.txt ├── piety.txt ├── apyshell.py ├── eventloop.py ├── README.md ├── pmacs_blocking.md └── pmacs_script.md ├── coroutines ├── term_timer.py ├── timer_loops.py ├── term_timer.txt ├── README.md ├── aterminal.py ├── atimers.py └── timer_loops.txt ├── unix ├── terminal_util.py ├── README.md ├── terminal.py └── shell.py ├── threads ├── threads_3.py ├── threads_2.py ├── tm.py ├── threads_3.txt ├── README.md ├── timers.py ├── threads_2.txt └── threads_1.txt ├── python ├── runner.py ├── README.md ├── pyhelp.py └── pycall.py ├── bin ├── README.md └── paths ├── vt_terminal ├── README.md ├── keyseq_1.py ├── key.py ├── keyseq.py ├── keyseq.txt └── display.py ├── tasking ├── README.md ├── writer.txt ├── pyshell.py └── writer.py ├── browser ├── search.py ├── urls.py └── get.py ├── console ├── README.md ├── redirect.py └── console.py ├── BRANCH.md ├── README_54.md └── README.md /screenshots/pmacs_script.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jon-jacky/Piety/HEAD/screenshots/pmacs_script.png -------------------------------------------------------------------------------- /screenshots/vedsel_script.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jon-jacky/Piety/HEAD/screenshots/vedsel_script.png -------------------------------------------------------------------------------- /screenshots/pmacs_script_terminal.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jon-jacky/Piety/HEAD/screenshots/pmacs_script_terminal.png -------------------------------------------------------------------------------- /editors/test/README.md: -------------------------------------------------------------------------------- 1 | 2 | test 3 | ==== 4 | 5 | Sample text files used for testing the editors 6 | 7 | Revised Sep 2024 8 | 9 | -------------------------------------------------------------------------------- /screenshots/editor_readme_x2_buffers.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jon-jacky/Piety/HEAD/screenshots/editor_readme_x2_buffers.png -------------------------------------------------------------------------------- /screenshots/browser_story_comment_links.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jon-jacky/Piety/HEAD/screenshots/browser_story_comment_links.png -------------------------------------------------------------------------------- /screenshots/editor_sked_edsel_directory.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jon-jacky/Piety/HEAD/screenshots/editor_sked_edsel_directory.png -------------------------------------------------------------------------------- /screenshots/browser_story_comment_links_sl.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jon-jacky/Piety/HEAD/screenshots/browser_story_comment_links_sl.png -------------------------------------------------------------------------------- /screenshots/editor_sked_edsel_directory_sl.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jon-jacky/Piety/HEAD/screenshots/editor_sked_edsel_directory_sl.png -------------------------------------------------------------------------------- /editors/demo/README.md: -------------------------------------------------------------------------------- 1 | 2 | demo 3 | ==== 4 | 5 | Files for the demonstration explained in *editors/demo.md* 6 | 7 | Revised Sep 2024 8 | 9 | -------------------------------------------------------------------------------- /screenshots/browser_story_comment_frontpage.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jon-jacky/Piety/HEAD/screenshots/browser_story_comment_frontpage.png -------------------------------------------------------------------------------- /screenshots/browser_story_comment_frontpage_sl.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jon-jacky/Piety/HEAD/screenshots/browser_story_comment_frontpage_sl.png -------------------------------------------------------------------------------- /editors/test/lines10.txt: -------------------------------------------------------------------------------- 1 | line 1 2 | line 2 3 | line 3 4 | line 4 5 | line 5 6 | line 6 7 | line 7 8 | line 8 9 | line 9 10 | Line 10 11 | -------------------------------------------------------------------------------- /doc/README.md: -------------------------------------------------------------------------------- 1 | Piety notes and documents 2 | ========================= 3 | 4 | **Piety** is a notional operating system to be written in Python. 5 | This directory contains some notes and documents. 6 | 7 | Revised May 2024 8 | 9 | -------------------------------------------------------------------------------- /piety/apm.py: -------------------------------------------------------------------------------- 1 | # apm.py - script to start asynchronous pmacs in a piety session 2 | # 3 | # ...$ python -m piety 4 | # >>>> run(apm.py) 5 | # ... window appears ... 6 | 7 | import sked 8 | from sked import * 9 | import edsel 10 | from edsel import * 11 | from apmacs import apm 12 | 13 | win(22) 14 | apm() 15 | 16 | 17 | -------------------------------------------------------------------------------- /coroutines/term_timer.py: -------------------------------------------------------------------------------- 1 | """ 2 | term_timer.py - Demonstrate interleaving of aterminal reader and an atimer task 3 | in an asyncio event loop. Directions in term_timer.txt 4 | """ 5 | 6 | import aterminal, atimers 7 | 8 | ta = aterminal.loop.create_task(atimers.atimer(5,5,'A')) 9 | 10 | aterminal.main() 11 | 12 | -------------------------------------------------------------------------------- /editors/test/lines30.txt: -------------------------------------------------------------------------------- 1 | line 1 2 | line 2 3 | line 3 4 | line 4 5 | line 5 6 | line 6 7 | line 7 8 | line 8 9 | line 9 10 | Line 10 11 | line 11 12 | line 12 13 | line 13 14 | line 14 15 | Line 15 16 | Line 16 17 | Line 17 18 | Line 18 19 | Line 19 20 | Line 20 21 | line 21 22 | line 22 23 | line 23 24 | line 24 25 | line 25 26 | line 26 27 | line 27 28 | line 28 29 | line 29 30 | line 30 31 | -------------------------------------------------------------------------------- /piety/atimer_script.py: -------------------------------------------------------------------------------- 1 | """ 2 | atimer_script.py - Demonstrate the Python shell and timer tasks interleaving 3 | in the Piety event loop. 4 | 5 | atimer_script requires the running piety event loop, so it must be started 6 | from the Piety shell with the run function. See atimer_script.txt. 7 | """ 8 | 9 | from atimers import atimer 10 | piety.create_task(atimer(10,1,'A')) 11 | piety.create_task(atimer(20,0.5,'B')) 12 | 13 | -------------------------------------------------------------------------------- /piety/v2pmacs_script.py: -------------------------------------------------------------------------------- 1 | # v2pmacs_script.py 2 | # Run this from Piety directory: $ python3 -im v2pmacs_script 3 | from runner import run # run instead of import to put all identifiers __main__ 4 | run('editors/vpm.py') # load Piety editor with viewer panel 5 | run('piety/eventloop.py') # create piety keyboard event loop, don't start it 6 | run('piety/vpmacs_script.py') # create timer task and its editor buffer 7 | piety_start() # start keyboard event loop and timer task 8 | 9 | -------------------------------------------------------------------------------- /unix/terminal_util.py: -------------------------------------------------------------------------------- 1 | """ 2 | terminal_util.py - functions that work when stdin is not a terminal 3 | use even when piping input from a script 4 | """ 5 | 6 | import subprocess # just for display dimensions 7 | 8 | def dimensions(): 9 | 'Return nlines, ncols. Works on Mac OS X, probably other Unix.' 10 | nlines = int(subprocess.check_output(['tput','lines'])) 11 | ncols = int(subprocess.check_output(['tput','cols'])) 12 | return nlines, ncols 13 | -------------------------------------------------------------------------------- /threads/threads_3.py: -------------------------------------------------------------------------------- 1 | # threads_3.phy - explanation in threads_3.txt. To run: 2 | # 3 | # ...$ python3 -im tm 4 | # >> import threads_3 5 | 6 | from edsel import e, o2, on 7 | from pyshell import tpm 8 | from timers import Timer 9 | from writer import Writer 10 | import threading 11 | from threading import Thread 12 | threads = threading.enumerate 13 | 14 | o2() 15 | e('a.txt') 16 | ta = Timer() 17 | abuf = Writer('a.txt') 18 | Thread(target=ta.timer,args=(1000,1,'A',abuf)).start() 19 | on() 20 | tpm() 21 | 22 | -------------------------------------------------------------------------------- /piety/vedsel_script.md: -------------------------------------------------------------------------------- 1 | 2 | vedsel_script 3 | ============= 4 | 5 | *vedsel_script.py* demonstrates two timer tasks running in two editor windows, 6 | in the Piety desktop with the viewer window showing the instructions for 7 | the similar *edsel_script.py*. 8 | 9 | vedsel_script.py running in the Piety desktop 11 | 12 | To run this demo, follow the directions in the comment header in 13 | *vedsel_script.py* 14 | 15 | Revised Sep 2025 16 | 17 | -------------------------------------------------------------------------------- /editors/test/lines40.txt: -------------------------------------------------------------------------------- 1 | line 1 2 | line 2 3 | line 3 4 | line 4 5 | line 5 6 | line 6 7 | line 7 8 | line 8 9 | line 9 10 | Line 10 11 | line 11 12 | line 12 13 | line 13 14 | line 14 15 | Line 15 16 | Line 16 17 | Line 17 18 | Line 18 19 | Line 19 20 | Line 20 21 | line 21 22 | line 22 23 | line 23 24 | line 24 25 | line 25 26 | line 26 27 | line 27 28 | line 28 29 | line 29 30 | line 30 31 | line 31 32 | line 32 33 | line 33 34 | line 34 35 | line 35 36 | line 36 37 | line 37 38 | line 38 39 | line 39 40 | line 40 41 | -------------------------------------------------------------------------------- /python/runner.py: -------------------------------------------------------------------------------- 1 | """ 2 | runner.py - Define the function run, which runs a script in the __main__ namespace 3 | without creating a new module. An alternative to import, uses exec. 4 | Identifiers assigned in the script appear in the __main__ namespace. 5 | """ 6 | 7 | import sys 8 | 9 | main_globals = sys.modules['__main__'].__dict__ 10 | 11 | def run(filename): 12 | 'Run script in filename in __main__ namespace without creating a new module' 13 | exec(open(filename).read(), main_globals) 14 | 15 | -------------------------------------------------------------------------------- /python/README.md: -------------------------------------------------------------------------------- 1 | 2 | python 3 | ====== 4 | 5 | Python language utilities. 6 | 7 | ### Files ### 8 | 9 | - **pycall.py** - Callable Python interpreter using standard library 10 | *code.InteractiveConsole*. 11 | 12 | - **pyhelp.py** - Call Python interpreter help, but do not rewrite 13 | the screen, instead write help text to standard output, which 14 | can be redirected to an editor buffer. 15 | 16 | - **runner.py** - Define a *run* function that runs a script in the 17 | *__main__* namespace without creating a module. 18 | 19 | Revised Mar 2025 20 | 21 | -------------------------------------------------------------------------------- /bin/README.md: -------------------------------------------------------------------------------- 1 | 2 | bin 3 | === 4 | 5 | Shell scripts that configure Piety on Unix-like platforms. 6 | 7 | Put this directory on your execution *PATH*. Or, install its contents 8 | in some directory that is on your *PATH* (for details, see *paths* in 9 | this directory). 10 | 11 | To select a Piety configuration, execute one of the following 12 | scripts, or put the commands from the script into your *.profile* or 13 | *.bashrc*: 14 | 15 | - **paths**: assigns paths for running Piety on a Unix-like host with 16 | a VT-100 compatible terminal. 17 | 18 | Revised Feb 2023 19 | 20 | -------------------------------------------------------------------------------- /editors/dm.py: -------------------------------------------------------------------------------- 1 | # Start dmacs editor with browser from command line in any dir: python3 -im dm 2 | # First must define PYTHONPATH by . /Users/jon/piety/bin/paths, once in session 3 | import sked 4 | from sked import * 5 | import edsel 6 | from edsel import * 7 | import dmacs 8 | from dmacs import dm 9 | import console 10 | from console import * 11 | import urls 12 | from urls import * 13 | import get 14 | from get import * 15 | import render 16 | from render import * 17 | import search 18 | from search import * 19 | tl = dmacs.terminal.set_line_mode # type tl() to restore echo after crash 20 | win(22) 21 | dm() 22 | -------------------------------------------------------------------------------- /piety/pmacs_script.py: -------------------------------------------------------------------------------- 1 | # pmacs_script.py Edit in one window while timer task updates the other: 2 | # 3 | # ...$ python3 -m piety 4 | # >>>> run('pmacs_script.py') 5 | # ... windows appear, you can start typing in scratch.txt window ... 6 | 7 | import sked 8 | from sked import * 9 | import edsel 10 | from edsel import * 11 | from atimers import ATimer 12 | from writer import Writer 13 | from apmacs import apm 14 | 15 | win(22) 16 | o2() 17 | e('a.txt') 18 | ta = ATimer() 19 | abuf = Writer('a.txt') 20 | piety.create_task(ta.atimer(1000,1,'A',abuf)) 21 | on() # next window 22 | apm() # begin display editing 23 | 24 | -------------------------------------------------------------------------------- /piety/edsel_script.py: -------------------------------------------------------------------------------- 1 | """ 2 | edsel_script.py - Display interleaving timer tasks in two editor windows. 3 | You can set the timer intervals and stop the tasks from the Python REPL. 4 | 5 | edsel_script requires the running piety event loop, so it must be started 6 | from the Piety shell with the run function. See edsel_script.txt. 7 | """ 8 | 9 | from atimers import ATimer 10 | from writer import Writer 11 | import sked 12 | from sked import * 13 | import edsel 14 | from edsel import * 15 | win(22) 16 | o2() 17 | e('a.txt') 18 | abuf = Writer('a.txt') 19 | ta = ATimer() 20 | piety.create_task(ta.atimer(10,1,'A',abuf)) 21 | on() 22 | e('b.txt') 23 | bbuf = Writer('b.txt') 24 | tb = ATimer() 25 | piety.create_task(tb.atimer(20,0.5,'B',bbuf)) 26 | 27 | -------------------------------------------------------------------------------- /vt_terminal/README.md: -------------------------------------------------------------------------------- 1 | 2 | vt_terminal 3 | =========== 4 | 5 | This directory contains Python modules for running terminal 6 | applications on a VT-100 compatible terminal, whose keyboard sends 7 | ASCII control codes and ANSI control sequences, and whose display 8 | responds to ANSI control sequences. Most terminal emulator programs 9 | such as *xterm* are VT-100 compatible. 10 | 11 | To run terminal applications on a VT-100 compatible terminal, put this 12 | directory on the PYTHONPATH. To run on a different kind of terminal, 13 | put modules with the same module names and the same function names in 14 | a different directory with a different name (*framebuffer* for 15 | example) and put that directory on the PYTHONPATH instead. 16 | 17 | Revised February 2015 18 | -------------------------------------------------------------------------------- /piety/atimer_script.txt: -------------------------------------------------------------------------------- 1 | atimer_script.txt - Explanation and directions for atimer_script.py 2 | 3 | atimer_script.py demonstrates the Python shell and timer tasks interleaving 4 | in the piety event loop. 5 | 6 | atimer_script requires the running piety event loop, so it must be started 7 | from the Piety shell with the run function: 8 | 9 | $ python3 -m piety 10 | >>>> run('atimer_script.py') 11 | >>>> B 1 2024-07-03 10:13:03.997791 12 | A 1 2024-07-03 10:13:04.498137 13 | B 2 2024-07-03 10:13:04.498673 14 | B 3 2024-07-03 10:13:05.002520 15 | A 2 2024-07-03 10:13:05.500561 16 | ... 17 | ... When both timers finish, type RET to get the propmt again ... 18 | >>>> ^D 19 | $ 20 | ... You can type ^D or exit() to exit from the Piety shell and event loop. 21 | 22 | -------------------------------------------------------------------------------- /bin/paths: -------------------------------------------------------------------------------- 1 | # Assign paths to run Piety out of Piety/ development directories. 2 | # Must set both execution path for commands and Python path for modules. 3 | # If you install Piety, commands and modules may go to some standard place 4 | # and you will no longer need these assignments. 5 | 6 | # define $PIETY, this definition assumes you have ~/Piety 7 | export PIETY=$HOME/Piety 8 | 9 | # execution path to bin, to run Piety commands: piety, possibly others 10 | export PATH=$PIETY/bin:$PATH 11 | 12 | # Python path for Piety on a Unix-like host with a VT100-compatible terminal. 13 | # For other platforms, replace unix and vt_terminal in this path. 14 | export PYTHONPATH=$PIETY/viewer:$PIETY/browser:$PIETY/console:$PIETY/python:$PIETY/unix:$PIETY/vt_terminal:$PIETY/editors:$PIETY/tasking:$PIETY/threads:$PIETY/coroutines:$PIETY/piety:$PYTHONPATH 15 | -------------------------------------------------------------------------------- /coroutines/timer_loops.py: -------------------------------------------------------------------------------- 1 | """ 2 | timer_loops.py - Demonstrate interleaving timer tasks in short-lived asyncio event 3 | loops without a Python shell. See timer_loops.txt. 4 | """ 5 | 6 | import asyncio as aio 7 | loop = aio.get_event_loop() 8 | print(loop) 9 | print(aio.all_tasks(loop)) 10 | 11 | from atimers import atimer 12 | ta = loop.create_task(atimer(5,5,'A')) 13 | print(ta) 14 | print(aio.all_tasks(loop)) 15 | 16 | loop.run_until_complete(ta) 17 | print(aio.all_tasks(loop)) 18 | print() 19 | 20 | # Set up tasks to interleave 21 | 22 | ta = loop.create_task(atimer(5,1,'A')) 23 | tb = loop.create_task(atimer(10,0.5,'B')) 24 | print(aio.all_tasks(loop)) 25 | 26 | loop.run_until_complete(ta) 27 | print(aio.all_tasks(loop)) 28 | 29 | # The other task hasn't completed yet. 30 | 31 | loop.run_until_complete(tb) 32 | print(aio.all_tasks(loop)) 33 | 34 | -------------------------------------------------------------------------------- /editors/pm.py: -------------------------------------------------------------------------------- 1 | # Start pmacs editor with browser from command line in any dir: python3 -im pm 2 | # First must define PYTHONPATH by . /home/jon/piety/bin/paths, once in session 3 | import sked 4 | from sked import * 5 | import edsel 6 | from edsel import * 7 | import dmacs 8 | from dmacs import dm # so we can revert to dmacs if pmacs is broken 9 | import console 10 | from console import * 11 | import editline 12 | import pmacs 13 | from pmacs import pm 14 | import urls 15 | from urls import * 16 | import get 17 | from get import * 18 | import render 19 | from render import * 20 | import search 21 | from search import * 22 | import viewer 23 | from viewer import * 24 | tl = pmacs.terminal.set_line_mode # type tl() to restore echo after crash 25 | from terminal_util import dimensions 26 | tlines, tcols = dimensions() 27 | win(tlines-8) # 8 lines in prompt + repl region 28 | pm() 29 | -------------------------------------------------------------------------------- /editors/vpm.py: -------------------------------------------------------------------------------- 1 | # Start pmacs editor full screen with viewer panel in any dir: python3 -im vpm 2 | # First must define PYTHONPATH by . /home/jon/piety/bin/paths, once in session 3 | import sked 4 | from sked import * 5 | import edsel 6 | from edsel import * 7 | import dmacs 8 | from dmacs import dm # so we can revert to dmacs if pmacs is broken 9 | import console 10 | from console import * 11 | import editline 12 | import pmacs 13 | from pmacs import pm 14 | import urls 15 | from urls import * 16 | import get 17 | from get import * 18 | import render 19 | from render import * 20 | import search 21 | from search import * 22 | import viewer 23 | from viewer import * 24 | tl = pmacs.terminal.set_line_mode # type tl() to restore echo after crash 25 | from terminal_util import dimensions 26 | tlines, tcols = dimensions() 27 | win(tlines-8) # 8 lines in prompt + repl region 28 | vwin() 29 | N() 30 | oe() 31 | #pm() 32 | -------------------------------------------------------------------------------- /threads/threads_2.py: -------------------------------------------------------------------------------- 1 | # Code for demo described in threads_2.txt. 2 | # To run this code, first at the system command line: 3 | # ...$ . ~/Piety/bin/paths 4 | # ...$ python3 -im tm 5 | # Then a window and the pysh prompt >> appears. At the psyh prompt: 6 | # >> import threads_2 7 | # Then a.txt and b.txt windows appear, with timer messages from each thread 8 | # >> from threads_2 import * 9 | # Now you can type commands described in threads_2.txt to control timers etc. 10 | # >> threads() 11 | # >> ta.delay = 0.5 12 | # ... etc. ... 13 | 14 | from edsel import e, o2 15 | from timers import Timer 16 | from writer import Writer 17 | import threading 18 | from threading import Thread 19 | threads = threading.enumerate 20 | 21 | e('a.txt') 22 | o2() 23 | e('b.txt') 24 | 25 | ta = Timer() 26 | tb = Timer() 27 | 28 | abuf = Writer('a.txt') 29 | bbuf = Writer('b.txt') 30 | 31 | Thread(target=tb.timer,args=(1000,1,'B',bbuf)).start() 32 | Thread(target=ta.timer,args=(1000,1,'A',abuf)).start() 33 | 34 | -------------------------------------------------------------------------------- /editors/bugs.md: -------------------------------------------------------------------------------- 1 | 2 | bugs 3 | ==== 4 | 5 | Unfixed bugs in the editors. At this time (Nov 2024) we know of two: 6 | 7 | - **frame-scrolls-up**: Sometimes when an editor writes a message 8 | in the srolling REPL region, the entire frame scrolls up, as if 9 | the scrolilng region were the whole terminal window, not just 10 | the REPL lines at the bottom. 11 | 12 | The workaround for this is to invoke *refresh_all* by typing *M-l* 13 | (hold the *alt* key while typing the *L* key). This restores all the 14 | frame contents to their correct locations and resets the scrolling region. 15 | 16 | - **junk-in-yank**: Sometimes invoking *yank* by typing *C-y* (hold the 17 | *ctrl* key while typing the *Y* key) pastes in some additional text 18 | from an earlier *kill-line* *C-k* command, in addition to the intended 19 | text from the immediately preceding *cut* or *kill-line*. 20 | 21 | The workaround for this is to simply delete the unwanted text. 22 | 23 | Revised Nov 2024 24 | 25 | 26 | -------------------------------------------------------------------------------- /coroutines/term_timer.txt: -------------------------------------------------------------------------------- 1 | 2 | term_timer.txt - Explanation and directions for term_timer.py. 3 | 4 | term_timer.py demonstrates interleaving of aterminal reader and an atimer task 5 | in an asyncio event loop. 6 | 7 | To run the demonstration, follow these directions: 8 | 9 | ...$ python3 -im term_timer 10 | > 11 | 12 | Then at the > prompt, type abc. Do not type (RETURN or ENTER). When 13 | 5 seconds have passed, the first timer message appears: 14 | 15 | ...$ python3 -im term_timer 16 | > abcA 1 2024-06-19 17:07:35.823368 17 | 18 | Then on the next line, type def and wait for next timer message. And so on 19 | until all 5 timer messages have appeared. Then type pqr. The program 20 | prints all the characters you typed: 21 | 22 | ...$ python3 -im term_timer 23 | > abcA 1 2024-06-19 17:07:35.823368 24 | defA 2 2024-06-19 17:07:40.827853 25 | ghiA 3 2024-06-19 17:07:45.834841 26 | jklA 4 2024-06-19 17:07:50.840380 27 | mnoA 5 2024-06-19 17:07:55.845229 28 | pqr 29 | abcdefghijklmnopqr 30 | >>> ^D 31 | ...$ 32 | 33 | -------------------------------------------------------------------------------- /unix/README.md: -------------------------------------------------------------------------------- 1 | 2 | unix 3 | ==== 4 | 5 | This directory contains Python modules for running Piety on a 6 | Unix-like host operating system (including Linux and Mac OS X). 7 | 8 | To run Piety on a Unix-like host operating system, put this directory 9 | on the PYTHONPATH. To run on a different kind of host, put modules 10 | with the same module names and function names in a different directory 11 | with a different name (*windows* for example) and put that directory 12 | on the PYTHONPATH instead. 13 | 14 | ### Files #### 15 | 16 | - **shell.py** - Python functions that wrap shell commands, so you can invoke 17 | the shell without exiting the Python session or using 18 | the host desktop. 19 | 20 | - **terminal.py** - functions to set terminal character mode or line mode, 21 | read/write a single character or a string. 22 | 23 | - **terminal_util.py** - function to get terminal dimensions: number of 24 | lines, columns. 25 | 26 | Revised Mar 2025 27 | 28 | -------------------------------------------------------------------------------- /python/pyhelp.py: -------------------------------------------------------------------------------- 1 | """ 2 | pyhelp.py - The help function here calls the Python interpreter help. But 3 | unlike builtin Python help, it does not rewrite the screen, it just writes 4 | the help text to the standard output, which can be redirected anywhere. 5 | """ 6 | 7 | import pydoc 8 | 9 | # Alias for builtin Python help so we can still use it after we define our help. 10 | # phelp = help # FIXME wait on this 11 | 12 | def help(topic): 13 | """ 14 | Print help on topic, a Python object: module, function etc. Prints to stdout. 15 | Uses the pydoc module render_doc function, does not invoke a shell process. 16 | Unlike builtin Python help, this function does not rewrite the screen, 17 | it just writes the help text to the standard output. 18 | To use builtin Python help, use the alias phelp defined above. 19 | FIXME: Unlike builtin Python help, this fcn does not accept string arguments. 20 | """ 21 | helptext = pydoc.render_doc(topic, "Help on %s") # one big string 22 | for line in helptext.splitlines(): 23 | print(line) 24 | 25 | -------------------------------------------------------------------------------- /coroutines/README.md: -------------------------------------------------------------------------------- 1 | 2 | coroutines 3 | ========== 4 | 5 | Experiments with tasks using Python coroutines and the *asyncio* event loop. 6 | 7 | All of these scripts assume you have already assigned *PYTHONPATH* by running 8 | this command: 9 | 10 | ...$ . ~/Piety/bin/paths 11 | 12 | The initial dot . in this command is essential. This command assumes 13 | the top level *Piety* directory is in your home directory. 14 | 15 | ### Files ### 16 | 17 | - **aterminal.py**: Read characters from the terminal in the *asyncio* event loop. 18 | 19 | - **atimers.py**: Coroutines that print timestamps at intervals, 20 | to run in tasking experiments. 21 | 22 | - **term_timer.py**: Demonstate interleaving of *aterminal* reader and 23 | an *atimer* task in an *asyncio* event loop. 24 | 25 | - **term_timer.txt**: Explanation and directinos for *term_timer.py*. 26 | 27 | - **timer_loops.py** - Demonstrate interleaving timer tasks in short-lived *asyncio* 28 | event loops without a Python shell. 29 | 30 | - **timer_loops.txt**: Explanation and directions for *timer_loops.py*. 31 | 32 | Revised Jul 2024 33 | 34 | -------------------------------------------------------------------------------- /python/pycall.py: -------------------------------------------------------------------------------- 1 | """ 2 | pycall.py - Callable Python interpreter using standard library code module. 3 | """ 4 | 5 | import sys, code 6 | 7 | main_globals = sys.modules['__main__'].__dict__ 8 | interpreter = code.InteractiveConsole(locals=main_globals) 9 | 10 | def pycall(cmd): 11 | """ 12 | Push cmd, a line of text, to Python code.InteractiveConsole interpreter 13 | Return True if continuation line expected, False otherwise. 14 | """ 15 | return interpreter.push(cmd) 16 | 17 | def main(): 18 | """ 19 | Home-made Python REPL. Type exit() to exit. 20 | """ 21 | ps1 = '>> ' # first line prompt, different from CPython >>> 22 | ps2 = '.. ' # continuation line prompt 23 | continuation = False # True when continuation line expected 24 | 25 | while True: 26 | prompt = ps2 if continuation else ps1 27 | cmd = input(prompt) # python -i makes readline editing work here. 28 | if cmd == 'exit()': # Trap here, do *not* exit calling Python session. 29 | break 30 | continuation = pycall(cmd) 31 | 32 | if __name__ == '__main__': 33 | main() 34 | -------------------------------------------------------------------------------- /piety/vpmacs_script.py: -------------------------------------------------------------------------------- 1 | # MODIFIED FROM pmacs_script.py - this version assumes vpm.py has already run 2 | # vpmacs_script.py Edit in one window while timer task updates the other: 3 | # 4 | # ...$ cd Piety/piety # so run(...) works without directory prefix 5 | # ...$ python3 -im vpm 6 | # >>> from runner import run 7 | # >>> run('piety.py') 8 | # Now event loop should be running, with async shell indicated by 4 >>>> 9 | # >>>> piety 10 | # ... EventLoop running=True ... 11 | # >>>> run('vpmacs_script.py') 12 | # ... windows appear, you can start typing in scratch.txt window ... 13 | 14 | # Comment out redundant imports, vpm already imported these 15 | #import sked 16 | #from sked import * 17 | #import edsel 18 | #from edsel import * 19 | 20 | # Imports needed for timer demo 21 | from atimers import ATimer 22 | from writer import Writer 23 | from apmacs import apm 24 | 25 | # vpm already called win() 26 | #win(22) 27 | 28 | # Timer demo 29 | o2() 30 | e('a.txt') 31 | ta = ATimer() 32 | abuf = Writer('a.txt') 33 | ta_task = piety.create_task(ta.atimer(1000,1,'A',abuf)) 34 | on() # next window 35 | # apm() # begin display editing # DEBUG - type this at command line. 36 | 37 | -------------------------------------------------------------------------------- /threads/tm.py: -------------------------------------------------------------------------------- 1 | # Start pmacs editor with pyshell from command line in any dir: python3 -im tm 2 | # Like pm script but also imports writer, timers, pyshell for tasking expts. 3 | # When script exits you see pmacs window and pysh >> (not >>>) REPL prompt. 4 | # Type tpm() to begin display editing, M-x to return to >>, exit() when done 5 | # First must define PYTHONPATH by . /Users/jon/piety/bin/paths, once in session 6 | import sked 7 | from sked import * 8 | import edsel 9 | from edsel import * 10 | import dmacs 11 | from dmacs import dm # so we can revert to dmacs if pmacs is broken 12 | import editline 13 | import pmacs 14 | from pmacs import pm, rpm # use rpm when starting from pysh instead of >>> 15 | import pyshell 16 | from pyshell import pysh, tpm # from pysh, use tpm not rpm to clear cmd_mode 17 | import writer # for tasks 18 | from writer import * 19 | import timers # for tasks 20 | from timers import * 21 | from contextlib import redirect_stdout # for tasks 22 | import threading 23 | from threading import Thread 24 | threads = threading.enumerate # NOT from threading import enumerate. Name clash! 25 | tl = pmacs.terminal.set_line_mode # type tl() to restore echo after crash 26 | win(22) 27 | pysh() # Now at pysh >> prompt type tpm() for display editing, exit() when done 28 | 29 | 30 | -------------------------------------------------------------------------------- /piety/vedsel_script.py: -------------------------------------------------------------------------------- 1 | # vedsel_script.py - based on edsel_script.py but run after vpm script 2 | # 3 | # ...$ cd Piety/piety # so run(...) works without directory prefix 4 | # ...$ python3 -im vpm 5 | # >>> from runner import run 6 | # >>> run('piety.py') 7 | # Now event loop should be running, with async shell indicated by 4 >>>> 8 | # >>>> piety 9 | # ... EventLoop running=True ... 10 | # >>>> run('vedsel_script.py') 11 | # ... windows appear, you can start typing in scratch.txt window ... 12 | """ 13 | vedsel_script.py - Display interleaving timer tasks in two editor windows. 14 | You can set the timer intervals and stop the tasks from the Python REPL. 15 | 16 | edsel_script requires the running piety event loop, so it must be started 17 | from the Piety shell with the run function. See edsel_script.txt. 18 | """ 19 | 20 | from atimers import ATimer 21 | from writer import Writer 22 | 23 | # Comment out redundant imports, alrady done by vpm script 24 | #import sked 25 | #from sked import * 26 | #import edsel 27 | #from edsel import * 28 | 29 | # Comment out, vpm already called win(...) 30 | # win(22) 31 | 32 | o2() 33 | e('a.txt') 34 | abuf = Writer('a.txt') 35 | ta = ATimer() 36 | ta_task = piety.create_task(ta.atimer(10,1,'A',abuf)) 37 | on() 38 | e('b.txt') 39 | bbuf = Writer('b.txt') 40 | tb = ATimer() 41 | tb_task = piety.create_task(tb.atimer(20,0.5,'B',bbuf)) 42 | 43 | -------------------------------------------------------------------------------- /piety/apmacs.py: -------------------------------------------------------------------------------- 1 | """ 2 | apmacs.py - Adapt our pmacs Emacs-like editor to run in an asyncio event loop. 3 | Define the asyncio reader function apmrun that handles each editor 4 | keystroke. Define the function apm to resume the pmacs editor 5 | from the Piety shell command prompt (after first running the apm.py 6 | script to load the editor modules and create the initial window). 7 | """ 8 | 9 | import terminal, key, display, pmacs, pyshell 10 | 11 | def apm(): 12 | """ 13 | apm() calls the pmacs editor from the piety shell prompt in the asycio event loop. 14 | After this the editor responds to emacs keys. Type M-x to return to the shell. 15 | """ 16 | pyshell.cmd_mode = False 17 | pmacs.setup() 18 | 19 | def apmrun(): 20 | """ 21 | Run each time sys.stdin detects a new character 22 | Call pymacs runcmd, check for exit 23 | """ 24 | c = terminal.getchar() # not blocking, asyncio calls apmrun when char is ready 25 | pmacs.runcmd(c) # assigns running = False to exit 26 | if not pmacs.running: 27 | pyshell.cmd_mode = True 28 | pmacs.restore() # calls restore_cursor_to_cmdline 29 | display.next_line() # these 3 lines copied from pyshell.runcmd key.cr case 30 | pyshell.cmd = '' 31 | pyshell.point = 0 32 | pyshell.setup() # prints prompt and refreshes command line 33 | 34 | 35 | -------------------------------------------------------------------------------- /piety/piety.py: -------------------------------------------------------------------------------- 1 | """ 2 | piety.py - Starts a Piety session. Creates an event loop named 3 | *piety*, adds the readers for the shell and the editor, and starts the 4 | event loop with the shell running. It also defines a funtion *run* 5 | which is needed to run other scripts in the event loop. 6 | 7 | NOTE: This module has been superceded by eventloop.py. 8 | We are keeping it here because it appears in some older scripts and 9 | documentation. 10 | """ 11 | 12 | import sys, asyncio 13 | import pyshell, apyshell, apmacs 14 | 15 | # So we can run scripts that name piety from the apysh >>>> prompt 16 | # Identifiers assigned in the script remain in the session. 17 | from runner import run 18 | 19 | # We could swap in in other shells and foreground jobs if we had them. 20 | # Hard code these for now because we have no others - maybe more in the future. 21 | shell = apyshell.apysh 22 | fgjob = apmacs.apmrun # foreground job 23 | 24 | def handler(): 25 | if pyshell.cmd_mode: 26 | shell() 27 | else: 28 | fgjob() 29 | 30 | apyshell.running = True 31 | apyshell.setup() # print >>>> prompt, etc. 32 | 33 | piety = asyncio.get_event_loop() 34 | piety.add_reader(sys.stdin, handler) 35 | piety.run_forever() 36 | 37 | # Statements in this script that follow run_forever() are not executed. 38 | # For example these are NOT executed: 39 | # from atimers import atimer 40 | # piety.create_task(atimer(5,1)) 41 | 42 | -------------------------------------------------------------------------------- /doc/platforms.md: -------------------------------------------------------------------------------- 1 | 2 | Tested platforms 3 | ================ 4 | 5 | Through November 2023, the Piety software was only 6 | run on one computer: a MacBook Pro (13 inch, early 2011), running Mac OS 7 | (through 10.11.6 El Capitan, the most recent version that runs on that 8 | hardware). It ran in the Mac OS Terminal, through version 2.6.2 (361.2). It 9 | ran on CPython downloaded from python.org, through version 3.9.0. 10 | 11 | Beginning in December 2023, Piety development moved to Linux running in 12 | a virtual machine on a Chromebook, a Lenovo Ideapad 3 Chrome 14M836 13 | purchased in April 2023. 14 | 15 | In Dec 2023, the Chromebook is running ChromeOs version 118.0.5993.164. 16 | The *uname -a* command says it is running *Linux penguin 5.15*, which I 17 | believe is derived from Debian. This is the Linux provided to ordinary 18 | users as part of the standard Chromebook software, it is not part of 19 | some 'developer mode'. The Python running in this Linux is CPython 20 | version 3.9.2. Piety runs in the Chromebook Linux Terminal app. 21 | 22 | The Chromebook software has updated itself several times in the last 23 | year. At this writing (12 Nov 2024) it is running ChromeOS Version 24 | 129.0.6668.110, Linux Penguin 6.6.46, and Python 3.11.2. 25 | 26 | Beginning in October 2024, Piety development also continued on an HP 27 | laptop model 14-dq0057nr. It is running Debian Linux 12.7.0 ('bookworm') 28 | on Linux kernel version 6.1.112-1 with Python 3.11.2. 29 | 30 | Revised Nov 2024 31 | -------------------------------------------------------------------------------- /tasking/README.md: -------------------------------------------------------------------------------- 1 | 2 | tasking 3 | ======= 4 | 5 | Modules that support tasking experiments with threads or coroutines 6 | and our editors. 7 | 8 | The editors are not just for creating text. Python commands including 9 | concurrent tasks can use the *writer* module here to redirect their output to 10 | editor buffers and windows, so the editors can be used for data capture and 11 | animated display. 12 | 13 | We need our custom *pysh* Python interpreter, defined in the *pyshell* 14 | module here, for these tasking experiments. It enables us to restore the 15 | cursor to the correct location in the Python command line after a background 16 | task updates an editor display window. This is not possible with the 17 | standard Python interpreter. 18 | 19 | The *pysh* command prompt is >> with just two darts, to distinguish it from 20 | the standard Python prompt >>> with three darts. 21 | 22 | ### Files ### 23 | 24 | - **pyshell.py**: Defines custom Python interpreter *pysh* that 25 | enables us to restore the cursor to the correct location in the 26 | Python command line after a background task updates an editor 27 | display window. 28 | 29 | - **writer.py**: Functions that put text into our editor buffers and windows, 30 | intended to be called from background tasks. Code here also restores 31 | the cursor to the correct location in the *pysh* Python 32 | command line, or in a display editing window, after a background task 33 | updates another window. 34 | 35 | - **writer.txt**: Notes on *writer.py*. 36 | 37 | Revised Jun 2024 38 | 39 | -------------------------------------------------------------------------------- /browser/search.py: -------------------------------------------------------------------------------- 1 | """ 2 | search.py - Search web sites by sending query URLs 3 | 4 | To search Hacker News: Sample query, list stories that match the query 5 | 'STEPS reinventing programming', ordered by date: 6 | 7 | https://hn.algolia.com/?dateRange=all&page=0&prefix=false&query=STEPS%20reinventing&sort=byDate&type=story 8 | 9 | The query part there is: 10 | 11 | &query=STEPS%20reinventing%20programming&sort=byDate&type=comment 12 | 13 | Never mind, it doesn't work. Sending the query just results in 14 | hn.algolia.com sending back a page of Javascript, which your 15 | browser has to run to actually execute the query. That page contains: 16 | 17 | 18 | 19 | Bah! The Piety browser will never support Javascript. 20 | """ 21 | 22 | import get 23 | 24 | hnprefix = 'https://hn.algolia.com/?dateRange=all&page=0&prefix=false&query=' 25 | 26 | def hnsearch(type, query): 27 | """ 28 | Search Hacker News by sending query to hn.algolia.com. 29 | type is 'story', 'comment', or 'user' 30 | query is a string of space-separtaed keywords: 'STEPS reninvention' 31 | BUT the server just returns a page of Javascript the browser must execute! 32 | """ 33 | hnquery = '%20'.join(query.split()) 34 | hntype = f'&sort=byDate&type={type}' 35 | get.gr(hnprefix + hnquery + hntype) 36 | 37 | def hnuser(user): 38 | """ 39 | Retrieve all comments posted by user, most recent first. 40 | Does *not* send query to Algolia or require Javascript - so it works! 41 | """ 42 | get.gr(f'https://news.ycombinator.com/threads?id={user}') 43 | 44 | def hnitem(id): 45 | 'Get the HN item page with integer (not string) id number' 46 | get.gr(f'https://news.ycombinator.com/item?id={id}') 47 | 48 | -------------------------------------------------------------------------------- /coroutines/aterminal.py: -------------------------------------------------------------------------------- 1 | """ 2 | aterminal.py - Read characters from the terminal in the asyncio event loop. 3 | 4 | ...$ python3 aterminal.py # To run once. Type some characters, then RET (or Enter) 5 | > abcdef 6 | abcdef 7 | ...$ 8 | 9 | ...$ python3 -i # To run multiple times. main() does not call loop.close(). 10 | >>> import aterminal 11 | >>> aterminal.main() 12 | > abcdef 13 | abcdef 14 | >>> aterminal.main() 15 | > ghijkl 16 | ghijkl 17 | >>> 18 | 19 | """ 20 | 21 | import sys, asyncio 22 | import terminal as term 23 | 24 | tl = term.set_line_mode # Type tl() to restore terminal echo etc. after a crash. 25 | 26 | loop = asyncio.get_event_loop() 27 | 28 | line = '' 29 | 30 | def setup(): 31 | 'Initialize empty line, set terminal to raw single char mode, print prompt.' 32 | global line 33 | line = '' 34 | term.set_char_mode() 35 | term.putstr('> ') # We want to see this *before* the first character is typed. 36 | 37 | def cleanup(): 38 | 'Stop the event loop, but dont close it. Restore term line mode, print line.' 39 | loop.stop() # escape from run_forever() 40 | term.set_line_mode() 41 | print() # must advance to next line 42 | print(line) 43 | 44 | def runcmd(c): 45 | 'Add single character to line. Test for exit char - if found, call cleanup.' 46 | global line 47 | term.putstr(c) 48 | line += c 49 | if c in '\n\r': 50 | cleanup() 51 | 52 | def handler(): 53 | 'Run each time sys.stdin detects a new character. Read the char, call runcmd.' 54 | c = term.getchar() 55 | runcmd(c) 56 | 57 | loop.add_reader(sys.stdin, handler) # FIXME? Use tty not stdin, as in display.py? 58 | 59 | def main(): 60 | 'Call setup and start the event loop. Event loop was created at module level.' 61 | setup() 62 | loop.run_forever() 63 | 64 | if __name__ == '__main__': 65 | main() 66 | loop.close() 67 | 68 | -------------------------------------------------------------------------------- /console/README.md: -------------------------------------------------------------------------------- 1 | 2 | Console 3 | ======== 4 | 5 | Modules that define commands to enable Piety development and other 6 | personal computing activities using only Python running Piety in the 7 | console terminal, without depending on a host OS to provide a desktop 8 | with multiple windows, the system shell, and other utilities. 9 | 10 | [Files](#Files) 11 | [Commands](#Commands) 12 | 13 | ### Files ### 14 | 15 | - **console.py**: Python wrappers for the host shell, some particular shell 16 | commands, and Python help, that redirect command output to 17 | editor buffers. 18 | 19 | - **redirect.py**: Redirect command output to editor buffers, 20 | so the buffers can act much like terminal windows with 21 | scroll back. 22 | 23 | ### Commands ### 24 | 25 | The *console* module provides a function *sh*, which runs any 26 | shell command string, and functions for particular shell commands: 27 | *pwd*, *cd*, *ls*, *lsl*, *lslt*, and *man*. It also provides 28 | a *help* function. 29 | 30 | The *cd* and *ls* functions can take an optional argument, a path string 31 | (usually a directory). The *ls*, *lsl* and *lslt* functions call the shell 32 | commands *ls -C, ls -l, ls -lt*, respectively. The *man* function takes 33 | the topic string as an argument, for example *man('ls')*. The *help* 34 | function takes a Python object (not a string) as an argument, for 35 | example *help(str)*. 36 | 37 | The *pwd*, *cd*, and *help* functions call specific functions in the Python 38 | standard library. The *sh*, *ls*, and *man*, functions run the host shell 39 | in a subprocess. 40 | 41 | The *sh*, *pwd*, *cd*, and *ls* commands all append their command string 42 | and command output to the single \*Console\* buffer. Each call to *man* 43 | and *help* creates a new buffer *topic.man* or *topic.help* that holds 44 | the manual text or help text on just that topic. 45 | 46 | Revised Apr 2025 47 | 48 | -------------------------------------------------------------------------------- /doc/modules.md: -------------------------------------------------------------------------------- 1 | 2 | Modular structure 3 | ================= 4 | 5 | The Piety directories and modules are designed to enable Piety to run on 6 | different platforms and in different configurations. 7 | 8 | A *platform* is a host operating system or a bare machine (including 9 | virtual machines), including the Python interpreter itself (CPython, 10 | PyPy, Micro Python, ...). A *configuration* is a collection of 11 | available devices, possibly including a console terminal, but also 12 | allowing for "headless" configurations with no console (as are 13 | sometimes used for embedded systems). 14 | 15 | It is possible to customize Piety systems by choosing different 16 | subsets and combinations of modules. To adapt to different 17 | platforms and configurations, you can create modules with the 18 | same name and same API (but different internals) stored in 19 | different directories. 20 | 21 | For example, we have the directory *unix* containing the module 22 | *terminal.py*. To support a different platform, we might add a directory 23 | *windows* that also contains a module *terminal.py* with the same 24 | function names and arguments but different function bodies. 25 | 26 | The *unix* and *windows* directories and their contents are platform dependent. 27 | But modules that import the *terminal* module and call its functions 28 | are platform independent. Plaatform dependent and platform independent 29 | modules must be kept in different directores so Piety can be configured 30 | for different platforms by defining different *PYTONPATH* 31 | 32 | For each platform, the platform dependent modules are included by adding 33 | their platform dependent directories to the *PYTHONPATH* and excluding the 34 | alternate platform dependent directories. To help with this, there are 35 | commands in the *bin* directory. 36 | 37 | Different configurations (that support different collections of hardware etc.) 38 | can be supported in the same way as different platforms. 39 | 40 | Revised May 2024 41 | 42 | 43 | -------------------------------------------------------------------------------- /editors/breakpt.py: -------------------------------------------------------------------------------- 1 | """ 2 | breakpt.py 3 | 4 | Enable debugging display editors with Pdb while they are running. 5 | 6 | Defines our breakpt function and assigns it to sys.breakpointhook. 7 | After you import this breakpt module, calls to the Python builtin function 8 | breakpoint will call our breakpt function. 9 | 10 | Our breakpt function sets the terminal to line mode, moves the cursor from 11 | the display window to the scrolling REPL region, then calls pdb.set_trace 12 | so you can use Pdb in the REPL region while window contents remain undisturbed 13 | on the display. When you are done with the debugger, type the Pdb c (continue) 14 | command. Then breakpt() returns the terminal to character mode and puts the 15 | cursor at dot in the display window, so you can resume using the display 16 | editor. 17 | 18 | In general, the cursor is not at dot when breakpt() is invoked, so when 19 | breakpt() returns it often does not return the cursor to the right place, 20 | and subsequent code does not have the intended effect. The code does not 21 | record the location of the cursor, so this is the best we can do. The 22 | workaround it to invoke refresh(), for example by typing C-l, right after 23 | returning from breakpt() with Pdb c. This puts the correct contents on 24 | the display and restores the cursor to the correct position. 25 | 26 | This module and its function are named breakpt to avoid a name clash with 27 | the builtin function breakpoint. 28 | """ 29 | 30 | import pdb, sys 31 | import terminal, display 32 | import sked as ed 33 | import edsel as fr 34 | 35 | def breakpt(): 36 | """ 37 | Run pdb in scrolling REPL region to preserve window contents. 38 | Use as breakpoint hook. 39 | """ 40 | x = 42 # local variable to examine with Pdb p 41 | terminal.set_line_mode() # Prepare for using debugger at breakpoint 42 | display.put_cursor(fr.tlines, 1) # Put cursor in REPL region for Pdb commands 43 | pdb.set_trace() # Enter Pdb debugger, use Pdb commands until Pdb c (continue) 44 | terminal.set_char_mode() # Restore terminal state 45 | display.put_cursor(fr.wline(ed.dot), ed.point + 1) # Restore cursor to window 46 | 47 | sys.breakpointhook = breakpt 48 | -------------------------------------------------------------------------------- /console/redirect.py: -------------------------------------------------------------------------------- 1 | """ 2 | redirect.py - The redirect() function here redirects command output to 3 | editor buffers, so the buffers can act much like terminal windows 4 | with scroll back. 5 | """ 6 | 7 | from contextlib import redirect_stdout 8 | 9 | # sked has module-level write function that writes to the current buffer. 10 | import sked as ed 11 | import edsel as fr # fr for frame 12 | 13 | def redirect(bufname, command, command_string): 14 | """ 15 | Redirect stdout from command to the editor buffer named bufname. 16 | command must be a callable without arguments that writes to stdout. 17 | command_string is the string that labels the command output in the buffer. 18 | If the bufname buffer does not exist, create it and make it current. 19 | If bufname already exists, make it the current buffer. 20 | """ 21 | # bufnames are the names of the buffers currently displayed in windows 22 | bufnames = { fr.windows[k]['bufname'] for k in fr.windows } 23 | # bufname buffer does not yet exist 24 | if not bufname in ed.buffers: 25 | fr.e(bufname) # create bufname buffer in the current window 26 | # bufname buffer exists but is not in any windows: 27 | elif bufname in ed.buffers and not bufname in bufnames: 28 | ed.b(bufname) # make bufname the current buffer in the current window 29 | # bufname buffer exists and is in a window but is not the current buffer 30 | elif (bufname in ed.buffers and bufname in bufnames 31 | and not bufname == ed.bufname): 32 | fr.on() # switch to other window - only works when there are just two 33 | # bufname buffer exists and is in a windows and is the current buffer. 34 | elif (bufname in ed.buffers and bufname in bufnames 35 | and bufname == ed.bufname): 36 | pass # we don't have to select buffer or switch window. 37 | else: # we never get here - all possibilities already covered 38 | pass # bufname is already the current buffer 39 | with redirect_stdout(ed): # output goes to write fcn in sked module 40 | print('>>> ' + command_string) # print command to label its output 41 | command() 42 | fr.scroll() # last line printed by command is at bottom of window 43 | -------------------------------------------------------------------------------- /piety/edsel_script.txt: -------------------------------------------------------------------------------- 1 | edsel_script.txt - Explanation and directions for edsel_script.py 2 | 3 | edsel_script.py displays interleaving timer tasks in two editor windows. 4 | Tou can set the timer intervals and stop the tasks from the Python REPL. 5 | 6 | edsel_script requires the running piety event loop, so it must be started 7 | from the Piety shell with the run function: 8 | 9 | $ python3 -m piety 10 | >>>> run('edsel_script.py') 11 | ... Two windows appear, 10 timer messages appear in a.txt, 20 in b.txt. ... 12 | ... 13 | ... After all the timer messages appear, you can restart the tasks, 14 | ... st the timer intervals, and stop the tasks from the Python REPL: 15 | >>>> piety.create_task(ta.atimer(120,1,'A',abuf)) 16 | > 17 | >>>> ta.delay 18 | 1 19 | >>>> ta.delay = 0.5 20 | ... Messages in a.txt appear twice as fast ... 21 | >>>> piety.create_task(tb.atimer(120,1,'B',bbuf)) 22 | > 23 | >>>> tb.delay = 0.1 24 | ... Messages in b.txt appear 10x as fast ... 25 | ... How fast can you the messages appear before the windows ... 26 | ... and command line become scrambled? ... 27 | ... When a.txt window stops updating: ... 28 | >>>> piety.create_task(ta.atimer(120,1,'A',abuf)) 29 | > 30 | ... Wait for a few new messages in a.txt ... 31 | >>> asyncio.all_tasks(piety) 32 | { wait_for=>, wait_for=>} 37 | >>>> ta.run 38 | True 39 | >>>> ta.run = False 40 | ... Messages stop appearing in a.txt ... 41 | >>> asyncio.all_tasks(piety) 42 | set() 43 | >>>> clr() 44 | ... restores full screen scrolling so windows will scroll away ... 45 | >>>> ^D 46 | $ 47 | ... You can type ^D or exit() to exit from the Piety shell and event loop. 48 | 49 | -------------------------------------------------------------------------------- /coroutines/atimers.py: -------------------------------------------------------------------------------- 1 | """ 2 | atimers.py - Coroutine that prints timestamps at intervals, 3 | to run in tasking experiments. 4 | 5 | This module is named atimers (with an s) but the coroutine is named atimer 6 | (no s) so we can 'from atimers import atimer' without name conflict. 7 | """ 8 | 9 | import sys, asyncio, datetime 10 | 11 | # Based on timer from timers.py 12 | async def atimer(n=1, delay=1.0, label='', destination=sys.stdout): 13 | """ 14 | Sleep for given delay (default 1.0 sec), then print timestamp message. 15 | Repeat n times (default 1). Message includes time and optional label. 16 | Optional label for distinguishing output from different function calls. 17 | Calls print() to print output to stdout, which can be redirected. 18 | Calls print('...\n\r', end='') so it works in terminal char and line modes, 19 | """ 20 | for i in range(n): 21 | await asyncio.sleep(delay) 22 | # For now use print with default args, which adds the \n itself. 23 | print(f'{label} {i+1} {datetime.datetime.now()}\n\r', end='', 24 | file=destination) 25 | 26 | class ATimer(): 27 | """ 28 | ATimer class, here delay and run are instance vars 29 | so we can control multiple timers independently 30 | """ 31 | def __init__(self): 32 | self.delay = 1.0 # can be edited while timer is running 33 | self.run = True # set False to exit before n runs out. 34 | 35 | async def atimer(self, n=1, delay=1.0, label='', destination=sys.stdout): 36 | """ 37 | destination must be a file-like object, must have a write method. 38 | """ 39 | self.delay = delay 40 | self.run = True 41 | for i in range(n): 42 | if not self.run: break 43 | await asyncio.sleep(self.delay) 44 | if destination == sys.stdout: # default 45 | print(f'{label} {i+1} {datetime.datetime.now()}\n\r', end='', 46 | file=destination) 47 | # We found that redirection with print(..., file=...) 48 | # does not work well with our Writer objects when threading. 49 | # Instead, just calling the object's write method does work. 50 | else: 51 | destination.write(f'{label} {i+1} {datetime.datetime.now()}\n\r') 52 | 53 | 54 | -------------------------------------------------------------------------------- /tasking/writer.txt: -------------------------------------------------------------------------------- 1 | writer.txt 2 | 3 | Notes on writer.py, which provides functions that put text into our 4 | editor buffers and windows, intended to be called from background tasks. 5 | 6 | The editors are not just for creating text. Python commands including 7 | concurrent tasks can use the writer module to redirect their output to 8 | editor buffers and windows, so the editors can be used for data capture and 9 | animated display. 10 | 11 | Any Python function named 'write' is magic: it can be invoked by any 12 | print statement or it can be the target of output redirection. 13 | 14 | The write function defined here in the writer module updates the sked 15 | current buffer and the edsel focus window. 16 | 17 | You do not invoke the write function explicitly. Instead, you specify 18 | the object (module or class instance) that contains the write function 19 | (or method). 20 | 21 | For example, a print function call with the optional argument 22 | file=writer sends the print output to the writer module, which invokes its 23 | write function to update the sked current buffer and the edsel 24 | focus window: 25 | 26 | print(..., file=writer) 27 | 28 | Or, you can redirect output from any code block to the writer module, 29 | so its write function can update the sked current buffer and the edsel focus window. 30 | For example: 31 | 32 | with redirect_stdout(writer) as buf: print('...') 33 | 34 | Sometimes we want to update a sked buffer other than the current buffer. 35 | The writer module write method can't do that. For that, we need an 36 | object with a write method that writes to the intended buffer. The 37 | buffers themselves are just dictionaries, not class instances, so they 38 | don't have write methods. Instead, the writer module defines a Writer 39 | class. 40 | 41 | We create an instance of the Writer class for each buffer where we want 42 | to send output. For example, to send output to the buffer named a.txt 43 | we create abuf: 44 | 45 | abuf = Writer('a.txt') 46 | 47 | The abuf object contains a write method that writes output to a.txt. 48 | To print to a.txt: 49 | 50 | print('...', file=abuf) 51 | 52 | To redirect output to a.txt: 53 | 54 | with redirect_stdout(abuf) as buf: print('...') 55 | 56 | The writer module also contains code that restores the cursor to the 57 | correct location in the Python REPL or an editing window after 58 | it writes text to the background task window. 59 | 60 | Revised May 2024 61 | 62 | -------------------------------------------------------------------------------- /piety/piety.txt: -------------------------------------------------------------------------------- 1 | piety.txt - Explanation and directions for piety.py 2 | 3 | The piety.py script starts a Piety session. It creates 4 | an event loop named piety, adds the readers for the shell and the 5 | editor, and starts the event loop with the shell running. It also defines 6 | a funtion run which is needed to run other scripts in the event loop. 7 | 8 | The run function adds all the identifiers defined in the script to the 9 | piety module namespace, so you can use them in subsequent interactive commands. 10 | 11 | Here is an example: 12 | 13 | $ python3 -m piety 14 | >>>> piety # Just for example, confirm the event loop is running 15 | <_UnixSelectorEventLoop running=True closed=False debug=False> 16 | >>>> from atimers import atimer # Type any Python statements at the prompt ... 17 | >>>> piety.create_task(atimer(3,1)) # ... including commands that use piety. 18 | > 19 | >>>> 1 2024-07-07 11:13:54.733618 20 | 2 2024-07-07 11:13:55.736089 21 | 3 2024-07-07 11:13:56.739600 22 | ... You must type RETURN (or ENTER) after the task exits to get the prompt ... 23 | >>>> run('edsel_script.py') # Run a script that uses the piety identifier 24 | ... script runs, putting windows on the display ... 25 | >>>> clr() 26 | ... restores full screen scrolling so windows will scroll away ... 27 | >>>> ^D 28 | $ 29 | ... You can type ^D or exit() to exit from the Piety shell and event loop. 30 | 31 | Statements in the piety module that follow run_forever() are not executed, 32 | so you cannot extend the piety module by adding more commands. 33 | You can type more commands at the >>>> prompt, or you can use the run 34 | command to run a script from another file. 35 | 36 | At the >>>> prompt, you cannot import scripts that import the piety module 37 | (to get the piety identifier) because the piety module has already started the 38 | asyncio event loop. You must use the run function imported by the piety 39 | module to run scripts. 40 | 41 | You can also run piety.py to start the piety event loop in an already running 42 | Python session: 43 | 44 | ...$ python3 -i 45 | ... 46 | >>> from runner import run 47 | >>> run('piety.py') 48 | >>>> 49 | ... 50 | >>>> ^D 51 | >>> 52 | ... ^D or exit() exits from the Piety shell and event loop ... 53 | ... back to the ordinary Python session ... 54 | 55 | -------------------------------------------------------------------------------- /console/console.py: -------------------------------------------------------------------------------- 1 | """ 2 | console.py - The functions here call each of the functions in the shell 3 | and pyhelp modules and redirect their output to an editor buffer. 4 | 5 | Those editor buffers can be used much like terminal windows with scroll 6 | back, so you can use system commands without exiting the Python session, or 7 | resorting to the host desktop. 8 | 9 | Shell command output is appended to the *Console* buffer, except 10 | man("topic") output is redirected to a new buffer named topic.man. 11 | help(topic) output is redirected to a new buffer named topic.help. 12 | 13 | Each of the functions here has the same name as the function it calls in 14 | the shell or pyhelp module, so in an interactive session, 'from shell 15 | import *' then 'from console import *' will write over the names of the 16 | functions in the shell module. 17 | """ 18 | 19 | from contextlib import redirect_stdout 20 | 21 | from redirect import redirect 22 | import shell, pyhelp 23 | import sked as ed 24 | import edsel as fr # frame 25 | 26 | def sh(cmd): 27 | 'Runs the shell in a subprocess, shell executes cmd' 28 | redirect('*Console*', lambda: shell.sh(cmd), f"sh('{cmd}')") 29 | 30 | def cd(path): 31 | 'Change directory, calls os.chdir' 32 | redirect('*Console*', lambda: shell.cd(path), f"cd('{path}')") 33 | 34 | def pwd(): 35 | 'Print working directory, calls os.cwd' 36 | redirect('*Console*', lambda: shell.pwd(), f"pwd()") 37 | 38 | def ls(path='.'): 39 | 'Runs ls -C for compact directory listing' 40 | redirect('*Console*', lambda: shell.ls(path), f"ls('{path}')") 41 | 42 | def lsl(path='.'): 43 | 'Runs ls -l for long form directory listing in alphabetic order' 44 | redirect('*Console*', lambda: shell.lsl(path), f"lsl('{path}')") 45 | 46 | def lslt(path='.'): 47 | 'Runs ls -lt for long form directory listing in chronological order' 48 | redirect('*Console*', lambda: shell.lslt(path), f"lslt('{path}')") 49 | 50 | def man(topic): 51 | 'Show man page on topic, a string. Save in new buffer named topic.man' 52 | bufname = topic + '.man' 53 | fr.e(bufname) 54 | with redirect_stdout(ed): 55 | shell.man(topic) 56 | fr.p(1) # put the cursor at the top of the buffer 57 | fr.refresh() 58 | 59 | def help(topic): 60 | """ 61 | Show help on topic, a Python object - module, function etc. 62 | Save in a new buffer named topic.help 63 | """ 64 | bufname = topic.__name__ + '.help' 65 | fr.e(bufname) 66 | with redirect_stdout(ed): 67 | pyhelp.help(topic) 68 | fr.p(1) # put the cursor at the top of the buffer 69 | fr.refresh() 70 | 71 | -------------------------------------------------------------------------------- /unix/terminal.py: -------------------------------------------------------------------------------- 1 | """ 2 | terminal.py - functions to set character mode or line mode, 3 | read/write single char or string 4 | 5 | This is a platform-dependent module. It uses the termios and tty 6 | modules, so it must run on a Unix-like host OS (including Linux and 7 | Mac OS X). For more about tty and termios see: 8 | 9 | http://docs.python.org/2/library/tty.html 10 | http://docs.python.org/2/library/termios.html 11 | http://hg.python.org/cpython/file/1dc925ee441a/Lib/tty.py 12 | 13 | http://man7.org/linux/man-pages/man3/termios.3.html 14 | 15 | """ 16 | 17 | import sys, tty, termios 18 | 19 | fd = sys.stdin.fileno() # Isn't sys.stdin always fileno 0 ? 20 | line_mode_settings = termios.tcgetattr(fd) # in case someone calls restore first 21 | 22 | # ...$ python 23 | # >>> import terminal 24 | # >>> terminal.line_mode_settings 25 | # [27394, 3, 19200, 536872395, 9600, 9600, ['\x04', '\xff', '\xff', '\x7f', '\x17', '\x15', '\x12', '\xff', '\x03', '\x1c', '\x1a', '\x19', '\x11', '\x13', '\x16', '\x0f', '\x01', '\x00', '\x14', '\xff']] 26 | 27 | def set_char_mode(): 28 | """ 29 | set sys.input to single character mode, save original mode 30 | """ 31 | # DON'T save settings again - we alread saved them on import, see above 32 | # this function should be idempotent. 33 | #global saved_settings 34 | #saved_settings = termios.tcgetattr(fd) 35 | # tty.setraw just calls termios.tcsetattr with particular flags 36 | # see http://hg.python.org/cpython/file/1dc925ee441a/Lib/tty.py 37 | tty.setraw(fd) 38 | 39 | def set_line_mode(): 40 | """ 41 | restore sys.input to line mode 42 | """ 43 | termios.tcsetattr(fd, termios.TCSAFLUSH, line_mode_settings) 44 | 45 | def getchar(): 46 | """ 47 | Get a single character from console keyboard, without waiting for RETURN 48 | """ 49 | return sys.stdin.read(1) 50 | 51 | def getchars(n): 52 | """ 53 | Get up to n characters from console keyboard, without waiting for RETURN 54 | """ 55 | return sys.stdin.read(n) 56 | 57 | def putstr(s): 58 | """ 59 | Print string (can be just one character) on stdout with no 60 | formatting (unlike plain Python print). Flush to force output immediately. 61 | If you want newline, you must explicitly include it in s. 62 | Prints to stdout, which may be redirected. 63 | """ 64 | print(s, end='', flush=True) 65 | 66 | # Test 67 | 68 | def main(): 69 | c = line = '' 70 | set_char_mode() # enter single character more 71 | putstr('> ') 72 | while not c == '\r': 73 | c = getchar() 74 | putstr(c) 75 | line += c 76 | set_line_mode() # return to normal mode 77 | print() 78 | print(line) 79 | 80 | if __name__ == '__main__': 81 | main() 82 | -------------------------------------------------------------------------------- /coroutines/timer_loops.txt: -------------------------------------------------------------------------------- 1 | timer_loops.txt - Explanation and directions for timer_loops.py. 2 | 3 | timer_loops.py demonstrates interleaving timer tasks in short-lived asyncio 4 | event loops without a Python shell. 5 | 6 | The timer_loops.py script is self-contained. To run the script: 7 | 8 | ...$ python3 -m timer+loops 9 | ... progress messages and timer messages appear, then script exits ... 10 | ...$ 11 | 12 | Alternatively, you can follow along with these directions and type 13 | each command at the Python prompt: 14 | 15 | ...$ . ~/Piety/bin/paths 16 | ...$ python3 -i 17 | ... 18 | >>> import asyncio as aio 19 | >> loop = aio.get_event_loop() 20 | >>> loop 21 | <_UnixSelectorEventLoop running=False closed=False debug=False> 22 | >>> aio.all_tasks(loop) 23 | set() 24 | 25 | Now we have a fresh event loop with no tasks. 26 | 27 | >> from atimers import atimer 28 | >>> ta = loop.create_task(atimer(5,5,'A')) 29 | >>> ta 30 | > 31 | >>> aio.all_tasks(loop) 32 | {>} 33 | 34 | >>> loop.run_until_complete(ta) 35 | A 1 2024-06-15 09:14:58.702109 36 | A 2 2024-06-15 09:15:03.708406 37 | A 3 2024-06-15 09:15:08.714560 38 | A 4 2024-06-15 09:15:13.720797 39 | A 5 2024-06-15 09:15:18.726920 40 | >>> 41 | >>> aio.all_tasks(loop) 42 | set() 43 | 44 | We don't get >>> prompt back until loop exits. 45 | Apparently coroutine exit deletes task from loop. 46 | 47 | Set up tasks to interleave, exit when first task exits. 48 | 49 | >>> ta = loop.create_task(atimer(5,1,'A')) 50 | >>> tb = loop.create_task(atimer(10,0.5,'B')) 51 | >>> aio.all_tasks(loop) 52 | {>, >} 55 | >>> loop.run_until_complete(ta) 56 | B 1 2024-06-15 09:19:10.924962 57 | A 1 2024-06-15 09:19:11.425150 58 | B 2 2024-06-15 09:19:11.425711 59 | B 3 2024-06-15 09:19:11.927582 60 | A 2 2024-06-15 09:19:12.427312 61 | B 4 2024-06-15 09:19:12.429206 62 | B 5 2024-06-15 09:19:12.931009 63 | A 3 2024-06-15 09:19:13.429746 64 | B 6 2024-06-15 09:19:13.432379 65 | B 7 2024-06-15 09:19:13.934095 66 | A 4 2024-06-15 09:19:14.431956 67 | B 8 2024-06-15 09:19:14.436029 68 | B 9 2024-06-15 09:19:14.937835 69 | A 5 2024-06-15 09:19:15.434077 70 | 71 | We still have the other task - run loop until it exits too. 72 | 73 | >>> aio.all_tasks(loop) 74 | { wait_for=>} 77 | 78 | The other task hasn't completed yet. 79 | 80 | >>> loop.run_until_complete(tb) 81 | B 10 2024-06-15 09:19:47.082362 82 | >>> aio.all_tasks(loop) 83 | set() 84 | 85 | -------------------------------------------------------------------------------- /piety/apyshell.py: -------------------------------------------------------------------------------- 1 | """ 2 | apyshell.py - Adapt our pysh custom Python shell to run in an asyncio event loop. 3 | Define the asyncio reader function apysh that handles each shell 4 | keystroke. 5 | 6 | Structure based on aterminal.py. Calls pyshell.py functions to do all the work. 7 | """ 8 | 9 | import sys, asyncio 10 | import key 11 | import terminal as term 12 | import pyshell as sh 13 | import apmacs # apmacs.apm() starts asyncio display editor in runcmd special case 14 | 15 | loop = None # must be global, used in both restore() and main() 16 | running = False # loop is running, assigned by restore and apysh 17 | 18 | def setup(): 19 | 'Assign new prompt strings, then call sh.setup.' 20 | # This is a hack to avoid making ps1 ps2 start_col arguments to sh.runcmd 21 | # FIXME? Save prompt strings so they can be restored? 22 | sh.ps1 = '>>>> ' # different from CPython >>> and pyshell >> 23 | sh.ps2 = '.... ' # line up with async_ps1 and async_start_col 24 | sh.start_col = 5 # not 3 25 | sh.setup() 26 | 27 | def restore(): 28 | 'Stop the event loop, but dont close it. Restore prompts, call sh.restore.' 29 | # FIXME? Any reason to restore prompt strings here? 30 | global running 31 | eventloop = asyncio.get_event_loop() # get the running event loop 32 | eventloop.stop() # escape from run_forever() 33 | running = False 34 | sh.restore() 35 | 36 | def runcmd(c): 37 | 'Call pyshell runcmd, but handle two special cases. See unline comments.' 38 | # 1. First test for exit, if exit call restore. 39 | if (c == key.C_d and sh.cmd == '') or (sh.cmd == 'exit()'): 40 | restore() 41 | # 2. After key.cr test for apm() command that starts display editing. 42 | # In that case do *not* call pyshell.runcmd key.cr case which ends with 43 | # restore_cursor_to_cmdline(); pustr(prompt). Instead handle inline here. 44 | elif (c == key.cr and sh.cmd == 'apm()'): 45 | apmacs.apm() # starts asyncio display editing 46 | else: 47 | sh.runcmd(c) # handles C_d differently when sh.cmd is not empty 48 | 49 | def apysh(): 50 | """ 51 | Run each time sys.stdin detects a new character when running in cmd_mode. 52 | Set up terminal on first call only, on all calls call runcmd(c). 53 | """ 54 | global running 55 | if not running: 56 | setup() 57 | running = True 58 | c = term.getchar() # not blocking, asyncio only calls apysh when char is ready. 59 | runcmd(c) 60 | 61 | def main(): 62 | 'Set up event loop, setup() the terminal, and start the event loop.' 63 | global loop 64 | loop = asyncio.get_event_loop() 65 | loop.add_reader(sys.stdin, handler) # FIXME? tty not stdin, like display.py? 66 | setup() 67 | loop.run_forever() 68 | 69 | if __name__ == '__main__': 70 | main() 71 | loop.close() 72 | 73 | -------------------------------------------------------------------------------- /threads/threads_3.txt: -------------------------------------------------------------------------------- 1 | threads_3.txt 2 | 3 | Experiments with Python threading using the pmacs editor with timer.py 4 | and writer.py, with our custom pysh REPL from pyshell.py. 5 | 6 | In this demonstration, we show a thread updating a buffer in one window, 7 | while we edit text in another buffer in its window, and type at our pysh 8 | REPL to control the thread. 9 | 10 | We need our custom pysh REPL for this demo so our code can 11 | restore the cursor to the correct location in the REPL as we 12 | type, after a task updates a window. This is not possible with 13 | the standard Python REPL. 14 | 15 | It is easiest to explain by doing this demonstration. Just 16 | follow along, typing the statements here as you go. 17 | 18 | (Alternatively, you can run the script in threads_2.py - see directions 19 | at the end of this file.) 20 | 21 | At the system command prompt (often $), define the PYTHONPATH, so this 22 | demo works in any directory. 23 | 24 | ...$ . ~/Piety/bin/paths 25 | 26 | Run the tm script (not pm) which imports the editor, timer, and writer 27 | modules, opens a window into the scratch.txt buffer, and starts the pysh REPL: 28 | 29 | ...$ python3 -im tm 30 | 31 | At the pysh prompt >> (not the standard Python prompt >>>), 32 | split the window into two: 33 | 34 | >> o2() 35 | 36 | Create a buffer in one of the windows: 37 | 38 | >> e('a.txt') 39 | a.txt, 0 lines 40 | 41 | Create a timer object, and a Writer object that the thread 42 | will use to redirect thread output to a window: 43 | 44 | >> ta = Timer() 45 | >> abuf = Writer('a.txt') 46 | 47 | Now start a thread that prints 1000 messages at 1 second 48 | intervals, redirected by the abuf writer to the a.txt window: 49 | 50 | >> Thread(target=ta.timer,args=(1000,1,'A',abuf)).start() 51 | 52 | The a.txt window updates with a new timer message every second. 53 | 54 | Type the command to change the focus to the other window. 55 | 56 | >> on() 57 | 58 | Type the command to move the cursor from the REPL into the editing window. 59 | 60 | >> tpm() 61 | 62 | Wait for one more message to appear in the a.txt window, then the cursor 63 | moves to the scratch.txt window. Now you can type and edit in 64 | scratch.txt while the timer messages appear in the a.txt window. 65 | 66 | Type M-x (hold down the alt key while typing x) to put the 67 | cursor back in the REPL. This command imports identifiers 68 | 69 | >> from threads_3 import * 70 | 71 | so you can type commands like 72 | 73 | >> ta.delay = 0.5 74 | 75 | to change the rate of the messages or 76 | 77 | >> threads() 78 | 79 | To show the running threads. 80 | 81 | As an alternative to typing the commands in this page, you can run the 82 | script in threads_3.py. Run the tm script as described above: python3 -im tm 83 | Then at the pysh prompt type 84 | 85 | >> import threads_3. 86 | 87 | Two windows will appear. Just start typing the in the scratch.txt window, 88 | as timer messages appear in the other window. 89 | 90 | Revised May 2024 91 | 92 | -------------------------------------------------------------------------------- /vt_terminal/keyseq_1.py: -------------------------------------------------------------------------------- 1 | """ 2 | keyseq_1.py - construct emacs-style key sequence from one or more characters. 3 | This is the old version that does not work in the asyncio loop. 4 | Compare to the new keyseq.py. 5 | """ 6 | 7 | import terminal, key 8 | 9 | prefix = '' # incomplete key sequence 10 | 11 | def keyseq(c): 12 | """ 13 | Construct emacs-style key sequence from one or more characters. 14 | On each call, pass in a single character. 15 | Do not block waiting for sequence to complete, return after each call. 16 | When sequence is complete, return the entire sequence (maybe single char). 17 | When sequence is incomplete, return the empty string ''. 18 | Pass in C_g (^G) to cancel incomplete sequence and just return C_g. 19 | """ 20 | global prefix 21 | 22 | # C_g is the unconditional cancel command, 23 | # always discard any prefix and just return C_g itself. 24 | if c == key.C_g: 25 | prefix = '' 26 | return key.C_g 27 | 28 | # No prefix, prefix character arrives, start prefix 29 | if prefix == '' and c in (key.esc, key.C_x, key.C_c): # more to come? 30 | prefix = c 31 | return '' 32 | 33 | # No prefix, ordinary character arrives, just return this character 34 | elif prefix == '': 35 | return c 36 | 37 | # Handle each prefix 38 | 39 | # esc prefix for meta keys and ANSI terminal control codes 40 | elif prefix == key.esc: 41 | # ANSI escape codes for terminal control, begin with esc-[ called csi 42 | if c == '[': 43 | prefix += '[' 44 | return '' # now prefix == key.csi, wait for rest of sequence 45 | 46 | # Meta keys, begin with esc then one other key but not [ 47 | else: 48 | kseq = prefix + c 49 | prefix = '' 50 | return kseq 51 | 52 | # CSI prefix for control keys 53 | elif prefix == key.csi: 54 | # For now we only support the four arrow keys: csi+'A' etc. 55 | # with just one char after csi, so we can return now 56 | kseq = prefix + c 57 | prefix = '' 58 | return kseq 59 | 60 | # ctrl-X prefix for window commands and buffer commands 61 | elif prefix == key.C_x: 62 | # C_x + one more key 63 | kseq = prefix + c 64 | prefix = '' 65 | return kseq 66 | 67 | # ctrl-C prefix for indent commands 68 | elif prefix == key.C_c: 69 | # C_c + one more key 70 | kseq = prefix + c 71 | prefix = '' 72 | return kseq 73 | 74 | # Unrecognized prefix - clear prefix and return this character 75 | else: 76 | prefix = '' 77 | return c 78 | 79 | def main(): 80 | 'Demonstrate keyseq' 81 | terminal.putstr('> ') 82 | terminal.set_char_mode() 83 | line = [] 84 | k = 'x' # anything but cr or '' 85 | while k != key.cr: 86 | c = terminal.getchar() 87 | k = keyseq(c) # return '' if incomplete prefix 88 | line += [k] # include [''] if incomplete prefix 89 | terminal.putstr(k) 90 | terminal.set_line_mode() 91 | print() 92 | print([ [c for c in k] for k in line ]) # show any esc or other unprintables 93 | 94 | if __name__ == '__main__': 95 | main() 96 | 97 | -------------------------------------------------------------------------------- /doc/gracle_excerpts.txt: -------------------------------------------------------------------------------- 1 | 2 | I can no longer find the Gracle paper on the web. It is sufficiently 3 | interesting that I typed in a few excerpts here. 4 | 5 | UPDATE: found at http://dept-info.labri.fr/~strandh/gracle.ps 6 | 7 | --- 8 | 9 | Gracle: A development and deployment environment for Common Lisp 10 | 11 | Robert Strandh, December 23, 2004 12 | 13 | 1 Introduction 14 | 15 | This document describes what we call a "development and deployment 16 | environment" that we would like to implement. ... we are planning to 17 | implement this environment on top of Linux, as an ordinary Linux 18 | process. 19 | 20 | Gracle differs in two important ways from ordinary operating systems 21 | ... in that it does not have files, and that it does not have 22 | processes. 23 | 24 | 1.1 Gracle does not have files 25 | 26 | Gracle ... does not make a distinction between primary and secondary 27 | storage. ... Gracle has a *single-level store*. 28 | 29 | EROS (Extremely Reliable Operating System) is an operating system 30 | witha a crash-proof single level store. We are planning to use the 31 | EROS model for Gracle, except that we do not plan to use the naked 32 | disk and instead implement the permanent store as a (single) Linux 33 | file. 34 | 35 | The great advantage of not having files ... Structured objects in main 36 | memory are all persistent. 37 | 38 | 1.2 Gracle does not have processes 39 | 40 | ... In Gracle, all threads of execution are *light-weight processes*, 41 | commonly known as *threads*. ... 42 | 43 | 1.3 Objects are not organized in a heirarchy 44 | 45 | We would like to eliminate the heirarchy of objects (files) ... Most 46 | often, the order of the directories in the path of an object in such a 47 | heirarchy is not meaningful to the user ... 48 | 49 | Also, we would like to be able to construct collections of objects on 50 | the fly, and not be limited to the collections that the directories in 51 | a hierarchy imposes ... 52 | 53 | While we could settle with ... Common Lisp (special variables), it 54 | seems useful to have some kind of "data base" of objects that are 55 | accessible by queries ... We distinguish between objects that are 56 | *archived* and objects that are not. The concept of archived objects 57 | is different from that of *persistent objects*. ... Archiving an 58 | object just means giving it certain properties and making it 59 | accessible by queries ... Such properties include creation dates and 60 | perhaps a number of *tags* that serve the same purpose as directory 61 | names in traditional hierarchical systems. ... 62 | 63 | In a traditional hierarchy, the *current directory* serves two 64 | different purposes. The first is to serve as the current collection 65 | of objects... the second purpose is to give newly created objects this 66 | prefix so that such objects become members of the same collection. 67 | 68 | For Gracle, we suggest using a set of *current properties* defining 69 | the current collection of objects, and a set of *assigned properties* 70 | that define what proprities newly archived objects will inherit. They 71 | are not the same, since the current properties might incude 72 | restrictions on date of creation and any other arbitrary filter ... 73 | 74 | 2. How Linux memory management works 75 | ... 76 | 77 | 3. How the EROS single-level store works 78 | ... 79 | 80 | 4. Implementing the single-level store 81 | ... 82 | -------------------------------------------------------------------------------- /threads/README.md: -------------------------------------------------------------------------------- 1 | 2 | threads 3 | ======= 4 | 5 | Experiments with tasking and concurrency using Python threads. 6 | 7 | These experiments run in our [pmacs](../editors/README.md) editor, 8 | which we start here from the script in *tm.py* rather than 9 | *pm.py*. The name *tm* is supposed to suggest "tasking *pmacs*". 10 | The *tm* script loads all the modules used by *pmacs*, some additional 11 | modules that support the tasking experiments, and starts our custom 12 | *pysh* (rhymes with fish) Python interpreter. 13 | 14 | To start editing in a display window, type the function call *tpm()* at 15 | the *pysh* prompt >>, instead of the *pm()* call you type at the standard 16 | Python prompt >>>. To return to the *pysh* command prompt, type M-x 17 | (meta x, hold the alt key and type x), just as you do in any *pmacs* 18 | session. 19 | 20 | These experiments are preserved here for completeness, but I have 21 | decided to use the [*asyncio* library](../piety) (also 22 | [here](../coroutines)) to support tasking and concurrency in Piety. I 23 | rejected *threading* because it uses the host operating system's 24 | threading library. My goal for Piety is to replace the host operating 25 | system. 26 | 27 | NOTE added Sep 2025: Some of these demos no longer work. In particular, 28 | in *threads_3* editing in the *scratch.txt* window while the timer 29 | thread updates the *a.txt* window no longer works --- the cursor in 30 | *scratch.txt* returns to column 1 on each timer tick, and control characters 31 | are not processed correctly. Also, in *threads_2*, typing commands at the 32 | Python REPL can result in scrambling the two windows that display the 33 | two timer threads. 34 | 35 | Apparently, changes I made since Summer 2024 have broken the threads demos. 36 | I spent some time trying to fix this but was unsuccessful. Meanwhile, I 37 | have decided to concentrate on *asyncio* and event loops instead of 38 | threads, so at this time I have no further plans to try to fix this. 39 | 40 | ### Files ### 41 | 42 | - **threads_1.txt**: Directions for experiments with Python threading 43 | using the functions in *timers.py* and *writer.py*. These experiments 44 | run with the standard Python interpreter and reveal its limitations. 45 | 46 | - **threads_2.py**: Script that runs the code explained in *threads_2.txt*. 47 | 48 | - **threads_2.txt**: Directions for experiments with Python threading 49 | using functions in *timer* and *writer* with out custom *pysh* interpreter. 50 | Here we show two threads rapidly updating two buffers 51 | in two windows, while we type at our *pysh* interpreter to control the threads. 52 | 53 | - **threads_3.py**: Script that runs the code explained in *threads_3.txt*. 54 | 55 | - **threads_3.txt**: Directions for further experiments with Python threading 56 | using functions in *timer* and *writer* with the *pysh* interpreter. 57 | Here we show a thread updating a buffer in one window, 58 | while we edit text in another buffer in its window, or type at our *pysh* 59 | interpreter. 60 | 61 | - **timers.py**: Functions to run in tasking experiments, that print 62 | timestamps at intervals. 63 | 64 | - **tm.py**: script that loads modules for threading experiments, including 65 | editors, *writer*, *timers*, classes and functions from *threading*, and 66 | then starts our custom *pysh* Python interpreter. 67 | 68 | Revised Sep 2025 69 | 70 | -------------------------------------------------------------------------------- /editors/demo/sked_fragments.py: -------------------------------------------------------------------------------- 1 | """ 2 | sked_fragments.phy - Fragments from sked.py, to edit for pmacs.py open_line demo. 3 | 4 | """ 5 | 6 | # First, edit these sections near the top of this file before the ######... line 7 | # to show the old openline behavior without automatic indentation. 8 | 9 | try: 10 | _ = dot 11 | except: 12 | # Code block with one level of indentation for testing openline 13 | buffer = ['\n'] 14 | # Add more indented lines here. You can copy them from the real sked.py. 15 | # You have to type the tab key, or type the space key four times, 16 | # to reach the required indentation. 17 | 18 | # ... many lines omitted ... 19 | 20 | def w(fname=None, set_saved=set_saved): 21 | # Indented comment block for testing wrap. 22 | # Add line in the middle, then select the three command lines 23 | # and wrap with M-q. The wrapped lines start at the left margin of 24 | # the window, which looks terrible. 25 | """ 26 | w(rite) buffer to file, default fname is in filename. 27 | ... more comment text follows ... 28 | """ 29 | global filename, bufname, saved 30 | # .. lines omitted ... 31 | if success: 32 | if filename != fname: # we saved buffer with a new, different filename 33 | # Code block with three levels of indentation 34 | filename = fname 35 | # Add more indented lines here. You have to type tab three times, 36 | # or type 12 spaces, to reach the required indentation. 37 | 38 | # Code block out-dented from preceding block 39 | set_saved(True) 40 | # Add more out-dented lines here. You have to type tab two times 41 | # or type eight spaces to reach the required indentation. 42 | 43 | # Next, revise and reload the code in pmacs.py that defines openline behavior 44 | # to add automatic indentation. 45 | 46 | ############################################################################## 47 | 48 | # Then, edit these sections after the the ##########... line 49 | # to show the new openline behavior with automatic indentation. 50 | 51 | try: 52 | _ = dot 53 | except: 54 | # Code block with one level of indentation for testing openline 55 | buffer = ['\n'] 56 | # Add more indented lines here. You can copy them from the real sked.py. 57 | # Now you only have to type ENTER (or RETURN) to reach the correct indentation. 58 | 59 | # ... many lines omitted ... 60 | 61 | def w(fname=None, set_saved=set_saved): 62 | # Indented comment block for testing wrap. 63 | # Add line in the middle, then select the three command lines 64 | # and wrap with M-q. The wrapped lines start at the left margin of 65 | # the window, which looks terrible. 66 | """ 67 | w(rite) buffer to file, default fname is in filename. 68 | ... more comment text follows ... 69 | """ 70 | global filename, bufname, saved 71 | # .. lines omitted ... 72 | if success: 73 | if filename != fname: # we saved buffer with a new, different filename 74 | # Code block with three levels of indentation 75 | filename = fname 76 | # Add more indented lines here. The cursor will automatically indent. 77 | 78 | 79 | # Code block out-dented from preceding block 80 | set_saved(True) 81 | # Add more out-dented lines here. 82 | 83 | -------------------------------------------------------------------------------- /piety/eventloop.py: -------------------------------------------------------------------------------- 1 | """ 2 | eventloop.py - Creates BUT DOES NOT START an asyncio event loop named 3 | piety that responds to terminal keystrokes. Our shells and editors run 4 | in this event loop, and it can also run other asyncio tasks. 5 | 6 | This event loop is named piety because it is essential for running 7 | asyncio tasks along with the functioning terminal in the Piety system. 8 | 9 | This script creates BUT DOES NOT START the piety event loop. To start 10 | the event loop, after this script type these commands at the standard 11 | Python REPL: 12 | 13 | >>> from eventloop import * 14 | ... 15 | >>> piety.create_task(...) 16 | ... 17 | >>> piety.run_forever() 18 | 19 | This gives you the opportunity to create tasks for this event loop to run 20 | before you start it -- then those tasks will also start when you 21 | call run_forever() 22 | 23 | After you type the piety.run_forever() command, you will see no prompt. 24 | Type RETURN to show the prompt. It shows four darts >>>> to indicate 25 | that the REPL is now running in the piety event loop. You may have to 26 | type RETURN once again to start any tasks that run from the event loop. 27 | 28 | Alternatively, you can type piety_start() at the >>> prompt. Then 29 | the ansyncio prompt >>>> will appear immediately. (Note that is 30 | and underscore _ not a dot). 31 | 32 | To stop the event loop and pause all the tasks it is running, type ^D or 33 | exit() at the >>>> prompt. NOw you see the >>> prompt again to indicate 34 | you are running in the standard Python REPL. You can restart the event 35 | loop and resume the tasks by typing the piety.run_forever() or piety_start() 36 | command again. 37 | 38 | Although this module is essential for using asyncio in Piety, it does 39 | not contain any async code -- it does not use the keywords async or 40 | await. Tasks that run in this event loop do use the async and await 41 | keywords. But the code in this module that handles keystrokes is just a 42 | handler, not a task. 43 | 44 | This module is based on the earlier piety.py. We renamed it to 45 | eventloop.py here to avoid the inconveniences that arise when a module 46 | has the same name as one of its contents: if you write 'from piety 47 | import *' then piety refers only to the eventloop, you can no longer use 48 | piety to refer to the module itself. 49 | 50 | We also removed piety.run_forever() from this script so you can create 51 | other tasks before starting the event loop. 52 | 53 | Also, we no longer do 'from import run' here. It makes more sense to 54 | import run before we run this module, so we can use to it run this very 55 | module. 56 | 57 | We are keeping the old piety.py module because it appears in some older 58 | scripts and documentation. 59 | 60 | The Python asyncio library uses the name event_loop, which is different 61 | from our name eventloop. 62 | """ 63 | 64 | import sys, asyncio 65 | import pyshell, apyshell, apmacs 66 | 67 | def handler(): 68 | if pyshell.cmd_mode: 69 | apyshell.apysh() # async shell 70 | else: 71 | apmacs.apmrun() # async display editor foreground job 72 | 73 | piety = asyncio.get_event_loop() 74 | piety.add_reader(sys.stdin, handler) 75 | 76 | def piety_start(): 77 | """ 78 | Alternative to piety.run_forever() 79 | so you don't need to type RET to get the >>>> asyncio shell prompt 80 | """ 81 | apyshell.setup() 82 | apyshell.running = True 83 | piety.run_forever() 84 | 85 | -------------------------------------------------------------------------------- /vt_terminal/key.py: -------------------------------------------------------------------------------- 1 | """ 2 | key.py - ASCII and ANSI control codes for the terminal keyboard 3 | 4 | ASCII control codes 5 | http://en.wikipedia.org/wiki/ASCII#ASCII_control_characters 6 | http://www.inwap.com/pdp10/ansicode.txt 7 | http://ascii-table.com/control-chars.php 8 | 9 | ANSI control sequences, Best explanation at: 10 | http://www.inwap.com/pdp10/ansicode.txt. 11 | especially section "Minimum requirements for VT100 emulation" 12 | See also: 13 | http://vt100.net/ 14 | http://en.wikipedia.org/wiki/ANSI_escape_code 15 | https://en.wikipedia.org/wiki/C0_and_C1_control_codes 16 | http://invisible-island.net/xterm/ctlseqs/ctlseqs.html (also dirs above) 17 | This page says 'Line and column numbers start at 1': 18 | http://www.umich.edu/~archive/apple2/misc/programmers/vt100.codes.txt 19 | 20 | """ 21 | 22 | # Names of characters 23 | 24 | space = ' ' 25 | 26 | # ASCII codes for control characters 27 | 28 | bel = '\a' # bell, ^G 29 | bs = '\b' # backspace, ^H 30 | cr = '\r' # carriage return, ^M 31 | lf = '\n' # line feed, ^J 32 | htab = '\t' # horizontal tab, ^I 33 | 34 | delete = '\x7F' # del is a python keyword, ^? 35 | 36 | C_at = '\x00' # ^@, nul, also obtained by ^space on many terminals 37 | C_a = '\x01' # ^A, soh 38 | C_b = '\x02' # ^B, stx 39 | C_c = '\x03' # ^C, etx 40 | C_d = '\x04' # ^D, eot 41 | C_e = '\x05' # ^E, enq 42 | C_f = '\x06' # ^F, ack 43 | C_g = '\a' # ^G, bel 44 | C_h = '\b' # ^H, bs 45 | C_i = '\t' # ^I, ht 46 | C_j = '\n' # ^J, lf 47 | C_k = '\v' # ^K, vt 48 | C_l = '\f' # ^L, ff 49 | C_m = '\r' # ^M, cr 50 | C_n = '\x0E' # ^N, so 51 | C_o = '\x0F' # ^O, si 52 | C_p = '\x10' # ^P, dle 53 | C_q = '\x11' # ^Q, dc1, xon 54 | C_r = '\x12' # ^R, dc2 55 | C_s = '\x13' # ^S, dc3, xoff 56 | C_t = '\x14' # ^T, dc4 57 | C_u = '\x15' # ^U, nak 58 | C_v = '\x16' # ^V, syn 59 | C_w = '\x17' # ^W, etb 60 | C_x = '\x18' # ^X, can 61 | C_y = '\x19' # ^Y, em 62 | C_z = '\x1a' # ^Z, sub 63 | C_space = '\x99' # placeholder for'\x0' # ^space, alias for ^@, nul 64 | 65 | # Define Meta keys, prefixed with esc 66 | 67 | esc = '\x1B' # \e does not work 'invalid \x escape' 68 | 69 | M_a = esc + 'a' 70 | M_b = esc + 'b' 71 | M_c = esc + 'c' 72 | M_d = esc + 'd' 73 | M_e = esc + 'e' 74 | M_f = esc + 'f' 75 | M_g = esc + 'g' 76 | M_h = esc + 'h' 77 | M_i = esc + 'i' 78 | M_j = esc + 'j' 79 | M_k = esc + 'k' 80 | M_l = esc + 'l' 81 | M_m = esc + 'm' 82 | M_n = esc + 'n' 83 | M_o = esc + 'o' 84 | M_p = esc + 'p' 85 | M_q = esc + 'q' 86 | M_r = esc + 'r' 87 | M_s = esc + 's' 88 | M_t = esc + 't' 89 | M_u = esc + 'u' 90 | M_v = esc + 'v' 91 | M_w = esc + 'w' 92 | M_x = esc + 'x' 93 | M_y = esc + 'y' 94 | M_z = esc + 'z' 95 | 96 | M_lt = esc + '<' # emacs go to top 97 | M_gt = esc + '>' # emacs go to end 98 | 99 | M_lp = esc + '(' # Piety go to top, other (viewer) window 100 | M_rp = esc + ')' # Piety go to bottom, other (viewer) window 101 | 102 | M_percent = esc + '%' # emacs replace string 103 | 104 | M_carat = esc + '^' # emacs join lines 105 | 106 | M_ret = esc + cr # emacs eww-open-in-new-buffer, browser open URL at point 107 | 108 | # ANSI codes for arrow keys 109 | 110 | csi = esc+'[' # ANSI control sequence introducer 111 | 112 | up = csi+'A' # cursor up, default 1 char 113 | down = csi+'B' # cursor down, default 1 char 114 | right = csi+'C' # cursor forward (right), default 1 char 115 | left = csi+'D' # cursor backward (left), default 1 char 116 | 117 | -------------------------------------------------------------------------------- /vt_terminal/keyseq.py: -------------------------------------------------------------------------------- 1 | """ 2 | keyseq.py - construct emacs-style key sequence from one or more characters. 3 | This is the new version that works in the asyncio event loop. 4 | Compare to the old version in keyseq_1.py 5 | """ 6 | 7 | import terminal, key 8 | 9 | prefix = '' # incomplete key sequence 10 | 11 | def keyseq(c): 12 | """ 13 | Construct emacs-style key sequence from one or more characters. 14 | On each call, pass in the first (maybe the only) character in the key sequence. 15 | If the first character is the prefix of a multi-character sequence, 16 | keep reading characters until a complete sequence has been read. 17 | When sequence is complete, return the entire sequence (maybe single char). 18 | Pass in C_g (^G) to cancel incomplete sequence and just return C_g. 19 | """ 20 | global prefix 21 | 22 | while True: # continue reading characters until a complete keyseq is reached 23 | 24 | # C_g is the unconditional cancel command, 25 | # always discard any prefix and just return C_g itself. 26 | if c == key.C_g: 27 | prefix = '' 28 | return key.C_g 29 | 30 | # No prefix, prefix character arrives, start prefix 31 | if prefix == '' and c in (key.esc, key.C_x, key.C_c): # more to come? 32 | prefix = c 33 | # return '' 34 | c = terminal.getchar() 35 | continue 36 | 37 | # No prefix, ordinary character arrives, just return this character 38 | elif prefix == '': 39 | return c 40 | 41 | # Handle each prefix 42 | 43 | # esc prefix for meta keys and ANSI terminal control codes 44 | elif prefix == key.esc: 45 | # ANSI escape codes for terminal control, begin with esc-[ called csi 46 | if c == '[': 47 | prefix += '[' 48 | # return '' # now prefix == key.csi, wait for rest of sequence 49 | c = terminal.getchar() 50 | continue 51 | 52 | # Meta keys, begin with esc then one other key but not [ 53 | else: 54 | kseq = prefix + c 55 | prefix = '' 56 | return kseq 57 | 58 | # CSI prefix for control keys 59 | elif prefix == key.csi: 60 | # For now we only support the four arrow keys: csi+'A' etc. 61 | # with just one char after csi, so we can return now 62 | kseq = prefix + c 63 | prefix = '' 64 | return kseq 65 | 66 | # ctrl-X prefix for window commands and buffer commands 67 | elif prefix == key.C_x: 68 | # C_x + one more key 69 | kseq = prefix + c 70 | prefix = '' 71 | return kseq 72 | 73 | # ctrl-C prefix for indent commands 74 | elif prefix == key.C_c: 75 | # C_c + one more key 76 | kseq = prefix + c 77 | prefix = '' 78 | return kseq 79 | 80 | # Unrecognized prefix - clear prefix and return this character 81 | else: 82 | prefix = '' 83 | return c 84 | 85 | def main(): 86 | 'Demonstrate keyseq' 87 | terminal.putstr('> ') 88 | terminal.set_char_mode() 89 | line = [] 90 | k = 'x' # anything but cr or '' 91 | while k != key.cr: 92 | c = terminal.getchar() 93 | k = keyseq(c) # return '' if incomplete prefix 94 | line += [k] # include [''] if incomplete prefix 95 | terminal.putstr(k) 96 | terminal.set_line_mode() 97 | print() 98 | print([ [c for c in k] for k in line ]) # show any esc or other unprintables 99 | 100 | if __name__ == '__main__': 101 | main() 102 | 103 | -------------------------------------------------------------------------------- /browser/urls.py: -------------------------------------------------------------------------------- 1 | """ 2 | urls.py - Sample URLs for testing. 3 | """ 4 | 5 | # Sample URLs 6 | 7 | # My home page at github and the same page -- just a file -- in my local repo. 8 | # Very simple, no style, many short paragraphs, most contain one or more links. 9 | home = 'https://jon-jacky.github.io/home/' # .../home/index.html 10 | home_file = 'file:///home/jon/home/index.html' 11 | 12 | # This page has some links which are relative URLs. 13 | # For testing browser code from a file URL when no Internet. 14 | zfile = 'file:///home/jon/z/z/index.html' 15 | 16 | # Another simple web page, with more text and fewer links 17 | # but several uses of
    unordered list and
  • list item. 18 | learned = "https://jon-jacky.github.io/tesc_cs/fofc/learned.html" 19 | 20 | # github page - Lots of stuff in addition to our content - 2227 lines! 21 | # Our content is in lines 1967 - 2062 out of 2227 22 | rationale = 'https://github.com/jon-jacky/Piety/blob/browser/doc/rationale.md' 23 | 24 | # github 'raw' page - just the markdown source for rational.md - only 136 lines 25 | rationale_raw = 'https://raw.githubusercontent.com/jon-jacky/Piety/refs/heads/browser/doc/rationale.md' 26 | 27 | # Mostly text web page, article content in lines 516 - 575 out of 743 28 | salmagundi = 'https://salmagundi.skidmore.edu/articles/747-martin-amis-and-the-changing-of-the-guard' 29 | 30 | # Mostly text web page with code fragments and itemized lists 31 | # Shows up fine in Firefox 32 | # BUT Piety browser gets urllib.error.HTTPError: HTTP Error 403: Forbidden 33 | # Maybe this site doesn't like our HTTP User Agent? Thinks it's a crawler? 34 | hell = 'https://chrisdone.com/posts/hell-year-in-review-2025/' 35 | 36 | # Mostly text web page with headings and code fragments 37 | forth = 'https://pygmy.utoh.org/3ins4th.html' 38 | 39 | # Mostly text web sites but with lots of other stuff 40 | askmefi = 'https://ask.metafilter.com/' 41 | mefi = 'https://www.metafilter.com/' 42 | hn = 'https://news.ycombinator.com/' 43 | hnnew = 'https://news.ycombinator.com/newest' # Must *not* have final / 44 | 45 | # Link blogs etc. 46 | trivium = 'http://leahneukirchen.org/trivium/' 47 | tbray='https://www.tbray.org/ongoing/' 48 | nelson='https://pinboard.in/u:nelson' 49 | 50 | # Kragen Sitaker's notes 51 | dercuano = 'https://dercuano.github.io/' 52 | derctuo = 'https://derctuo.github.io/' 53 | dernocua = 'https://dernocua.github.io/' 54 | 55 | # Reviving the Dillo browser 56 | dillo = 'https://dillo-browser.github.io/' 57 | dilloslides = 'https://dillo-browser.github.io/fosdem-2025/' 58 | 59 | # Simple page with text and two images with alt tags 60 | grug = 'https://grugbrain.dev/' 61 | 62 | # Big page 6100 words, 600 photos, for testing image tags. 63 | gorton = 'https://aresluna.org/the-hardest-working-font-in-manhattan/' 64 | 65 | # Guardian article with lots of text but also lots of clutter 66 | # Gets HTML page with 481 lines but it's almost all clutter. 67 | # Article text starts on line 472 near the end of the page, 68 | # has only a few lines from the start of the article then ends. 69 | # The page itself must load more pages. 70 | grauniad = 'https://www.theguardian.com/technology/2023/jul/25/joseph-weizenbaum-inventor-eliza-chatbot-turned-against-artificial-intelligence-ai' 71 | 72 | # These URLs raise errors 73 | 74 | # urllib.error.URLERROR: 75 | nohost = 'https:nowhere.com' 76 | 77 | # urllib.error.URLERROR: 78 | unrecognized = 'https://nowhere.com' 79 | 80 | # urllib.error.HTTPError: HTTP Error 404: Not Found 81 | notfound = 'https://jon-jacky.github.io/home/missing.html' 82 | 83 | -------------------------------------------------------------------------------- /threads/timers.py: -------------------------------------------------------------------------------- 1 | """ 2 | timers.py - functions to run in tasking experiments. 3 | 4 | This module is named timers (plural). It contains a function named 5 | timer (single). So we can say 'from timers import timer' and later 6 | 'reload timers' with no name clash. 7 | 8 | threads.txt explains how to do some experiments with these functions. 9 | """ 10 | 11 | import time, datetime, sys 12 | import edsel # used by etimer and ptimer 13 | 14 | # This timer writes to stdout, can be redirected 15 | 16 | def timer(n=1, delay=1.0, label=''): 17 | """ 18 | Sleep for given delay (default 1.0 sec), then print timestamp message. 19 | Repeat n times (default 1). Message includes time and optional label. 20 | Optional label for distinguishing output from different function calls. 21 | Calls print() to print output to stdout, which can be redirected. 22 | Calls print('...\n\r', end='') so it works in terminal char and line modes, 23 | """ 24 | for i in range(n): 25 | time.sleep(delay) 26 | # For now use print with default args, which adds the \n itself. 27 | print(f'{label} {i+1} {datetime.datetime.now()}\n\r', end='') 28 | 29 | # This timer uses print(..., destination=...) to write to any buffer. 30 | 31 | def ptimer(n=1, delay=1.0, label='', destination=sys.stdout): 32 | """ 33 | Similar to timer function above, but instead of print() to stdout, 34 | has an additional destination argument which can be any object with 35 | a method named write. destination can be a Writer object that 36 | writes to any editor buffer. Default destination writes to REPL. 37 | """ 38 | for i in range(n): 39 | time.sleep(delay) 40 | print(f'{label} {i+1} {datetime.datetime.now()}\n\r', end='', 41 | file=destination) 42 | 43 | vdelay = 1.0 # used by vtimer below, can be edited while vtimer is running 44 | vrun = True # used by vtimer, set False to exit before n runs out. 45 | 46 | def vtimer(n=1, delay=1.0, label='', destination=sys.stdout): 47 | 'Like ptimer above, but uses global vdelay that can be edited while running' 48 | global vdelay, vrun 49 | vdelay = delay 50 | vrun = True # in case it was set False on an earlier run 51 | for i in range(n): 52 | if not vrun: break 53 | time.sleep(vdelay) 54 | print(f'{label} {i+1} {datetime.datetime.now()}\n\r', end='', 55 | file=destination) 56 | 57 | class Timer(): 58 | """ 59 | Timer class, like vtimer fcn above but here delay and run are instance vars 60 | so we can control multiple timers independently 61 | """ 62 | def __init__(self): 63 | self.delay = 1.0 # can be edited while timer is running 64 | self.run = True # set False to exit before n runs out. 65 | 66 | def timer(self, n=1, delay=1.0, label='', destination=sys.stdout): 67 | """ 68 | destination must be a file-like object, must have a write method. 69 | """ 70 | self.delay = delay 71 | self.run = True 72 | for i in range(n): 73 | if not self.run: break 74 | time.sleep(self.delay) 75 | if destination == sys.stdout: # default 76 | print(f'{label} {i+1} {datetime.datetime.now()}\n\r', end='', 77 | file=destination) 78 | # We found that redirection with print(..., file=...) 79 | # does not work well with our Writer objects when threading. 80 | # Instead, just calling the object's write method does work. 81 | else: 82 | destination.write(f'{label} {i+1} {datetime.datetime.now()}\n\r') 83 | 84 | -------------------------------------------------------------------------------- /doc/term.txt: -------------------------------------------------------------------------------- 1 | 2 | Notes on terminals 3 | 4 | http://xn--rpa.cc/essays/term - everything you ever wanted to know 5 | about terminals (but were afraid to ask) by Lexi Summer Hale 6 | 7 | "so here's a short tutorial on ansi escape codes and terminal control, 8 | because you philistines won't stop using ncurses ..." 9 | 10 | "almost all UI changes in a terminal are accomplished through in-band 11 | signalling. these signals are triggered with the ASCII/UTF-8 character 12 | ‹ESC› (0x1B or 27). it's the same ‹ESC› character that you send to the 13 | terminal when you press the Escape key on your keyboard or a key 14 | sequence involving the Alt key. (typing ‹A-c› for instance sends the 15 | characters ‹ESC› and ‹c› in very rapid succession; this is why you'll 16 | notice a delay in some terminal programs after you press the escape 17 | key — it's waiting to try and determine whether the user hit Escape or 18 | an alt-key chord.)" 19 | 20 | "but hang on, where's that semicolon coming from? it turns out, ansi 21 | escape codes let you specify multiple formats per sequence. you can 22 | separate each command with a ;. this would allow us to write 23 | formatting commands like fmt(underline with bright with no italic), 24 | which translates into \x1b[4;1;23m at compile time." 25 | 26 | "to pick from a 256-color palette, we use a slightly different sort of 27 | escape: \x1b[38;5;(color)m to set the foreground and 28 | \x1b[48;5;(color)m to set the background, where (color) is the palette 29 | index we want to address. these escapes are even more unwieldy than 30 | the 8+8 color selectors, so it's even more important to have good 31 | abstraction." 32 | 33 | "of course, this is still pretty restrictive. 8-bit color may have 34 | been enough to '90s CD-ROM games on Windows, but it's long past it's 35 | expiration date. using true color is much more flexible. we can do 36 | this through the escape sequence \x1b[38;2;(r);(g);(b)m where each 37 | component is an integer between 0 and 255. 38 | 39 | sadly, true color isn't supported on many terminals, urxvt tragically 40 | included. for this reason, your program should never rely on it, and 41 | abstract these settings away to be configured by the user. defaulting 42 | to 8-bit color is a good choice, as every reasonable modern terminal 43 | has supported it for a long time now." 44 | 45 | "the first thing you should always do when writing a TUI application 46 | is to send the TI or smcup escape. this notifies the terminal to 47 | switch to TUI mode (the "alternate buffer"), protecting the existing 48 | buffer so that it won't be overwritten and users can return to it when 49 | your application closes." 50 | 51 | "now we've set the stage for our slick ncurses-free TUI, we just need 52 | to figure out how to put things on it. ..." 53 | 54 | "that's it for the tutorial. i hope you learned something and will 55 | reconsider using fucking ncurses next time ..." 56 | 57 | "my hope is that this tutorial will curtail some of the more 58 | egregiously trivial uses of ncurses ..." 59 | 60 | Lots of sample C code in this page demonstrating everything she 61 | discusses. 62 | 63 | One comment in https://news.ycombinator.com/item?id=18125167: 64 | "Sorry, unfortunately hardcoding escape sequences still won't work. 65 | The smcup this uses is the one for st and kitty, and it's not valid 66 | for xterm, iterm, konsole, vte (e.g. gnome-terminal),...." 67 | 68 | https://news.ycombinator.com/item?id=4545265 69 | C code sample using VMIN and VTIME "... Then if a read on your tty 70 | only returns the escape character, you know it was the escape key and 71 | not arrow keys or whatever.. It's fairly simple to do, but certainly 72 | hackish." 73 | -------------------------------------------------------------------------------- /doc/term.md: -------------------------------------------------------------------------------- 1 | 2 | Notes on terminals 3 | 4 | - everything you ever wanted to know 5 | about terminals (but were afraid to ask) by Lexi Summer Hale 6 | 7 | "so here's a short tutorial on ansi escape codes and terminal control, 8 | because you philistines won't stop using ncurses ..." 9 | 10 | "almost all UI changes in a terminal are accomplished through in-band 11 | signalling. these signals are triggered with the ASCII/UTF-8 character 12 | ‹ESC› (0x1B or 27). it's the same ‹ESC› character that you send to the 13 | terminal when you press the Escape key on your keyboard or a key 14 | sequence involving the Alt key. (typing ‹A-c› for instance sends the 15 | characters ‹ESC› and ‹c› in very rapid succession; this is why you'll 16 | notice a delay in some terminal programs after you press the escape 17 | key — it's waiting to try and determine whether the user hit Escape or 18 | an alt-key chord.)" 19 | 20 | "but hang on, where's that semicolon coming from? it turns out, ansi 21 | escape codes let you specify multiple formats per sequence. you can 22 | separate each command with a ;. this would allow us to write 23 | formatting commands like fmt(underline with bright with no italic), 24 | which translates into \x1b[4;1;23m at compile time." 25 | 26 | "to pick from a 256-color palette, we use a slightly different sort of 27 | escape: \x1b[38;5;(color)m to set the foreground and 28 | \x1b[48;5;(color)m to set the background, where (color) is the palette 29 | index we want to address. these escapes are even more unwieldy than 30 | the 8+8 color selectors, so it's even more important to have good 31 | abstraction." 32 | 33 | "of course, this is still pretty restrictive. 8-bit color may have 34 | been enough to '90s CD-ROM games on Windows, but it's long past it's 35 | expiration date. using true color is much more flexible. we can do 36 | this through the escape sequence \x1b[38;2;(r);(g);(b)m where each 37 | component is an integer between 0 and 255. 38 | 39 | sadly, true color isn't supported on many terminals, urxvt tragically 40 | included. for this reason, your program should never rely on it, and 41 | abstract these settings away to be configured by the user. defaulting 42 | to 8-bit color is a good choice, as every reasonable modern terminal 43 | has supported it for a long time now." 44 | 45 | "the first thing you should always do when writing a TUI application 46 | is to send the TI or smcup escape. this notifies the terminal to 47 | switch to TUI mode (the "alternate buffer"), protecting the existing 48 | buffer so that it won't be overwritten and users can return to it when 49 | your application closes." 50 | 51 | "now we've set the stage for our slick ncurses-free TUI, we just need 52 | to figure out how to put things on it. ..." 53 | 54 | "that's it for the tutorial. i hope you learned something and will 55 | reconsider using fucking ncurses next time ..." 56 | 57 | "my hope is that this tutorial will curtail some of the more 58 | egregiously trivial uses of ncurses ..." 59 | 60 | Lots of sample C code in this page demonstrating everything she 61 | discusses. 62 | 63 | One comment in : 64 | "Sorry, unfortunately hardcoding escape sequences still won't work. 65 | The smcup this uses is the one for st and kitty, and it's not valid 66 | for xterm, iterm, konsole, vte (e.g. gnome-terminal),...." 67 | 68 | 69 | C code sample using VMIN and VTIME "... Then if a read on your tty 70 | only returns the escape character, you know it was the escape key and 71 | not arrow keys or whatever.. It's fairly simple to do, but certainly 72 | hackish." 73 | -------------------------------------------------------------------------------- /threads/threads_2.txt: -------------------------------------------------------------------------------- 1 | threads_2.txt 2 | 3 | Experiments with Python threading using the pmacs editor with timers.py 4 | and writer.py, with our custom pysh REPL from pyshell.py. 5 | 6 | In this demonstration, we show two threads rapidly updating two buffers 7 | in two windows, while we type at our pysh REPL to control the threads. 8 | 9 | We need our custom pysh REPL for this demo so our windowing code can 10 | restore the cursor to the correct location in the REPL command line as we 11 | type, after it updates each of the windows. This is not possible with 12 | the standard Python REPL. 13 | 14 | It is easiest to explain by doing this demonstration. Just 15 | follow along, typing the statements here as you go. 16 | 17 | (Alternatively, you can run the script in threads_2.py - see directions 18 | at the end of this file.) 19 | 20 | At the system command prompt (often $), define the PYTHONPATH, so this 21 | demo works in any directory. 22 | 23 | ...$ . ~/Piety/bin/paths 24 | 25 | Run the tm script (not pm) which imports the editor, timer, and writer 26 | modules, opens a window, and starts the pysh REPL: 27 | 28 | ...$ python3 -im tm 29 | 30 | At the pysh prompt >> (not the standard Python prompt >>>), 31 | create two buffers in two windows: 32 | 33 | >> e('a.txt') 34 | a.txt, 0 lines 35 | >> o2() 36 | >> e('b.txt') 37 | b.txt, 0 lines 38 | 39 | Now we have two windows. Create two Timer objects, and two Writer objects 40 | that the threads will use to redirect thread output to the windows: 41 | 42 | >> ta = Timer() 43 | >> tb = Timer() 44 | 45 | >> abuf = Writer('a.txt') 46 | >> bbuf = Writer('b.txt') 47 | 48 | Now start the A thread, that prints 1000 messages at 1 second 49 | intervals, redirected by the abuf Writer to the a.txt window: 50 | 51 | >> Thread(target=ta.timer,args=(1000,1,'A',abuf)).start() 52 | 53 | The a.txt window updates with a new timer message every second, 54 | but we can still type in the pysh REPL to start another thread: 55 | 56 | >> Thread(target=tb.timer,args=(1000,1,'B',bbuf)).start() 57 | 58 | Now the b.txt window is also updating at 1/sec. We can type commands to 59 | change the speed of a timer by updating its delay attribute: 60 | 61 | >> tb.delay = 0.5 62 | 63 | Now the b.txt window updates 2/sec. We can still type at the pysh REPL 64 | 65 | >> ta.delay = 0.1 66 | 67 | Now the a.txt window updates 10/sec. 68 | 69 | We can still type at the pysh REPL. Experiment with speeding up and 70 | slowing down the updates in both windows. How fast can you go and still 71 | type at the REPL without losing control of the cursor? 72 | 73 | Call the threads function to list the running threads: 74 | 75 | >> threads() 76 | [<_MainThread(MainThread, started 547740599488)>, , ] 79 | 80 | Each thread exits after it prints 1000 messages, as we specified in our 81 | Thread(...) command. Or, you can stop a thread at any time: 82 | 83 | >> ta.run = False 84 | >> threads() 85 | [<_MainThread(MainThread, started 547740599488)>, ] 87 | 88 | Both threads (other than MainThread) must be stopped before you can 89 | exit the pysh and standard Python REPLs without using ctrl-C to 90 | interrupt them. 91 | 92 | As an alternative to typing the commands in this page, you can run 93 | the script in threads_2.py. First at the system prompt type the command to 94 | set the PYTHONPATH: ...$ . ~/Piety/bin/paths then type the command 95 | to open a window and start the pysh REPL: ...$ python3 -im tm 96 | Then at the pysh prompt type >> import threads_2 to open two windows 97 | and start both threads, then >> from threads_2 import * to enable commands 98 | to display threads >> threads() and control them >> ta.delay = 0.5 etc. 99 | 100 | 101 | Revised May 2024 102 | 103 | 104 | 105 | -------------------------------------------------------------------------------- /piety/README.md: -------------------------------------------------------------------------------- 1 | 2 | piety 3 | ===== 4 | 5 | The *piety* script in this directory begins a Piety session by starting 6 | an *asyncio* event loop. Other scripts demonstrate Piety features 7 | or start applications. 8 | 9 | Piety provides concurrency with a Python *asyncio* event loop. Tasks 10 | are implemented by Python *coroutines* or *readers* (event handlers) that 11 | run in an event loop. 12 | 13 | Piety provides readers for *pysh*, its custom Python shell, and *pmacs*, its 14 | Emacs-like editor. These enable the shell and the editor to run without 15 | blocking in an event loop, so other tasks can run concurrently, as you 16 | type commands in the shell or edit text in the editor. You can control 17 | other tasks from the shell and display task output in editor windows. 18 | 19 | The *piety* script creates 20 | an event loop named *piety*, adds the readers for the shell and the 21 | editor, and starts the event loop with the shell running. It also imports 22 | a function *run* which is needed to run other scripts in the event loop. 23 | 24 | This directory contains several scripts that start applications or 25 | demonstrate Piety features. *pmacs_script.py* is the most recent 26 | and shows the most features. 27 | 28 | These scripts cannot run standalone. First 29 | you must run the *piety* script to start a Piety session, then run the 30 | script within the session using the *run* function. For example, to 31 | start the *pmacs* editor: *run('apm.py')* 32 | 33 | All of these scripts assume you have already assigned *PYTHONPATH* by 34 | running this command: 35 | 36 | . ~/Piety/bin/paths 37 | 38 | The initial dot . in this command is essential. This command assumes 39 | the top level *Piety* directory is in your home directory. 40 | 41 | The *eventloop* script in this directory creates an *asyncio* event 42 | loop but does not start it. That is more convenient in some contexts. 43 | Contrast *vpmacs_script.py* which uses *piety.py* to *v2pmacs_script.py* 44 | which uses *eventloop.py* 45 | 46 | ### Files ### 47 | 48 | - **apm.py**: Script to start the *pmacs* editor in an *asyncio* event loop. 49 | 50 | - **apmacs.py**: Adapt the *pmacs* editor to run in an *asyncio* event loop. 51 | 52 | - **apyshell.py**: Adapt the *pysh* custom Python shell to run in an *asyncio* 53 | event loop. 54 | 55 | - **atimer_script.py**: Demonstrate the Python shell and timer tasks interleaving 56 | in the Piety event loop. 57 | 58 | - **atimer_script.txt**: Explanation and directions for *atimer_script.py*. 59 | 60 | - **edsel_script.py**: Display interleaving timer tasks in two editor windows. 61 | Set the timer intervals and stop the tasks from the Python REPL. 62 | 63 | - **edsel_script.txt**: Explanation and directions for *edsel_script.py*. 64 | 65 | - **eventloop.py**: Creates the Piety *asyncio* event loop but does 66 | not start it. 67 | 68 | - **piety.py**: Begin a Piety session by starting the *asyncio* event loop. 69 | Import a function *run* which is needed to run other scripts in the event loop. 70 | 71 | - **piety.txt**: Explanation and directions for *piety.py*. 72 | 73 | - **pmacs_blocking.md**: Explanation and directions for demonstrating 74 | cooperative multitasking and blocking using *pmacs_script.py*. 75 | 76 | - **pmacs_script.md**: Explanation and directions for *pmacs_script.py*. 77 | Includes a screenshot. 78 | 79 | - **pmacs_script.py**: Edit in one window while a timer task updates the other. 80 | Set the timer interval, and stop and start timer tasks from the 81 | Python REPL. 82 | 83 | - **vedsel_script.md**: Explanation of *vedsel_script.py*. 84 | Includes a screenshot. 85 | 86 | - **vedsel_script.py**: Similar to *edsel_script.py*, but in the Piety 87 | desktop with the viewer window. See instructions in comment header. 88 | 89 | - **vpmacs_script.py**: Similar to *pmacs_script.py*, but in the Piety 90 | desktop with the viewer window. See instructions in comment header. 91 | 92 | - **v2pmacs_script.py**: Similar to *vpmacs_script.py*, but uses 93 | *eventloop.py* not *piety.py*. See instructions in comment header. 94 | 95 | 96 | Revised Sep 2025 97 | 98 | -------------------------------------------------------------------------------- /piety/pmacs_blocking.md: -------------------------------------------------------------------------------- 1 | 2 | Cooperative multitasking and blocking 3 | ===================================== 4 | 5 | *pmacs_script.py* can demonstrate some key features of Piety: *cooperative 6 | multitasking* and an unwelcome potential consequence, *blocking*. 7 | 8 | Piety provides concurrency with a Python *asyncio* event loop. This 9 | provides *cooperative multitasking*. After an event -- a timeout or a 10 | keystroke -- a task or reader runs code that executes briefly, then exits by 11 | executing *return* or *yield*. Then the system can respond to another event, 12 | and another task or reader can run. In this way, multiple tasks and readers can 13 | interleave -- if each reader and task cooperates by yielding control 14 | promptly. If any task or reader executes code that runs for a long time, or 15 | *blocks* -- waits for an event that has not yet occurred -- no other tasks 16 | can run, and the system stops responding to events -- the whole session 17 | is *blocked*. Tasks and readers that run in an event loop should be coded 18 | so blocking does not occur. 19 | 20 | Usually, the *pmacs* editor and the *pysh* shell are *non-blocking*. Code 21 | that reads input from the keyboard is called from a *reader* that is only 22 | called by the event loop when data is ready, after a key is typed. 23 | The *piety.add_reader()* call in *piety.py* sets this up. 24 | 25 | Code called from a reader should be non-blocking - it should quickly handle 26 | a single keystroke, then exit. In contrast, the Python builtin function 27 | *input*, which reads strings from a sequence of keystrokes that the user 28 | types at the terminal, blocks for the entire time that the user is typing -- 29 | or thinking -- until they type *enter* to complete the string. The standard 30 | library *readline* function blocks in the same way. 31 | 32 | It is easy to demonstrate blocking with *pmacs_script.py*. Just run the 33 | script in the usual way: 34 | 35 | ...$ python3 -m piety 36 | >>>> run('pmacs_script.py') 37 | ... 38 | 39 | Now the two windows appear, with timer messages appearing in the upper window, 40 | and an editing cursor in the lower window. Type *M-x* to put the cursor 41 | at the *>>>>* prompt, and call *input*: 42 | 43 | >>>> input('Input: ') 44 | Input: 45 | 46 | *input* prints the prompt, and waits for you to type a string. 47 | The messages stop appearing in the timer window. The session is blocked. 48 | Now type any string at the prompt, then type *enter*. 49 | 50 | >>>> input('Input: ') 51 | Input: anything 52 | 'anything' 53 | >>>> 54 | 55 | The Python interpreter prints the returned value as usual -- it is the 56 | string you typed -- and messages resume appearing in the timer window. The 57 | session is unblocked. 58 | 59 | Most code in *pmacs* and *pysh* avoids calling *input* or *readline*. 60 | Instead, it calls non-blocking functions from our *editline* module, which 61 | each handle one keystroke, building up strings one character at a time. One 62 | of the reasons we wrote a custom editor and a custom Python shell for Piety is to 63 | ensure that these utilities are non-blocking, so they can interleave with 64 | other tasks in an event loop. 65 | 66 | However, at this time *pmacs* still includes some blocking code. After you 67 | type *C-s* to enter a search string,, or *C-x b* to switch to another 68 | buffer, or *C-x C-f* to load a file, *pmacs* prints a prompt, then calls 69 | *input* to read the search string, buffer name, or file name. The session 70 | blocks while you are typing, until you type *enter* to complete the string, 71 | or type *???* to cancel the operation. This is easy to demonstrate 72 | in a *pmacs_script.py* session. It could be fixed by replacing 73 | that *input* with calls to our *editline* functions. 74 | 75 | *pmacs* also blocks when there is a multi-key command, 76 | such as *C-c >* to indent. After you type *C-c*, *pmacs* blocks 77 | in a call to *terminal.getchar*, waiting for you to 78 | type *>*. This is also easy to demonstrate in a *pmacs_script.py* 79 | session. It could be fixed by exiting fron the reader and returning control 80 | to the event loop after the first key in a multi-key command, instead 81 | of waiting at *getchar* for the next key. 82 | 83 | Revised Aug 2024 84 | 85 | -------------------------------------------------------------------------------- /unix/shell.py: -------------------------------------------------------------------------------- 1 | """ 2 | shell.py - Python functions that wrap shell commands, so you can invoke 3 | the shell without exiting the Python session or using the 4 | host desktop. 5 | 6 | This module provides a function sh(command), which runs any shell command, 7 | and functions for particular shell commands: pwd, cd, man, ls -C, ls -l, ls -lt 8 | 9 | The pwd and cd functions call specific functions in the Python standard library. 10 | The other functions run the host shell in a subprocess. 11 | 12 | These functions all write shell command output on the standard output, so the 13 | output can be redirected anywhere. 14 | """ 15 | 16 | import subprocess, os, pydoc 17 | import contextlib # redirect_stdout stores intermediate results in StringIO 18 | import io # StringIO 19 | 20 | width = 80 # width of command output for ls() and man() commands 21 | 22 | def sh(command, shellenv=None): 23 | """ 24 | Invoke shell command, a string, in a shell subprocess. 25 | shellenv is an optional environment to be used by the shell subprocess. 26 | See https://docs.python.org/3/library/subprocess.html 27 | Capture command output and print it on the calling process standard output, 28 | so 'with redirect_stdout' works. 29 | Call print on each line of output, to work with our writer module. 30 | """ 31 | cp = subprocess.run(command, shell=True, text=True, capture_output=True, 32 | env=shellenv) 33 | if cp.stdout: 34 | for line in cp.stdout.splitlines(): 35 | print(line) 36 | if cp.stderr: 37 | for line in cp.stderr.splitlines(): 38 | print(line) 39 | 40 | def cd(path): 41 | """ 42 | Change current directory to path, a string 43 | Uses os module chdir function so it does change the directory 44 | of the Python session. 45 | Invoking cd in a shell subprocess by sh('cd ...') does not. 46 | """ 47 | os.chdir(path) 48 | 49 | def pwd(): 50 | """ 51 | Print current working directory on stdout. 52 | Uses os module getcwd function, not a shell subprocess. 53 | """ 54 | print(os.getcwd()) # returns a string 55 | 56 | def man(topic): 57 | """ 58 | Print man page on topic, a string. 59 | Invokes the man command in a shell subprocess, writes output on stdout. 60 | """ 61 | manenv = os.environ.copy() 62 | manenv['MANWIDTH'] = str(width) 63 | sh('man ' + topic, shellenv=manenv) 64 | 65 | def ls(path='.'): 66 | """ 67 | Call the shell directory listing command ls -C for a compact listing. 68 | Argument is file or directory path string, default . the current directory. 69 | Invokes the ls command in a shell subprocess, writes output on stdout. 70 | """ 71 | sh(f'ls -C -w {width} ' + path) 72 | 73 | def lslXXX(path='.'): 74 | """ 75 | SUPERCEDED BY lsl DERIVED FROM lstfx BELOW 76 | Call the shell directory listing command ls -l for a long form listing, 77 | sorted alphabetically. 78 | Argument is file or directory path string, default . the current directory. 79 | Invokes the ls command in a shell subprocess, writes output on stdout. 80 | """ 81 | sh('ls -l '+path) 82 | 83 | def lsltXXX(path='.'): 84 | """ 85 | SUPERCEDED BY lslt DERIVED FROM lstfx BELOW 86 | Call the shell directory listing command ls -lt for a long form listing, 87 | sorted most recent first. 88 | Argument is file or directory path string, default . the current directory. 89 | Invokes the ls command in a shell subprocess, writes output on stdout. 90 | """ 91 | sh('ls -lt '+path) 92 | 93 | def lslxf(path='.', cmd='ls -l'): 94 | """ 95 | lsl with eXtra Formatting. 96 | In each line, prefix filename at the end with its path (from path= arg). 97 | 98 | Call cmd, a shell directory listing command such as 'ls -l' or 'ls -lt' 99 | for a long form listing, default cmd is 'ls -l' to sort alphabetically. 100 | path is file or directory path string, default '.' the current directory. 101 | Invokes the ls command in a shell subprocess, writes output on stdout. 102 | """ 103 | lslines = io.StringIO('') 104 | with contextlib.redirect_stdout(lslines): 105 | sh(cmd+' '+path) 106 | for line in lslines.getvalue().splitlines(): 107 | stats, spc, fname = line.rpartition(' ') 108 | pathstr = '' if path == '.' else path + '/' 109 | print(stats + spc + pathstr + fname) 110 | 111 | def lsl(path='.'): lslxf(path, 'ls -l') 112 | 113 | def lslt(path='.'): lslxf(path, 'ls -lt') 114 | 115 | -------------------------------------------------------------------------------- /vt_terminal/keyseq.txt: -------------------------------------------------------------------------------- 1 | 2 | keyseq.txt 3 | ========== 4 | 5 | Notes on keyseq.py and keyseq_1.py 6 | 7 | Both the keyseq and keyseq_1 modules contain a keyseq function. The function 8 | bodies are different in the two modules. We expect to use keyseq from now 9 | on. We are keeping the older keyseq_1 just for reference. 10 | 11 | Both keyseq functions construct keycodes from sequences of one or more 12 | characters. 13 | 14 | There are three kinds of keycode sequences: 15 | 16 | 1. A single printable character or a single control character, delivered 17 | by a single keystroke. For example, the 'a' character send by typing the a key, 18 | or the C-a (control-a or ^A character) sent by holding down the control key 19 | while typing the a key. 20 | 21 | 2. A sequence begining with a single control character, followed by one 22 | or more control characters or printing characters, each delivered by its own 23 | keystroke. Many emacs-like editor commands have this form: C-x C-f etc. 24 | 25 | 3. Escape sequences, which begin with a single esc character, followed by 26 | one or more characters, which are all delivered by a single keystroke. For 27 | example emacs-like editor commands like M-f (meta f, formed by holding down 28 | the alt key while typing the f key, which causes the terminal to send esc 29 | followed by 'f'). Also, ANSI escape sequences, which are used to control 30 | the terminal, which consist of the esc character, then the [ character, then 31 | additional characters. Pressing a keyboard arrow key moves the cursor on 32 | the terminal and causes the terminal to send an ANSI escape sequence. 33 | 34 | Both keyseq functions take one character as an argument. This argument might 35 | be a single complete keycode. In that case, both keycode functions simply return 36 | that same character. 37 | 38 | Or, the argument to a keyseq might be the first character - the prefix - 39 | of a multi-character sequence. In that case, both keyseq functions accumulate 40 | the characters in the sequence and return the complete sequence when it is 41 | finally available. 42 | 43 | We call keyseq_1.keyseq once with each character in the sequence. It returns 44 | the empty string '' until the sequence is complete, and then it returns 45 | the entire sequence. So we have to call keyseq_1.keyseq multiple times to 46 | collect the whole sequence. If it returns '', our applications do not process 47 | the returned value. Returning after each character is supposed to prevent the 48 | calling application program from blocking, waiting for the complete sequence 49 | to appear. 50 | 51 | We call keyseq.keyseq once with the first character in the sequence. If that is 52 | the complete sequence, keyseq.keyseq returns it. If it is a prefix, keyseq.keyseq 53 | itself reads more characters until the sequence is complete, and then it returns 54 | the entire sequence. So we only have to call keyseq.keyseq once, with the 55 | first character in the sequence. keyseq.keyseq always returns the complete 56 | sequence, which our applications can then process. To prevent the 57 | application program from blocking, keyseq.keyseq should only be called when 58 | the entire sequence is available to be read (in some input buffer, 59 | presumably). 60 | 61 | Both keyseq modules work in ordinary (synchronous) Python code. 62 | 63 | Only the newer keyseq module works in our applications when keys are 64 | processed by a reader invoked from the Python asyncio event loop. The 65 | older keyseq_1 does not work in our applications when invoked from the event 66 | loop. 67 | 68 | We found that the event loop only invokes the reader after every keystroke 69 | on the keyboard. When a keystroke sends multiple characters, as it does for 70 | escape sequences (editor M- commands and ANSI escape sequences), the event 71 | loop only invokes the reader once, so keyseq must then read all of those 72 | characters and return the entire sequence for downstream processing 73 | by our applications. This is what keyseq.keyseq does. 74 | 75 | We wrote keyseq_1 with the assumption that the event loop would invoke the 76 | reader on every character in a multicharacter sequence. This assumption is 77 | not true for escape sequences. Each call to keycode_1.keycode only processes 78 | a single character, so it does not work with escape sequences and the 79 | event loop in our applications. 80 | 81 | The two keyseq modules are very similar. The body of keyseq.keyseq only 82 | contains a few more lines than keyseq_1.keyseq in order to keep reading 83 | characters to the end of the sequence. 84 | 85 | NOTE: at this time keyseq.keyseq *does* block waiting for subsequent characters 86 | while reading editing commands like C-x C-f, which use use a keystroke for 87 | each character. This is a BUG which we expect to fix, by simply returning '' 88 | for prefix characters, like keyseq_1. Our application programs still 89 | recognize '' as an indicator that the sequence is not yet complete. 90 | We are leaving the bug in the keyseq code for now so we can observe its 91 | effects. 92 | 93 | Revised Jun 2024 94 | 95 | -------------------------------------------------------------------------------- /vt_terminal/display.py: -------------------------------------------------------------------------------- 1 | """ 2 | display - Update the terminal display using ANSI control sequences. 3 | """ 4 | 5 | # This putstr always writes to display even when stdout is redirected 6 | 7 | import os 8 | 9 | ttyname = os.ctermid() # usually returns '/dev/tty' 10 | tty = open(ttyname, 'w') 11 | 12 | # Differs from terminal.putstr which writes to stdout and might be redirected 13 | def putstr(s): 14 | """ 15 | Print string (can be just one character) on display with no 16 | formatting (unlike plain Python print). Flush to force output immediately. 17 | If you want newline, you must explicitly include it in s. 18 | Always print to tty (terminal) device even when stdout is redirected. 19 | """ 20 | print(s, end='', flush=True, file=tty) 21 | 22 | esc = '\x1B' # \e does not work 'invalid \x escape' 23 | csi = esc+'[' # ANSI control sequence introducer 24 | 25 | cha = csi+'%dG' # cursor horizontal absolute, column %d 26 | cub = csi+'D' # cursor backward (left), default 1 char 27 | cuf = csi+'C' # cursor forward (right), default 1 char 28 | cup = csi+'%d;%dH' # cursor position %d line, %d column 29 | dch = csi+'%dP' # delete chars, remove %d chars at current position 30 | ed = csi+'J' # erase display from cursor to end 31 | eu = csi+'1J' # erase display from top to cursor 32 | el = csi+'%dK' # erase in line, %d is 0 start, 1 end, or 2 all 33 | el_end = el % 0 # 0: erase from cursor to end of line 34 | el_begin = el % 1 # 1: erase from beginning of line to cursor 35 | el_all = el % 2 # 2: erase entire line 36 | ich = csi+'%d@' # insert chars, make room for %d chars at current position 37 | decstbm = csi+'%d;%dr' # DEC Set Top Bottom Margins (set scrolling region 38 | # %d,%d is top, bottom, so 23;24 is bottom two lines 39 | # then it sets cursor at the top of the page 40 | decstbmn = csi+';r' # decstbm default: set scrolling region to full screen 41 | 42 | sgr = csi + '%s' + 'm' # set graphic rendition. %s is ;-separated integers like 43 | # bold+inverse: esc[0;1;7m by sgr % ';'.join('017') 44 | 45 | # sgr, attribute values, see http://www.inwap.com/pdp10/ansicode.txt 46 | clear = 0 # clears attributes (not transparent!) 47 | white_bg = 47 # gray on mac terminal 48 | bold = 1 49 | blink = 5 50 | reverse = 7 # reverse video, 'negative image' 51 | 52 | def attrs(*attributes): 53 | """ 54 | Convert variable length arg list of integers to ansi attributes string 55 | Then the sgr control sequence is just sgr % attrs(*attributes) 56 | """ 57 | return ';'.join([ str(i) for i in attributes ]) 58 | 59 | def render(text, *attributes): 60 | """ 61 | Print text with one or more attributes, each given by separate int arg, 62 | then clear attributes, but do not print newline. 63 | """ 64 | putstr(sgr % attrs(*attributes) + text + sgr % attrs(clear)) 65 | 66 | # used by line 67 | 68 | def insert_char(key): 69 | 'Insert character in front of cursor' 70 | putstr((ich % 1) + key) # open space to insert char 71 | 72 | def insert_string(string): 73 | putstr((ich % len(string)) + string) 74 | 75 | def delete_char(): 76 | 'Delete character under the cursor' 77 | putstr(dch % 1) 78 | 79 | def delete_nchars(n): 80 | 'Delete n characters under, then after the cursor' 81 | putstr(dch % n) 82 | 83 | def delete_backward_char(): 84 | 'Delete character before cursor' 85 | putstr(cub + dch % 1) 86 | 87 | def forward_char(): 88 | putstr(cuf) # move just one char 89 | 90 | def backward_char(): 91 | putstr(cub) 92 | 93 | def move_to_column(column): 94 | putstr(cha % column) 95 | 96 | # line also uses kill_line, defined below 97 | 98 | # used by edsel and window, they also use render (above) 99 | 100 | def erase(): # erase_display in gnu readline, erase from cursor to end 101 | putstr(ed) 102 | 103 | def erase_above(): # erase from top of display to cursor 104 | putstr(eu) 105 | 106 | def put_cursor(line, column): # not in emacs or gnu readline 107 | putstr(cup % (line, column)) 108 | 109 | def kill_line(): 110 | 'Erase from cursor to end of line' 111 | putstr(el_end) 112 | 113 | def kill_whole_line(): 114 | 'Erase entire line' 115 | putstr(el_all) 116 | 117 | def discard(): 118 | 'Erase from beginning of line to cursor' 119 | putstr(el_begin) 120 | 121 | def set_scroll(ltop, lbottom): 122 | 'Set scrolling region to lines ltop through lbottom (line numbers)' 123 | putstr(decstbm % (ltop, lbottom)) 124 | 125 | def set_scroll_all(): 126 | 'Set scrolling region to entire display' 127 | putstr(decstbmn) 128 | 129 | def put_render(line, column, text, *attributes): 130 | """ 131 | At line, column, print text with attributes 132 | but without newline, then clear attributes. 133 | """ 134 | put_cursor(line, column) 135 | putstr(sgr % attrs(*attributes) + text + sgr % attrs(clear)) 136 | 137 | def next_line(): 138 | 'replacement for print() in terminal char mode, explicitly sends \n\r' 139 | putstr('\n\r') 140 | -------------------------------------------------------------------------------- /piety/pmacs_script.md: -------------------------------------------------------------------------------- 1 | 2 | pmacs_script 3 | ============ 4 | 5 | *pmacs_script.py* demonstrates several Piety features. 6 | 7 | *pmacs_script.py* shows our custom Python shell *pysh* and our Emacs-like 8 | editor *pmacs* running without blocking in a Python *asyncio* event loop, 9 | while a timer task runs concurrently, as you type commands in the shell 10 | or edit text in the editor. Timer messages appears in an editor window, and you 11 | can control the speed of the timer from the shell -- or stop it and 12 | create another one. 13 | 14 | Just follow these directions: 15 | 16 | First you must assign *PYTHONPATH* by running this command at the system 17 | command prompt: 18 | 19 | ...$ . ~/Piety/bin/paths 20 | 21 | The initial dot . in this command is essential. This command assumes 22 | the top level *Piety* directory is in your home directory. 23 | 24 | Then make sure you are in the *~/Piety/piety* directory. This is needed 25 | for the Piety *run* command. 26 | 27 | ...$ cd ~/Piety/piety 28 | 29 | Run the *piety.py* script at the system command prompt to start the 30 | Piety event loop: 31 | 32 | ...$ python3 -m piety 33 | >>>> 34 | 35 | Our *pysh* custom Python interpreter command prompt appears: *>>>>*. It has 36 | four darts, not three, to distinguish it from the standard Python prompt. It 37 | works much like the standard Python interpreter. Any Python statement should 38 | work as usual. You can use all the same inline editing commands, and retrieve 39 | commands from its history. 40 | 41 | Confirm that the Piety event loop is running. The name of this event loop is also 42 | *piety*: 43 | 44 | >>>> piety 45 | <_UnixSelectorEventLoop running=True closed=False debug=False> 46 | 47 | Confirm that no tasks are yet running. The *pysh* shell is not a task, it is 48 | just a *reader* (a keyboard event handler): 49 | 50 | >>>> asyncio.all_tasks(piety) 51 | set() 52 | 53 | Run *pmacs_script* to start the *pmacs* editor and the timer task. If your 54 | default directory is not *~/Piety/piety*, you must type the path to that 55 | directory here: 56 | 57 | >>>> run('pmacs_script.py') 58 | a.txt, 0 lines 59 | 60 | Two editor windows appear in the terminal. The upper window, showing the 61 | *pmacs* editor buffer *a.txt*, shows messages written by the timer task 62 | appearing once per second. The lower window, showing editor buffer 63 | *scratch.txt*, is empty, with a cursor at the beginning of the first line. 64 | 65 | Type some text into the lower window and confirm that the timer messages continue 66 | appearing, without interfering with your typing. The *pmacs* editor works as 67 | usual. 68 | 69 | There is a *pysh* prompt *>>>>* at the bottom of the terminal window. To 70 | put the cursor there, type the command *M-x* ("meta x") by holding down the 71 | keyboard *alt* key while you type the *x* key. 72 | 73 | pmacs_script.py running in a terminal> 75 | 76 | Confirm the timer task is running. 77 | 78 | >>>> asyncio.all_tasks(piety) 79 | { wait_for=>} 82 | 83 | Again, the *pmacs* editor is not a task, it is just a *reader*. 84 | 85 | Examine the timer object and confirm that the timer interval is one second: 86 | 87 | >>>> ta 88 | 89 | >>>> ta.delay 90 | 1 91 | 92 | Set the timer interval to 0.1 to print messages ten times a second. 93 | 94 | >>>> ta.delay = 0.1 95 | 96 | See the messages appear rapidly in the *a.txt* window. Let the task 97 | run until 1000 messages appear. Then it stops. Confirm that the task 98 | has exited. You can just type the up-arrow key or C-p (*control p*, hold 99 | down the *ctrl* key while typing *n*) to retrieve the previous *all_tasks* command. 100 | 101 | >>>> asyncio.all_tasks(piety) 102 | set() 103 | 104 | Type this command to start another timer task. It writes 1000 messages at 105 | 1 second intervals to *a.txt*, all labelled *A*. 106 | 107 | >>>> piety.create_task(ta.atimer(1000,1,'A',abuf)) 108 | > 110 | 111 | Messages again appear in the *a.txt* window. 112 | 113 | To resume editing, type this command: 114 | 115 | >>>> apm() 116 | 117 | Now the cursor returns to the *scratch.txt* widow, at the location where you 118 | left it. You can alternate *M-x* and *apm()* commands to swtich between 119 | typing Python commands to *pysh* and editing text in the window. 120 | 121 | It is instructive to see how short you can make *ta.delay* -- how rapidly you 122 | can write messages in the timer window -- while still being able to edit 123 | or type commands, without scrambling any text. 124 | 125 | You can stop the task before it writes 1000 messesages by assigning 126 | *ta.run = False*: 127 | 128 | >>>> asyncio.all_tasks(piety) 129 | { wait_for=>} 132 | >>>> ta.run 133 | True 134 | >>>> ta.run = False 135 | >>>> asyncio.all_tasks(piety) 136 | set() 137 | 138 | To finish this experiment, type the *clr()* command to resume scrolling in 139 | the terminal so the windows will scroll out of sight. Then type *C-d* (hold 140 | down *ctrl* while typing *d*) at the prompt to exit from Piety back to the 141 | system command interpreter. 142 | 143 | >>>> clr() 144 | >>>> (C-d, does not echo) 145 | .... $ 146 | 147 | Instead of *C-d*, you can also type *exit()*. 148 | 149 | Revised Aug 2024 150 | 151 | -------------------------------------------------------------------------------- /tasking/pyshell.py: -------------------------------------------------------------------------------- 1 | """ 2 | pyshell.py - Custom Python REPL that uses our editcommand function 3 | instead of std input function. 4 | 5 | We need this to support tasking. It enables other tasks writing to 6 | the terminal to interleave with typing characters at our REPL, and enables us 7 | to restore the cursor to the correct position in the command line after 8 | another task moves it. 9 | 10 | pyshell defines the pysh function, which is the actual custom REPL. 11 | The module and the function have different names so we can 'import pyshell' 12 | then 'from pyshell import pysh' then 'reload pyshell' without name conflict. 13 | 14 | 'pyshell' is pronounced pie shell. 'pysh' rhymes with fish. 15 | """ 16 | 17 | import terminal_util, terminal, key, keyseq, display 18 | import editcommand as el #we renamed editline to editcommand but keep el abbrev., 19 | import pmacs 20 | # import edsel # NOT! This pyshell module might be used without edsel, so we 21 | # duplicate tlines and restore_cursor_to_cmdline from edsel here. 22 | 23 | from pycall import pycall # uses Python library code.InteractiveConsole 24 | 25 | cmd = '' # Python command 26 | point = 0 # index of cursor in cmd 27 | running = True # pysh main loop is running, set False to exit 28 | continuation = False # True when Python continuation line expected 29 | tlines = 24 # N of lines in frame, including all windows. Copied from edsel. 30 | 31 | ps1 = '>> ' # first line prompt, different from CPython >>> 32 | ps2 = '.. ' # continuation line prompt 33 | prompt = ps1 # initally. prompt is global so we can inspect it in the REPL. 34 | start_col = 3 # index of start of cmd on line, allowing for prompt ps1 or ps2 35 | 36 | # prompt and continuation are global so we can read them in the REPL 37 | 38 | history = [''] # list of command strings, most recent at index 0 39 | i_cmd = -1 # integer index into history, code will assign to 0 or greater 40 | max_cmds = 100 # maximum number of commands in history. 20 is not enough! 41 | 42 | # cmd_mode is needed to restore terminal cursor after it is used by a task. 43 | cmd_mode = True # True in Python REPL, False when editing in display window. 44 | 45 | def tpm(): 46 | """ 47 | tpm "tasking pmacs" 48 | From the pysh Python REPL, use the tpm() command to clear the cmd_mode flag 49 | and begin the pmacs editor for editing buffers display windows. 50 | Exit from display editing with M-x: set cmd_mode flag and return to REPL. 51 | tpm() must be issued from pysh REPL not standard Python REPL 52 | because it assumes terminal is already in char mode. 53 | """ 54 | global cmd_mode 55 | cmd_mode = False 56 | pmacs.rpm() # raw pmacs - assumes terminal is already in char mode 57 | cmd_mode = True 58 | 59 | def setup(): 60 | """ 61 | Set up terminal before entering pysh main loop. 62 | """ 63 | global tlines, running, continuation 64 | terminal.set_char_mode() 65 | tlines, _ = terminal_util.dimensions() # Copied from edsel 66 | display.putstr(ps1) # ps1 is >> prompt 67 | el.refresh(cmd, point, start_col) # following prompt on same line 68 | running = True # previous exit() or C_d may have set it False 69 | continuation = False # True when continuation line expected 70 | 71 | def restore(): 72 | """ 73 | Restore terminal after exiting pysh main loop. 74 | """ 75 | terminal.set_line_mode() 76 | print() # advance to next line for Python prompt 77 | 78 | # Copied from edsel 79 | def restore_cursor_to_cmdline(): 80 | display.put_cursor(tlines, 1) 81 | 82 | def runcmd(c): 83 | """ 84 | Body of pysh main loop: handle a single character from the terminal. 85 | """ 86 | global cmd, point, prompt, continuation, history, i_cmd, running 87 | k = keyseq.keyseq(c) 88 | if k: # keyseq returns '' if key sequence is not complete 89 | if k == key.cr: # RET finishes entering cmd and runs Python cmd 90 | history.insert(0,cmd) 91 | if len(history) > max_cmds: history.pop() 92 | i_cmd = 0 93 | display.next_line() 94 | if cmd == 'exit()': # Trap here, do *not* exit Python 95 | cmd = '' # otherwise cmd is exit() at next pysh() call... 96 | point = 0 # ... and point is 5 97 | running = False 98 | else: 99 | terminal.set_line_mode() 100 | continuation = pycall(cmd) # Run the Python cmd 101 | terminal.set_char_mode() 102 | cmd = '' 103 | point = 0 104 | i_cmd = -1 # code will assign it to 0 or greater 105 | prompt = ps2 if continuation else ps1 106 | restore_cursor_to_cmdline() # Defined here, duplicates fcn in edsel 107 | display.putstr(prompt) 108 | el.refresh(cmd, point, start_col) 109 | elif k == key.C_d and cmd =='': # ^D on empty line exits, like 'exit()' 110 | cmd = '' # no command on line 111 | point = 0 112 | running = False 113 | elif k in (key.C_p, key.up): 114 | if i_cmd < len(history)-1: i_cmd += 1 115 | cmd = history[i_cmd] 116 | point = len(cmd) 117 | el.refresh(cmd, point, start_col) 118 | elif k in (key.C_n, key.down): 119 | if i_cmd >= 0: i_cmd -= 1 # reaches -1 after most recent... 120 | if i_cmd < 0: cmd = '' # ... then set cmd empty 121 | cmd = history[i_cmd] 122 | point = len(cmd) 123 | el.refresh(cmd, point, start_col) 124 | else: 125 | cmd, point = el.runcmd(k, cmd, point, start_col) # edit cmd 126 | 127 | def pysh(): 128 | """ 129 | Custom Python REPL that uses our editcommand instead of builtin input fcn 130 | so other tasks can interleave and we can restore cursor in Python cmd line. 131 | To exit pysh, type 'exit()' or ctrl-d. 'pysh' rhymes with fish. 132 | """ 133 | setup() 134 | while running: 135 | c = terminal.getchar() # blocking 136 | runcmd(c) 137 | restore() 138 | 139 | -------------------------------------------------------------------------------- /doc/other.md: -------------------------------------------------------------------------------- 1 | Here are some examples of pertinent system software and utilities 2 | written in languages other than Python. 3 | 4 | | Component | Type | Language | Description| 5 | |-----------|------|----------|------------| 6 | | [/grub-core/fs](http://bzr.savannah.gnu.org/lh/grub/trunk/grub/files/head:/grub-core/fs/) ([via](http://ask.metafilter.com/230728/NTFS-FAT-HFS-Drowning-in-Acronyms#3339280)) | Read-only file systems | C | "The bootloader Grub 2 includes minimal, read-only drivers for a gazillion different file systems. ... it's much simpler code than most drivers, since it's read-only. Plus, you can even run it in userland, which means you can easily attach a debugger and see exactly what's going on." | 7 | | [Lua](http://lua-users.org/lists/lua-l/2012-04/msg00331.html), [syntax](http://www.lua.org/manual/5.2/manual.html#9), [source](http://www.lua.org/source/5.2/), [papers](http://www.lua.org/docs.html#papers) especially [implementation](http://www.lua.org/doc/jucs05.pdf), [LuaJIT](http://luajit.org/faq.html) | Embeddable scripting langage 1993 -- | Lua, C | "simple, efficient, portable, and lightweight", "an order of magnitude or more smaller than ... (Python etc)", "much faster than most scripting languages", "first language in wide use to adopt a register-based virtual machine" | 8 | | [Rubinius](http://rubini.us/), also [here](http://redartisan.com/2007/10/5/rubinius-getting-started) and [here](http://razzledazzle.it/1:origin-story/3:rubinius) | Interpreter/Compiler | Ruby | "Rubinius is an alternate implementation of the Ruby virtual machine, loosely based on the architecture and implementation of Smalltalk-80." "The Rubinius bytecode virtual machine is written in C++, incorporating LLVM to compile bytecode to machine code at runtime. The bytecode compiler and vast majority of the core classes are written in pure Ruby." Interesting contrast to [PyPy](http://pypy.org/). | 9 | |[PEG](http://www.vpri.org/pdf/tr2010003_PEG.pdf) (Parsing Expression Grammar) | Compiler | various | "PEG-based transformer provides front-, middle-and back-end stages in a simple compiler", "highly suited to implementing its own implementation language". Applied in [OMeta](http://www.vpri.org/pdf/tr2007003_ometa.pdf), also [here](http://tinlizzie.org/ometa/) | 10 | |[c4](https://github.com/rswier/c4), C in four functions, also explained on [HN](https://news.ycombinator.com/item?id=9642582) | Interpeter | C | "the master class in minimal interpreted C implementations", "a simple lexer ... feeds a ... parser that spits out bytecode for a simple stack machine" 496 lines | 11 | | [multischeme](https://github.com/ojarjur/multischeme), also [here](https://news.ycombinator.com/item?id=7166755) | Multitasking library | Scheme | "perform preemptive multitasking entirely at compile time (or at program load time for apps not included when you build a binary for your OS)", "much faster than using the native primitives usually used for building multitasking support" | 12 | | [linenoise](https://github.com/antirez/linenoise), also [HN](https://news.ycombinator.com/item?id=1209646) | *readline* replacement | C | "small self-contained alternative to readline and libedit ... minimal, zero-config, BSD-licensed ... used in Redis, MongoDB, and Android." *linenoise.c* header has a good explanation of ANSI control sequences. | 13 | | [yaft](https://github.com/uobikiemukot/yaft), also [HN](https://news.ycombinator.com/item?id=13342689) | Framebuffer terminal | C | "simple terminal emulator for minimalist" | 14 | | [pie](https://github.com/5HT/pie) | Editor | Erlang | "Emacs written in Erlang", "Text is stored as a tree of binaries ... Buffers are small servers" | 15 | | [Chaos](http://sdegutis.github.io/chaos/), also [here](https://github.com/sdegutis/chaos) and [here](https://github.com/sdegutis/chaos/blob/master/Chaos/editor.lua) | Editor | Lua | "written using its own extension API which is written in Lua! ... The only part written in native code is the part that draws the characters ... It's just a grid of fixed-width text. ... no fancy GUI ... clear advantages over text editors with fancy rendering engines." For Mac OS X. | 16 | | [Edit](http://c9x.me/edit/), also [HN](https://news.ycombinator.com/item?id=8354435) | Editor | C | "A relaxing mix of vi and Acme" | 17 | | [bootedit](https://web.archive.org/web/20110615061121/http://lists.canonical.org/pipermail/kragen-discuss/2011-June/001168.html) | Editor | Lua | "a text-editor bootstrapping exercise: starting by writing a very simple text editor with cat, and gradually making it more featureful, using the text editor to write itself." | 18 | | [picolisp](http://software-lab.de/doc/tut.html) | Lisp REPL + tools | PicoLisp | vi-style line editor ... history ... inspect data and code in the running system ... tracing and single-stepping | 19 | | [ok](https://github.com/JohnEarnest/ok) | K language REPL + tools | Javascript | "a few special commands" for editing, tracing ... | 20 | | [robinson](http://limpet.net/mbrubeck/2014/08/08/toy-layout-engine-1.html), also [here](https://github.com/mbrubeck/robinson) | Browser engine | Rust | "toy web rendering engine ... " for HTML + CSS | 21 | | [gitlet](http://gitlet.maryrosecook.com/), also [github](https://github.com/maryrosecook/gitlet) | Distributed version control | Javascript | "Git implemented in JavaScript" "not intended to be fast or feature complete" "1000 lines, the implementation of the main API commands is only 350 lines." | 22 | | [Java Optimized Processor (JOP)](http://www.jopdesign.com/) also [here](http://www.jopdesign.com/doc/rtarch.pdf) and [here](https://github.com/jop-devel/jop) | Java Virtual Machine implemented in hardware | Java, VHDL | "... direct implementation of all bytecodes in hardware is not a useful approach ... Microcode is the native instruction set for JOP. Bytecodes are translated, during their execution, into JOP microcode. This translation merely adds one pipeline stage to the core processor and results in no execution overheads ... 43 of the 201 different bytecodes are implemented by a single microcode instruction, 93 by a microcode sequence, and 40 bytecodes are implemented in Java." | 23 | | [LispMicrocontroller](https://github.com/jbush001/LispMicrocontroller), also [Wiki](https://github.com/jbush001/LispMicrocontroller/wiki) | Lisp machine, 2013 -- | Verilog, Python, Lisp | "instruction set ... optimized for LISP" "code is compiled off-line into machine code (by compile.py), which can be loaded into the program ROM" "When the compiler starts, it automatically reads the file 'runtime.lisp', which has a number of fundamental primitives that are written in LISP (for example, memory allocation and garbage collection)."| 24 | 25 | -------------------------------------------------------------------------------- /tasking/writer.py: -------------------------------------------------------------------------------- 1 | """ 2 | writer.py - Functions that put text into editor buffers and windows, 3 | intended to be called from background tasks. 4 | 5 | See writer.txt for more notes and explanation. 6 | """ 7 | 8 | import display 9 | import sked as ed 10 | import edsel as fr # short for 'frame' 11 | import editline as el 12 | import pmacs as pm 13 | import pyshell as sh 14 | 15 | # Redefine these functions from edsel to also restore cursor to point 16 | 17 | def restore_cursor_to_cmdline(): 18 | 'Unlike version in edsel, this version also sets column to point' 19 | fr.restore_cursor_to_cmdline() # puts cursor in col 1 20 | el.move_to_point(sh.point, sh.start_col) 21 | 22 | saved_focus = -1 # index of focus window *before* we switch to print tick msg 23 | 24 | def restore_cursor(): 25 | ## sh.cmd_mode = False # DEBUG for testing n_windows() == 2 case from REPL 26 | if sh.cmd_mode: # editing/running Python commands at pysh REPL 27 | restore_cursor_to_cmdline() # redefined above, not the version in edsel 28 | # This next case assumes other window is tocus window that needs restoring 29 | # BUT if current window is intended focus window, this switches focus away 30 | elif fr.n_windows() == 2: # HACK only works in this n == 2 special case 31 | ## display.putstr(f'SWITCH WINDOW from {ed.bufname}') # DEBUG 32 | fr.save_window(fr.focus) 33 | fr.focus = saved_focus 34 | fr.restore_window(saved_focus) 35 | pm.restore_cursor_to_window() 36 | ## display.putstr(f'AT POINT in {ed.bufname}') # DEBUG 37 | else: # There is no other window, cursor is already where it is needed. 38 | pass 39 | 40 | # Local refresh and recenter in this module are copied from edsel 41 | # except here refresh does not call update_status 42 | # so it doesn't call restore_cursor_to_cmdline, which we don't want. 43 | 44 | def refresh(): 45 | """ 46 | Refresh the focus window. 47 | (Re)Display lines from segment, marker, status without moving segment. 48 | """ 49 | display.put_cursor(fr.wintop, 1) # top line in window 50 | fr.erase_lines(fr.wheight-1) # erase entire window contents above status line 51 | fr.update_window() # FIXME did we really have to erase_lines before this? 52 | fr.put_marker(ed.dot, display.white_bg) 53 | # fr.update_status() # NOT! we don't want restore_cursor_to_cmdline 54 | 55 | def recenter(): 56 | 'Move buffer segment to put dot in center, display segment, marker, status' 57 | # global buftop # use edsel.buftop instead 58 | fr.buftop = fr.locate_segment(ed.dot) 59 | refresh() # redefined above, not the version in edsel 60 | 61 | def write(line): 62 | """ 63 | Append line to end of current sked buffer and display in edsel focus window. 64 | line is a string that does not end with \n, this write() adds it. 65 | """ 66 | if line not in ('', '\n'): # redirect_stdout and file=... append extra \n 67 | ed.buffer.append(line.rstrip('\n\r') + '\n') # line might have many \n 68 | ed.dot = ed.S() # last line in buffer, which we just added. 69 | if fr.in_window(ed.dot): 70 | display.put_cursor(fr.wline(ed.dot), 1) 71 | display.putstr(line[:fr.width]) 72 | else: 73 | recenter() # redefined above, not the version in edsel 74 | restore_cursor() 75 | 76 | def writebuf(bname, line): 77 | """ 78 | Write to a buffer that might not be the current buffer. 79 | If the named buffer is present in saved buffers, append line to its end. 80 | If the named buffer is visible in the focus window, update that window. 81 | line is a string that does not end with \n, this function adds it. 82 | """ 83 | if line not in ('', '\n'): # redirect_stdout and file=... append extra \n 84 | if bname in ed.buffers: 85 | # bname may not be ed.bufname, named buffer may not be current buffer 86 | ed.buffers[bname]['buffer'].append(line.rstrip('\n\r') + '\n') 87 | ed.buffers[bname]['dot'] = len(ed.buffers[bname]['buffer'])-1 88 | # Current buffer text lines are the same as text lines in saved buffers 89 | # BUT current buffer dot might not be the same, so must assign here 90 | if bname == ed.bufname: ed.dot = ed.S() 91 | # If the named buffer is visible in the focus window, update that window 92 | # Focus window dot might not be at the end of the buffer 93 | if bname == ed.bufname: # name of current buffer 94 | if fr.in_window(ed.dot): # assumes focus window shows current buffer 95 | display.put_cursor(fr.wline(ed.dot), 1) 96 | display.putstr(line[:fr.width]) 97 | else: 98 | recenter() # redefined above, not the version in edsel 99 | restore_cursor() 100 | 101 | def writebuf_show(bname, line): 102 | """ 103 | Call writebuf to update buffer bname with line. 104 | If buffer bname is in focus window, called writebuf will display it. 105 | If buffer bname is in a window that is not the focus window, first make 106 | that the focus window, then call writebuf to update buffer and display it. 107 | If buffer bname is not in a window on the display, writebuf will update 108 | it but not display it. 109 | If buffer bname is not in buffers, writebuf does not attempt to update it. 110 | """ 111 | global saved_focus # index of focus window before weswitch to print tick msg 112 | wk = -1 # can't be a window key 113 | # search for key of window that shows buffer bname 114 | for wk in fr.windows: # wk is integer window key 115 | if fr.windows[wk]['bufname'] == bname: 116 | break 117 | # if window with named buffer found, change focus - based on fr.on code 118 | if wk in fr.wkeys and wk != fr.focus: # if not found, wk is still -1 119 | fr.save_window(fr.focus) 120 | saved_focus = fr.focus 121 | fr.focus = fr.wkeys[wk] 122 | fr.restore_window(wk) 123 | writebuf(bname, line) 124 | 125 | class Writer(): 126 | """ 127 | Provide a method named write so we can redirect output to the named buffer. 128 | Our buffers are just dicts not objects so they have no write method. 129 | Usage: abuf = Writer('a.txt') then: with redirect_stdout(abuf) as buf: ... 130 | Recall that a is the name of the sked/edsel append fcn, can't use a = .. 131 | """ 132 | def __init__(self, bufname): self.bufname = bufname 133 | def write(self, line): writebuf_show(self.bufname, line) 134 | 135 | -------------------------------------------------------------------------------- /BRANCH.md: -------------------------------------------------------------------------------- 1 | 2 | branches 3 | ======== 4 | 5 | This is the *rewrite* branch. 6 | 7 | Beginning in Feb 2023, a total rewrite of the Piety system is underway 8 | here in the *rewrite* branch and its branches, to shorten and simplify 9 | the code, and to make the programming environment more responsive. 10 | 11 | The *rewrite* branch is now the main branch. I will never merge it back 12 | into the *master* branch. 13 | 14 | Recent work in the *rewrite* branch: 15 | 16 | - 19 Oct2025: Begin *micropython-unix* branch to see if we can get our 17 | present Piety code to run under the micropython we built from the latest 18 | source from *micropython.org*. 19 | 20 | - 19 Oct 2025: Rename *micropython* branch to *micropython-snap* 21 | and abandon it. 22 | 23 | - 30 Sep 2025: Begin *micropython* branch to see if we can get our 24 | present Piety code to run under the micropython.we installed with the 25 | *snap* package manager. 26 | 27 | - 28 Jul 2025: Merge the *reader* branch back into the *rewrite* branch. 28 | 29 | - 6 Apr 2025: Begin *reader* branch to experiment with a viewer window 30 | beside the editor windows. 31 | 32 | - 6 Apr 2025: Merge the *browser* branch back into the *rewrite* branch. 33 | 34 | - 23 Jan 2025; Begin *browser* branch to write a text-only web browser 35 | closely integrated with the Piety editors, where downloaded web pages 36 | are stored and displayed in editor buffers. 37 | 38 | - 22 Jan 2025: Merge the *console* branch back into the *rewrite* branch. 39 | 40 | - 23 Dec 2024: Begin *console* branch to enable Piety development using 41 | only Python running in one Linux console terminal, without using the 42 | host desktop or shell. 43 | 44 | - 11 Nov 2024: Merge the *edmore* branch back into the *rewrite* branch. 45 | 46 | - 29 Aug 2024: Begin *edmore* branch for editor improvements and bug fixes. 47 | 48 | - 7 Aug 2024: Merge the *eventloop* branch back into the *rewrite* branch. 49 | 50 | - 16 Jun 2024: Begin *eventloop* branch to make coroutine versions of the 51 | *pysh* custom Python interpreter and *pmacs* editor, that can run 52 | as concurrent tasks in an *asyncio* event loop. 53 | 54 | - 2 Jun 2024: Reorganize directories. Split *tasks* directory into 55 | three: *tasking*, *threads*, and *coroutines*, rename *shells* to *python*. 56 | 57 | - 27 May 2024: Begin experimenting with coroutines in the *tasks* directory, 58 | still in the *rewrite* branch. 59 | 60 | - 24 May 2024: Finished for now adding and reorganizing links on Python 61 | language compilers, libraries, and tools in *doc/compilers.md* and 62 | *utilities.md*. 63 | 64 | - 11 May 2024: Change default branch at Github from *master* to *rewrite*. 65 | 66 | - 11 May 2024: Finish revising .md files at top level and under *doc*. 67 | 68 | - 9 May 2024: Begin revising top-level *README.md* and other *.md* files 69 | at top level and under *doc* to make them consistent with code in 70 | the *rewrite* branch. Do this work in the *rewrite* branch. 71 | 72 | - 9 May 2024: Revise top-level README.md in *master* branch: 73 | put directions to *rewrite* branch right at the top so they 74 | can't be missed. 75 | 76 | - 9 May 2024: Finish work on tasking with threads for now. Already in 77 | the *rewrite* branch, so no merge is needed. 78 | 79 | - 23 Apr 2024: Resume work on tasking with threads, but in the *rewrite* branch. 80 | 81 | - 18 Apr 2024: Finish fixing editor bugs for now. Merge *edfix* branch 82 | back into *rewrite* branch. 83 | 84 | - 9 Apr 2024: Begin *edfix* branch to fix bugs recently found in the editors. 85 | 86 | - 9 Apr 2024: Finish work on the custom Python REPL and tasking code 87 | for now. Merge *pysh* branch back into *rewrite* branch. 88 | 89 | - 27 Feb 2024: Begin *pysh* branch to provide a custom Python REPL that 90 | uses our *editline* so we can restore the cursor to the correct 91 | position in the command line after another thread moves it. 92 | 93 | - 27 Feb 2024: Finish experiments with threading for now. Merge 94 | *tasks* branch back into *rewrite* branch. 95 | 96 | - 12 Jan 2024: Begin *tasks* branch off *rewrite* brnach for experiments 97 | with tasks and concurrency using Python threads. 98 | 99 | - 12 Jan 2024: Finish work on the editors for now. Merge *ed* branch 100 | back into *rewrite* branch. 101 | 102 | - 11 Jan 2024: Multiple windows working in edsel, dmacs, and pmacs. 103 | Merge *window* branch back into *ed* branch. 104 | 105 | - 21 Oct 2023: Make *window* branch to *ed* branch. Add multiple windows 106 | to *pmacs* editor. 107 | 108 | - 21 Oct 2023: Merge *ed* branch back into *rewrite* branch. However, 109 | work continues in the *ed* branch and its branches. 110 | 111 | - 21 Oct 2023: *pmacs* emacs-like editor working. Merge *editline* branch 112 | back into *ed* branch, with completed *editline* and *pmacs* modules. 113 | 114 | - 13 Jul 2023: make *editline* branch to *ed* branch. 115 | Add *editline* module, edit and display a string using readline control keys. 116 | Add *pmacs*, edit text in lines anywhere in buffer, not just in append mode. 117 | 118 | - 2 Jul 2023: Merge *ed* branch back into *rewrite* branch. However, 119 | work continues in the *ed* branch and its branches. 120 | 121 | - 1 Jul 2023: dmacs editor working (no longer called pmacs), merge 122 | *pmacs* branch back into *ed* branch. 123 | 124 | - 28 May 2023: Make *pmacs* branch to *ed* branch to invoke editor functions 125 | with emacs keycodes so you don't have to invoke the functions from the 126 | Python REPL. 127 | 128 | - 28 May 2023: Indent, wrap, and join functions working, merge *format* 129 | branch back into *ed* branch. 130 | 131 | - 19 May 2023: Make *format* branch of *ed* branch to add indent and 132 | wrap functions to *sked* and *edsel*. 133 | 134 | - 19 May 2023: rename *frame.py* and *frameinit.py* to *edsel.py* and 135 | *edselinit.py*. 136 | 137 | - 12 May 2023: revised a(ppend) command now working, merge *inwindow* branch 138 | back into *ed* branch. 139 | 140 | - 22 Apr 2023: Make *inwindow* branch of *ed* branch to work on display code: 141 | Type lines into the a (append) command in place in the display window. 142 | 143 | - 22 Apr 2023: Revise editors/README.md to describe recent work on 144 | *sked* and *frame* in the *patch* and *fparam* branches. 145 | 146 | - 20 Apr 2023: All sked functions are now displaying in the *fparam* branch, 147 | merge back into the *ed* branch. 148 | 149 | - 8 Apr 2023 Make *fparam* branch of *ed* branch to work on display code. 150 | Instead of patching functions in *sked*, display functions are passed 151 | as parameters to functions in *sked*. 152 | 153 | - 8 Apr 2023: Reach stopping place in *patch* branch, merge back into *ed*. 154 | 155 | - 20 Feb 2023: Make *patch* branch of *ed* branch to work on display code 156 | in *frame*. Display functions are assigned to ('patch') placeholder 157 | functions in *sked*. 158 | 159 | - 9 Feb 2023: Remove *shells* directory and *pycall*, not needed. 160 | 161 | - 2 Feb 2023: Make the *ed* branch for work on our line editor *sked.py*. 162 | 163 | - 1 Feb 2023: Add *shells* directory with *pycall* callable Python interpreter. 164 | 165 | - 1 Feb 2023: Delete most files and directories for a fresh start. 166 | Revise *BRANCH.md*, *DIRECTORIES.md*, and *bin/paths*. 167 | 168 | -------------------------------------------------------------------------------- /editors/breakpt.md: -------------------------------------------------------------------------------- 1 | 2 | breakpt 3 | ======= 4 | 5 | The *breakpt* module defines and assigns a *breakpoint hook* that 6 | makes it possible to use *Pdb* to debug the display editor *pmacs* while it 7 | is running, without disturbing its window contents. 8 | 9 | We do not use the debugger much in Piety. We always have the Python REPL 10 | available, so invoking functions and examining global variables is usually 11 | sufficient to reveal what the code is doing. But sometimes it is helpful 12 | to examine the local variables within functions, and we need the debugger for that. 13 | Our *breakpt* makes it easy to use the debugger in our usual workflow, without 14 | interruping the editor in a long-running Python session. 15 | 16 | To use the debugger, type the command *import breakpt* in the interactive 17 | Python REPL. Then edit the module that contains the code you want to debug. 18 | At the point in the code where you want to enter the 19 | debugger, add a call to the builtin function *breakpoint*. Then reload that 20 | module. 21 | 22 | Perform some editing operation that uses the code you want to debug. 23 | When execution reaches the call to *breakpoint*, the cursor leaves the 24 | display window and moves to the scrolling REPL region at the bottom of the 25 | terminal. The debugger prompt *(Pdb)* appears there. Now you can type 26 | *pdb* commands. When you are finished using the debugger, type the *pdb* *c* 27 | (continue) command. The debugger exits and the cursor moves back up into the 28 | display window. The program resumes running normally. 29 | 30 | For example, debug the function *erase_bottom* in *edsel.py*. This 31 | function erases lines that may be left over after the end of the buffer at the 32 | bottom of a window, after some editing operation makes the buffer shorter: 33 | 34 | def erase_bottom(): 35 | """ 36 | Erase any old lines left over between end of buffer and bottom of window. 37 | Leave cursor after last line erased. Do not update any globals. 38 | """ 39 | nlines = (wheight-1) - (wline(ed.dot)-wintop) # n of lines to window status line 40 | nblines = ed.S() - ed.dot # n of lines to end of buffer 41 | nelines = nlines - nblines # n of empty lines at end of window 42 | ### breakpoint() # DEBUG Uncomment this line for breakpoint demo. See breakpt.md. 43 | erase_lines(nelines) # Make empty lines at end of window. 44 | 45 | This function is short and simple, so of course we got it wrong at first. We 46 | thought we could find the error if we could see the values of the local 47 | variables *nlines*, *nblines*, and *nelines*. So we put in a call to 48 | *breakpoint* after all three were assigned, and ran the code by editing in a 49 | buffer until we hit the breakpoint. In the debugger, we saw that the value 50 | of *nlines* was negative -- obviously wrong! We realized we had forgotten to 51 | subtract *wintop*. We exited the debugger, and in that same editor session 52 | we immediately corrected the code, commented out the breakpoint, and ran the 53 | code again to confirm the correction worked. 54 | 55 | Here are more details about this fix: 56 | 57 | To see this code working correctly, put the cursor in a window and scroll 58 | down to the end of the buffer so there are some empty lines at the bottom of 59 | the window. The file *test/lines40.txt* is good for this because it is easy 60 | to tell which lines have been deleted and moved. Delete a few lines near the 61 | end of the buffer with *C-k* (kill line) or *C-w* (cut). The lines following 62 | the deletion move up, leaving more empty lines at the bottom of the window. 63 | 64 | Activate our *breakpt* function. While display editing, type *M-x* to get to 65 | the Python REPL. At the Python prompt, type the statement *import breakpt*. 66 | Then type the function call *pm()* to return to display editing. 67 | 68 | Now edit *edsel.py*, find the function *erase_bottom*, and uncomment the 69 | line with *breakpoint()* by removing the comment characters *#* from the 70 | beginning of the line. Reload *edsel.py* by typing *C-x C-r* (reload). 71 | 72 | Once again, put the cursor in a window and scroll down to the end of the 73 | buffer, leaving some empty lines at the bottom of the window. Delete 74 | some lines near the end of the buffer. This causes execution to reach 75 | the breakpoint. The remaining lines near the end of the window 76 | move up, but the same lines also remain below them at the end of the window. 77 | They are not erased, because the breakpoint comes before the code that 78 | erases them. 79 | 80 | When execution reaches the breakpoint, the cursor moves from the window to 81 | the scrolling REPL. The debugger runs, prints the location of the breakpoint 82 | in the code, and prints the *(Pdb)* debugger prompt: 83 | 84 | > /home/jon/Piety/editors/breakpt.py(43)breakpt() 85 | -> pdb.set_trace() # Enter Pdb debugger, use Pdb commands until Pdb c (continue) 86 | (Pdb) 87 | 88 | Type the debugger command *w* (where) to print the function call stack: 89 | 90 | (Pdb) w 91 | ... 92 | ... other functions ... 93 | ... 94 | -> ed.d(start, end, append, display_d) 95 | /home/jon/Piety/editors/sked.py(410)d() 96 | -> move_dot(new_dot) 97 | /home/jon/Piety/editors/edsel.py(245)display_d() 98 | -> erase_bottom() 99 | /home/jon/Piety/editors/edsel.py(119)erase_bottom() 100 | -> breakpoint() # DEBUG Uncomment this line for breakpoint demo. See breakpt.md. 101 | > /home/jon/Piety/editors/breakpt.py(43)breakpt() 102 | -> pdb.set_trace() # Enter Pdb debugger, use Pdb commands until Pdb c (continue) 103 | (Pdb) 104 | 105 | This shows that execution has stopped in our *breakpt* function, at the 106 | statement *pdb.set_trace()*. We want to examine the local variables in 107 | *erase_bottom*, so we type the debugger command *u* (up) to move up the stack. 108 | Then we type several *p* (print) commands to show the variable values. See the 109 | code for the meaning of these values: 110 | 111 | (Pdb) u 112 | > /home/jon/Piety/editors/edsel.py(119)erase_bottom() 113 | -> breakpoint() # DEBUG Uncomment this line for breakpoint demo. See breakpt.md. 114 | (Pdb) p nlines 115 | 9 116 | (Pdb) p nblines 117 | 1 118 | (Pdb) p nelines 119 | 8 120 | (Pdb) 121 | 122 | These all appear to be correct -- we fixed the error already. Before we fixed 123 | the error, *nlines* was a negative number. 124 | 125 | Now type the *c* (continue) command to exit the debugger, move the cursor 126 | back up into the window, and resume running the program normally: 127 | 128 | (Pdb) c 129 | 130 | However, the window contents do not update correctly, because the cursor 131 | does not return to the correct location in the window, so subsequent code 132 | does not have the intended effect. The code does not record the location 133 | of the cursor when the debugger is activated, so this is the best we can do. 134 | Type the refresh command *C-l* to restore the correct window contents. 135 | 136 | After running this demonstration, be sure to return to the *erase_bottom* 137 | function in the *edsel.py* buffer again to comment out the *breakpoint()* 138 | call. Otherwise, you will hit the breakpoint again every time you delete 139 | lines from any buffer. 140 | 141 | After we hit a breakpoint, we can use debugger commands to examine the stack, 142 | move up and down the stack, and examine local variables in each function 143 | call on the stack, all without disturbing window contents. However, it is 144 | not useful to step through code. Statements that send commands 145 | to update the display window will not work as intended, because the cursor 146 | is not in the window, it is in the scrolling REPL region. The commands will 147 | merely scramble the REPL region, rendering it illegible. 148 | 149 | Revised Oct 2024 150 | 151 | -------------------------------------------------------------------------------- /README_54.md: -------------------------------------------------------------------------------- 1 | 2 | Piety 3 | ===== 4 | 5 | **Piety** is an operating system written in Python. 6 | 7 | [Motivation and Goals](#Motivation-and-Goals) 8 | [Current Status](#Current-Status) 9 | [No Dependencies](#No-Dependencies) 10 | [Demos](#Demos) 11 | [Roadmap](#Roadmap) 12 | [Tested Platforms](#Tested-Platforms) 13 | [Footnotes](#Footnotes) 14 | 15 | ## Motivation and Goals ## 16 | 17 | Piety is a small but self-contained personal 18 | computer operating system for programmers. It 19 | provides a responsive and malleable platform for 20 | writing and programming. Its internals are easy to 21 | understand and modify. 22 | 23 | Piety uses a single programming language -- Python 24 | -- for both the applications and the operating 25 | system. You can use the language interpreter to 26 | inspect and manipulate any data in the running 27 | system. Changes to application and system code are 28 | effective immediately, without having to stop and 29 | restart the system. 30 | 31 | Piety is a reaction against the complexity and 32 | disempowerment of today's dominant computer 33 | systems. I take inspiration from the single user, 34 | single language, special hardware systems of the 35 | 1970s and 80s: Smalltalk, Lisp machines, Oberon 36 | (see [doc/precursors.md](doc/precursors.md)). 37 | Piety is an experiment to see if I can put 38 | together something similar today, but using a 39 | familiar programming language running on ordinary 40 | hardware. Let's see how far we can get with just 41 | Python. There is already a lot of work by others 42 | that we might be able to adapt or use as models 43 | (see [doc/utilities.md](doc/utilities.md)). For 44 | other projects in a similar spirit, again see 45 | [doc/precursors.md](doc/precursors.md). 46 | 47 | ## Current Status ## 48 | 49 | For now, Piety runs in an ordinary Python 50 | interpreter session in a single terminal on a host 51 | operating system. The Python interpreter with its 52 | runtime is the virtual machine where the Piety OS 53 | now runs, analogous to the QEMU virtual machine in 54 | many other operating system projects. 55 | 56 | Piety provides a [display 57 | editor](editors/README.md), a [customized Python 58 | interpreter](tasking/pyshell.py) that also acts as 59 | the system [shell](console/README.md), a 60 | [customized debugger](editors/breakpt.md), and its 61 | own [web browser](browser/README.md). The display 62 | editor can support multiple buffers and windows in 63 | the terminal, and also a region for the Python 64 | interpreter. The debugger can work in the 65 | interpreter region without disturbing window 66 | contents. Together these provide a minimal but 67 | self-contained programming environment within a 68 | single Python terminal session. 69 | 70 | Piety development is self-hosted in this 71 | programming environment. Code is added and revised 72 | in a long-running Python session. Code is imported 73 | and reloaded into the session without restarting 74 | or losing work in progress. To make this possible 75 | we adopted a [particular workflow and coding 76 | style](editors/HOW.md). 77 | 78 | Piety provides concurrency with a Python *asyncio* 79 | event loop. Tasks are implemented by Python 80 | *coroutines* or *readers* (event handlers) that 81 | run in an event loop. 82 | 83 | Piety provides *asyncio* readers for its custom 84 | Python shell and its editor. These enable the 85 | shell and the editor to run without blocking in an 86 | event loop, so other tasks can run concurrently, 87 | as you type commands in the shell or edit text in 88 | the editor. 89 | 90 | The editor is not just for creating text. Python 91 | commands including concurrent tasks can redirect 92 | their output to editor buffers and windows, so the 93 | editor can be used for data capture and animated 94 | display. We use it for [experiments](piety) in 95 | tasking and concurrency where tasks update windows 96 | as we control their behavior by typing commands at 97 | the Python interpreter. 98 | 99 | We also use the editor as our [web 100 | browser](browser/README.md). Downloaded web pages 101 | are stored and displayed in editor buffers. 102 | 103 | Here is more about some Piety [design 104 | decisions](doc/rationale.md) and their rationales. 105 | 106 | The present version of Piety was started from 107 | scratch in February 2023. Its development is 108 | ongoing here in the *rewrite* branch of the 109 | *Piety* repository. An archive of the earlier 110 | version of Piety that was abandoned in January 111 | 2023 is here in the *master* branch and *version1* 112 | tag. The *rewrite* branch is now the main branch; 113 | I will never merge it back into *master*. 114 | 115 | ## No Dependencies ## 116 | 117 | The Piety system has no dependencies, other than 118 | Python itself, including a few standard library 119 | modules. 120 | 121 | With no dependencies, it is easy to try out Piety. 122 | Just clone this Piety repository and use your 123 | system's built-in *python* (or *python3*) command 124 | to run the scripts. You do not need to set up any 125 | Python environment. 126 | 127 | ## Demos ## 128 | 129 | Some interactive demonstrations are described in 130 | [pmacs_script.md](piety/pmacs_script.md) and 131 | [edsel_script.txt](piety/edsel_script.txt) and 132 | [pmacs_blocking.md](piety/pmacs_blocking.md) and 133 | [audoindent.md](editors/autoindent.md) and 134 | [breakpt.md](editors/breakpt.md). 135 | These pages give instructions so you can do the 136 | demos yourself. 137 | 138 | A few of the demos are not very interactive, so 139 | can be run from scripts: 140 | [pmacs_script.py](piety/pmacs_script.py) and 141 | [edsel_script.py](piety/edsel_script.py). 142 | 143 | We don't have any screenshots or animations of 144 | these demos. Honestly, there is not much to see -- 145 | it just looks like Emacs in a terminal. To gain 146 | any understanding, you have to read along in the 147 | those files and work through the demos yourself. 148 | 149 | ## Roadmap ## 150 | 151 | I hope someday to run Piety on a bare machine with 152 | no other operating system, but only a Python 153 | interpreter with minimal support. 154 | 155 | Piety divides naturally into two independent 156 | parts: the *hosted* part and the *native* part. 157 | The hosted part can run in any Python interpreter. 158 | It includes the editors, shells, tasking, the 159 | programming environment, and any tools and 160 | applications we might write. The native part 161 | includes the Python interpreter itself, and the 162 | support needed to run the interpreter on the 163 | computer hardware. Almost any general-purpose 164 | operating system can serve as the support, but the 165 | goal is to replace that with a special-purpose 166 | operating system which is itself mostly written in 167 | Python. 168 | 169 | All the work I have done so far, including the 170 | programming environment, is in the hosted part. I 171 | have researched [several 172 | approaches](doc/baremachine.md) to building the 173 | native part. I hope to begin soon. 174 | 175 | The obvious first step is to configure a minimal 176 | Linux running Python as its process 1. This would 177 | provide a system that boots into a Python prompt 178 | and provides a Python-only system to the user and 179 | application programmer. This is all we need to 180 | perform the essential experiment to see if it is 181 | feasible to continue Piety development and other 182 | personal computing activities using only Python 183 | running Piety in the console, without depending on 184 | a host OS to provide a desktop with multiple 185 | windows, the system shell, and other utilities. 186 | 187 | ## Tested Platforms ## 188 | 189 | The Piety software has run on a MacBook Pro 190 | running Mac OS, a Lenovo Chromebook running Linux, 191 | and an HP laptop running Linux. Details in 192 | [platforms.md](doc/platforms.md). 193 | 194 | ### Footnotes ### 195 | 196 | The phrase "complexity and disempowerment" is from 197 | a posting by 198 | [jl6](https://news.ycombinator.com/item?id=24917101) 199 | 200 | Revised Apr 2025 201 | 202 | 203 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | 2 | Piety 3 | ===== 4 | 5 | **Piety** is an operating system written in Python. 6 | 7 | [Motivation and Goals](#Motivation-and-Goals) 8 | [Current Status](#Current-Status) 9 | [No Dependencies](#No-Dependencies) 10 | [Demos](#Demos) 11 | [Screenshots](#Screenshots) 12 | [Roadmap](#Roadmap) 13 | [Tested Platforms](#Tested-Platforms) 14 | [Footnotes](#Footnotes) 15 | 16 | ## Motivation and Goals ## 17 | 18 | Piety is a small but self-contained personal computer operating system for 19 | programmers. It provides a responsive and malleable platform for writing 20 | and programming. Its internals are easy to understand and modify. 21 | 22 | Piety uses a single programming language -- Python -- for both the 23 | applications and the operating system. You can use the language interpreter 24 | to inspect and manipulate any data in the running system. Changes to 25 | application and system code are effective immediately, without having to 26 | stop and restart the system. 27 | 28 | Piety is a reaction against the complexity and disempowerment of today's 29 | dominant computer systems. I take inspiration from the single user, single 30 | language, special hardware systems of the 1970s and 80s: Smalltalk, Lisp 31 | machines, Oberon (see [doc/precursors.md](doc/precursors.md)). Piety is 32 | an experiment to see if I can put together something similar today, but 33 | using a familiar programming language running on ordinary hardware. Let's 34 | see how far we can get with just Python. There is already a lot of work by 35 | others that we might be able to adapt or use as models (see 36 | [doc/utilities.md](doc/utilities.md)). For other projects in a similar 37 | spirit, again see [doc/precursors.md](doc/precursors.md). 38 | 39 | ## Current Status ## 40 | 41 | For now, Piety runs in an ordinary Python interpreter session in a single 42 | terminal on a host operating system. The Python interpreter with its runtime 43 | is the virtual machine where the Piety OS now runs, analogous to the QEMU 44 | virtual machine in many other operating system projects. 45 | 46 | Piety provides a [display editor](editors/README.md), a [customized 47 | Python interpreter](tasking/pyshell.py) that also acts as the system 48 | [shell](console/README.md), a [customized debugger](editors/breakpt.md), 49 | and its own [web browser](browser/README.md). These, and other Python 50 | applications, can all be presented together in a 51 | [desktop](viewer/README.md) controlled by a custom window manager in a 52 | single full-screen terminal. 53 | 54 | The display editor can support multiple buffers and windows in the 55 | terminal, and also a region for the Python interpreter. The debugger can 56 | work in the interpreter region without disturbing window contents. 57 | Together these provide a minimal but self-contained programming 58 | environment within a single Python terminal session. 59 | 60 | Piety development is self-hosted in this programming environment. Code is 61 | added and revised in a long-running Python session. Code is imported and 62 | reloaded into the session without restarting or losing work in progress. 63 | To make this possible we adopted a 64 | [particular workflow and coding style](editors/HOW.md). 65 | 66 | Piety provides concurrency with a Python *asyncio* event loop. Tasks 67 | are implemented by Python *coroutines* or *readers* (event handlers) that 68 | run in an event loop. 69 | 70 | Piety provides *asyncio* readers for its custom Python shell and its 71 | editor. These enable the shell and the editor to run without 72 | blocking in an event loop, so other tasks can run concurrently, as you 73 | type commands in the shell or edit text in the editor. 74 | 75 | The editor is not just for creating text. Python commands including 76 | concurrent tasks can redirect their output to editor buffers and windows, so 77 | the editor can be used for data capture and animated display. We 78 | use it for [experiments](piety) in tasking and concurrency 79 | where tasks update windows as we control their behavior by typing commands 80 | at the Python interpreter. 81 | 82 | We also use the editor as our [web browser](browser/README.md). 83 | Downloaded web pages are stored and displayed in editor buffers. 84 | 85 | Here is more about some Piety [design decisions](doc/rationale.md) and their 86 | rationales. 87 | 88 | The present version of Piety was started from scratch in February 2023. Its 89 | development is ongoing here in the *rewrite* branch of the *Piety* repository. 90 | An archive of the earlier version of Piety that was abandoned in January 2023 91 | is here in the *master* branch and *version1* tag. The *rewrite* branch 92 | is now the main branch; I will never merge it back into *master*. 93 | 94 | ## No Dependencies ## 95 | 96 | The Piety system has no dependencies, other than Python itself, 97 | including a few standard library modules. 98 | 99 | With no dependencies, it is easy to try out Piety. Just clone this Piety 100 | repository and use your system's built-in *python* (or *python3*) 101 | command to run the scripts. You do not need to set up any Python 102 | environment. 103 | 104 | ## Demos ## 105 | 106 | Some interactive demonstrations are described in 107 | [pmacs_script.md](piety/pmacs_script.md) and 108 | [edsel_script.txt](piety/edsel_script.txt) and 109 | [pmacs_blocking.md](piety/pmacs_blocking.md) and 110 | [audoindent.md](editors/autoindent.md) and 111 | [breakpt.md](editors/breakpt.md). 112 | These pages give instructions so you can do the demos yourself. 113 | 114 | A few of the demos are not very interactive, so can be run from scripts: 115 | [pmacs_script.py](piety/pmacs_script.py) and 116 | [edsel_script.py](piety/edsel_script.py). 117 | 118 | ## Screenshots ## 119 | 120 | The pages about the Piety [desktop](viewer/README.md), 121 | [browser](browser/README.md), and 122 | the event loop demos [here](piety/pmacs_script.md) 123 | and [here](piety/vedsel_script.md) include screenshots. 124 | This [directory](screenshots) contains a few more. 125 | 126 | ## Roadmap ## 127 | 128 | I hope someday to run Piety on a bare machine with no other 129 | operating system, but only a Python interpreter with minimal support. 130 | 131 | Piety divides naturally into two independent parts: the *hosted* part and 132 | the *native* part. The hosted part can run in any Python interpreter. It 133 | includes the editors, shells, tasking, the programming environment, 134 | and any tools and applications we might write. The native part includes the 135 | Python interpreter itself, and the support needed to run the interpreter on 136 | the computer hardware. Almost any general-purpose operating system can 137 | serve as the support, but the goal is to replace that with a special-purpose 138 | operating system which is itself mostly written in Python. 139 | 140 | All the work I have done so far, including the programming environment, is 141 | in the hosted part. I have researched 142 | [several approaches](doc/baremachine.md) to building the native part. 143 | I hope to begin soon. 144 | 145 | The obvious first step is to configure a minimal Linux running 146 | Python as its process 1. This would provide a system that boots into a 147 | Python prompt and provides a Python-only system to the user and 148 | application programmer. This is all we need to perform the essential 149 | experiment to see if it is feasible to continue Piety development and 150 | other personal computing activities using only Python running Piety in 151 | the console, without depending on a host OS to provide a desktop with 152 | multiple windows, the system shell, and other utilities. 153 | 154 | ## Tested Platforms ## 155 | 156 | The Piety software has run on a MacBook Pro running Mac OS, 157 | a Lenovo Chromebook running Linux, and an HP laptop running Linux. 158 | Details in [platforms.md](doc/platforms.md). 159 | 160 | ### Footnotes ### 161 | 162 | The phrase "complexity and disempowerment" is from a posting by 163 | [jl6](https://news.ycombinator.com/item?id=24917101) 164 | 165 | Revised Sep 2025 166 | 167 | 168 | -------------------------------------------------------------------------------- /doc/rationale.md: -------------------------------------------------------------------------------- 1 | 2 | Design decisions and rationales 3 | =============================== 4 | 5 | Some design decisions for Piety, and their rationales: 6 | 7 | [Piety](..) is a small personal computer operating system for 8 | programmers. 9 | 10 | Piety uses a single programming language -- Python -- for both the 11 | applications and the operating system. 12 | 13 | Piety has no dependencies, other than Python itself. 14 | 15 | Code is added and revised in a long-running Python session, without 16 | restarting or losing work in progress. 17 | 18 | Piety provides concurrency with a Python *asyncio* event loop. 19 | 20 | [Personal computer](#Personal-computer) 21 | [For programmers](#For-programmers) 22 | [Python](#Python) 23 | [No dependencies](#No-dependencies) 24 | [Operating system](#Operating-system) 25 | [Long-running session](#Long-running-session) 26 | [Small](#Small) 27 | [Concurrency with *asyncio*](#Concurrency-with-asyncio) 28 | 29 | ### Personal computer ### 30 | 31 | Piety is intended to support a truly *personal* computer, whose software 32 | is created (or selected and modified) by its owner, to express their own 33 | preferences and inclinations. A personal computer enables its owner to work 34 | -- or just pass the time -- in the way that is most comfortable and satisfying for 35 | them, no matter how unusual or eccentric that might be. Piety is a deliberate 36 | reaction against the prevailing trend to try to build a system that everyone 37 | will use, that will take over the world. 38 | 39 | ### For programmers ### 40 | 41 | I see Piety as an *activity*: a series of experiments, an 42 | ongoing project. I often add features, but I am not trying to 43 | make a finished product. 44 | 45 | I resolved to write Piety from scratch, with no tools, starting from a 46 | Python prompt in a terminal window. First, I made a simple text editor, 47 | then used that to build the rest. I did write the first two hundred 48 | lines or so -- a few pages -- in another editor, but after that Piety 49 | development has been completely self-hosted in Piety itself. 50 | Building Piety up from almost nothing was an essential part of the 51 | experience for me. It still informs all my use of the system. 52 | 53 | I do not expect anyone else to use Piety routinely. It bears too many of 54 | my own peculiar preferences, and limitations that are severe but 55 | tolerable to me. But other programmers might try it out, or just look into 56 | the code and documents, to get ideas, techniques, and examples 57 | they could use to build systems that express their own preferences. 58 | 59 | ### Python ### 60 | 61 | Python is typically used from an interactive interpreter that enables the 62 | user to inspect and modify the running session, including importing and 63 | reloading entire modules. Lisp and Smalltalk introduced this way of 64 | working, and it is now provided by many other languages, but today Python is 65 | ubiquitous and is most familiar to me. 66 | 67 | Python already provides most of the operating system functionality that a 68 | typical user sees. Its interactive interpreter, along with its standard 69 | libraries, can do everything a Unix shell can do. Its generators and 70 | coroutines provide concurrency. Its standard libraries provide a network stack. 71 | Many [examples](utilities.md) show how to do systems programming in Python. 72 | And, thanks to its popularity and long history, there are Python libraries 73 | and applications for almost anything you might want to do with a computer. 74 | 75 | Given all this, why do we need a user-facing operating system at all? 76 | For now, we need it to run programs that are not in Python. But this comes at 77 | the cost of adding many additional things you must know to use the computer. 78 | Maybe we can simplify our computing life by dispensing with all that, and 79 | just use Python for everything. 80 | 81 | ### No dependencies ### 82 | 83 | Piety has no dependencies, other than the language and libraries included 84 | in the standard Python distribution. 85 | 86 | This makes development easy. Python and its standard libraries come already 87 | installed on every system I have used. I do not need to set up any 88 | Python environment, other than defining a single shell command to put the 89 | Piety directories on the PYTHONPATH. I do not need to search the 90 | internet for packages and documentation. Everything I need for Piety can 91 | be found in the local Python installation, often by using the Python 92 | *help* command. 93 | 94 | This also makes it easy for others to try out Piety. They can just 95 | clone this Piety repository and use their system's built-in *python* (or 96 | *python3*) command to run the scripts. 97 | 98 | ### Operating system ### 99 | 100 | In a computer whose operating system is written in Python, almost all code 101 | in the system, including the usual operating system functions as well as 102 | applications, runs in the Python interpreter. To boot the system, (almost) 103 | the first thing we must do is start Python, which then runs everything else. 104 | 105 | ### Long-running session ### 106 | 107 | An entire life cycle of the system from startup to shutdown runs in a single 108 | Python session - a single long-running invocation of the Python interpreter. 109 | 110 | To develop and use new software in a single long-running session, it must be 111 | possible to import and reload a module without restarting the session or 112 | losing work in progress. Reloading a module must make new or revised 113 | functionality available immediately, yet the state of the session -- the 114 | values of all its variables -- must be preserved across reloads, including 115 | the state of the reloaded module itself. 116 | 117 | We preserve state across reloads by only initializing variables the first time 118 | a module is imported, using the coding technique described in 119 | [How we program](../editors/HOW.md#Reloading-modules). 120 | 121 | 122 | ### Small ### 123 | 124 | Piety has to be small, because it is written by one person in his spare time. 125 | 126 | I use existing Python language constructs and the standard library instead 127 | of writing new code as much as I can, even if that requires limiting 128 | functionality and adopting a restricted programming style. I try to avoid 129 | hard problems, or replace them with easier ones instead. 130 | 131 | For example, the standard Python function *reload* does not update existing 132 | objects with their new definitions from the reloaded module. There are 133 | some complicated "hot reloading" packages that overcome this, but I just 134 | accept the limitation, avoid using objects to store persistent application 135 | data, and use dictionaries instead (again see 136 | [How we program](../editors/HOW.md#Modules-and-dictionaries-instead-of-classes-and-objects)) 137 | 138 | 139 | ### Concurrency with *asyncio* ### 140 | 141 | Piety provides concurrency with a Python *asyncio* event loop. 142 | 143 | I also considered the Python *threading* library (here are some 144 | [experiments](../threads)), but I rejected it because it uses 145 | the host operating system's threading library. My goal for Piety is to 146 | replace the host operating system. 147 | 148 | The *asyncio* library is all written in Python. It is built on the Python 149 | interpreter itself, based on generators and *yield*. It seems it should be 150 | able to run without much support from the host operating system. 151 | 152 | Concurrency using the *asyncio* event loop provides *cooperative multitasking*, 153 | which requires a particular coding style. Each task responds to events -- 154 | such as key presses -- and the code that handles each event runs to completion 155 | before the system can handle other events or run other tasks. 156 | Code must not *block* -- wait for events or data that are not yet available. 157 | 158 | The Piety editor and Python shell are coded so they can run without blocking 159 | in the event loop. Here is an 160 | [explanation and demonstration](../piety/pmacs_blocking.md). 161 | 162 | Revised Jul 2025 163 | 164 | -------------------------------------------------------------------------------- /browser/get.py: -------------------------------------------------------------------------------- 1 | """ 2 | get.py - Get a web page and store it in a Piety editor buffer. 3 | """ 4 | 5 | from urllib import request, parse 6 | from pathlib import Path 7 | import re 8 | 9 | import sked as ed, edsel as fr # fr for frame 10 | import render 11 | import key, dmacs # so we can add browser keycode entries to keymap 12 | 13 | # Simple regular expressions for matching URLs 14 | # Match https:// or file:// prefix and all that follows 15 | # up to whitespace or , ' " 16 | # Much simpler than URL regexps found on the Internet, 17 | # Matches many invalid URLs, but in our application that's not a problem, 18 | httpre = 'https?://[^,\'"\s]+' 19 | filere = 'file://[^,\'"\s]+' 20 | httprep = re.compile(httpre) # p for pattern 21 | filerep = re.compile(filere) 22 | norep = re.compile('No URL here') 23 | 24 | fnref = '\[\d+\]' # footnote reference - decimal digits inside [...] 25 | fnrefp = re.compile(fnref) 26 | fnline = '^\s*\d+\.\s+' # footnote line - whitespace, digits, period, whitepace 27 | fnlinep = re.compile(fnline) 28 | 29 | def xaurl(s): 30 | """ 31 | Return (eXtract) Absolute URL found in string s. 32 | An absolute URL begins with http:// or https:// or file:// 33 | Return empty string if no absolute URL found. 34 | We expect caller has passed a string that looks like it holds a URL. 35 | """ 36 | urlrep = httprep if 'http' in s else filerep if 'file' in s else norep 37 | m = urlrep.search(s) 38 | return m.group() if m else '' 39 | 40 | def xrurl(s): 41 | """ 42 | EXtract Relative URL from a string formatted as one of our footnotes, 43 | like these: 44 | 45 | 10. toolkit.html 46 | 11. ../z-lectures/z-lectures.html 47 | 48 | or like these: 49 | 50 | 47. /food-drink 51 | 48. /384528/My-very-first-Can-I-eat-this-question 52 | 53 | For now, we just return the first string of non-whitespace characters 54 | after the first one or more whitespace characters, if there is one 55 | If no such string, return '' to indicate no URL found. 56 | """ 57 | words = s.split() 58 | if len(words) != 2: # FIXME? Crudest check that s has relative URL, unsound 59 | return '' # indicates no URL found 60 | rurl = words[1] 61 | return rurl[2:] if rurl.startswith('..') else rurl # FIXME? Special case! 62 | 63 | def xurl(s): 64 | """ 65 | eXtract absolute or relative URL from string s 66 | Or return '' if no URL found. 67 | """ 68 | url = xaurl(s) # extract absolute URL, '' if not found 69 | if not url: # absolute URL not found on line - must be relative URL 70 | rurl = xrurl(s) # find relative URL, returns '' if none found 71 | url = ed.filename + rurl if rurl else '' # ed.filename stores base URL 72 | return url 73 | 74 | # URLs in baseurls are prefixes of web page absolute urls 75 | # used as base urls for fetching other pages from that site using relative urls. 76 | # The absolute url may include a suffix like 'newest' or 'news?p=2' 77 | # that must be omitted when forming a url from the base url + relative url. 78 | # The base URL is supposed to be identified by a tag in the page itself 79 | # but here we just handle particular base urls as special cases. 80 | baseurls = ('https://news.ycombinator.com/', # Hacker News 81 | 'https://www.tbray.org/', # Tim Bray's blog, Ongoing 82 | 'https://github.com/', # Github generates and serves long relative urls 83 | 'https://dercuano.github.io/', # Kragen Sitaker's notes, Dercuano 84 | 'https://derctuo.github.io/', # Kragen Sitaker's notes, Derctuo 85 | 'https://dernocua.github.io/', # Kragen Sitaker's notes, Dernocua 86 | 'https://courses.cs.northwestern.edu/325/readings/graham/', 87 | 'file:///home/jon/z/z/', # Our own Z notes, for testing 88 | ) 89 | 90 | url = '' # URL is global so we can use it in r(url) also for easy debugging 91 | response = None # response is global so we can inspect it at the REPL 92 | 93 | def g(aurl): 94 | """ 95 | (g)et web page from aurl and store it in its own Piety editor buffer. 96 | """ 97 | global url, response 98 | url = aurl 99 | 100 | # See https://docs.python.org/3/howto/urllib2.html 101 | # Default User-Agent is Python-urllib/n.m which often gets 403: Forbidden 102 | req = request.Request(url, None, {'User-Agent': 'Piety browser'}) 103 | 104 | print('Loading page...') # sometimes there is quite a delay in urlopen 105 | # If urlopen fails just let it crash, return to >>> and don't create buffer 106 | # Any error message from urlopen will be printed in REPL. 107 | 108 | response = request.urlopen(req) 109 | 110 | # If we get this far, urlopen must have succeeded. Create and fill buffer. 111 | purl = parse.urlparse(url) # return Parse object 112 | ppath = Path(purl.path) # extract ppath, a Path object, from Parse object 113 | bufname = ppath.name if ppath.name else purl.netloc # .name might be empty 114 | if not '.' in bufname: bufname += '.html' # avoid filename collision in e() 115 | fr.e(bufname) # create empty buffer, assign local bufname to ed.bufname 116 | # Special case handling of base URLs from particular web sites - see above 117 | baseurl = url # default, often the base url is the same as the page url 118 | for burl in baseurls: 119 | if url.startswith(burl): 120 | baseurl = burl 121 | ed.filename = baseurl # replace filename created by e() with baseurl 122 | ed.buffers[bufname]['filename'] = baseurl # replace filename created by e() 123 | buffer = ['\n'] # So content starts at index 1 not 0, like other buffers. 124 | ed.buffer.append(f'\n') # put page URL on first line 125 | ed.buffer.append('\n') 126 | # Fill in buffer text 127 | for line in response.readlines(): 128 | ed.buffer.append(line.decode('utf-8')) # FIXME? get encoding from HTTP 129 | fr.refresh() 130 | ed.dot = 1 # first line of content, top line of window, is index 1 not 0 131 | print(f'{ed.bufname}, {len(ed.buffer)} lines') # after '0 lines' from e() 132 | 133 | def gx(): 134 | """ 135 | Get web page at URL eXtracted from current line in current buffer. 136 | We expect user has selected a line 137 | that looks like it holds an absolute or relative URL. 138 | We expect that line is one of our footnotes on a rendered web page. 139 | We have arranged that the filename associated with the buffer that 140 | holds that rendered web page is actually the absolute URL of that page, 141 | so it is the base URL of any relative URLs that appear in footnotes. 142 | If there is no URL on the line, do nothing 143 | """ 144 | global url # So we can examine it in REPL 145 | url = xurl(ed.buffer[ed.dot]) # relative or absolute URL, '' if not found 146 | if url: # url is '' if no URL found on line 147 | g(url) 148 | 149 | def gr(url): 150 | 'Get and Render web page at url' 151 | g(url) 152 | render.r(url) 153 | 154 | def grx(): 155 | 'Get and Render web page at url eXtracted from current line in buffer' 156 | gx() 157 | render.r(url) # gx assigns global url 158 | 159 | def fnnum(line): 160 | """ 161 | Return integer footnote number of line, a string, or '' if not a footnote. 162 | Footnote number, if there is one, matches fnlinep regular expression 163 | """ 164 | m = fnlinep.search(line) 165 | fnstr = m.group() if m else '' 166 | return int(fnstr.strip()[:-1]) if fnstr else '' 167 | 168 | def fnurl(n): 169 | """ 170 | Return URL in buffer at footnote n or '' if footnote n not found. 171 | Example footnote line, a relative URL: '187. ?p=2\n' 172 | """ 173 | for line in ed.buffer: 174 | if n == fnnum(line): 175 | return xurl(line) # return breaks from loop 176 | return '' # not found 177 | 178 | def gf(n): 179 | 'Get web page whose URL is in footnote n' 180 | global url 181 | url = fnurl(n) 182 | g(url) 183 | 184 | def grf(n): 185 | 'Get and render web page whose URL is in footnote n' 186 | gf(n) 187 | render.r(url) # gf assigns global url 188 | 189 | def fnrefnum(): 190 | """ 191 | Return integer footnote reference number next on current line. 192 | Return 0 if no footnote found. 193 | """ 194 | m = fnrefp.search(ed.buffer[ed.dot][ed.point:]) 195 | fnrefn = m.group() if m else '' 196 | return int(fnrefn[1:-1]) if fnrefn else 0 197 | 198 | def gfx(): 199 | 'Get web page at next Footnote eXtracted from current line.' 200 | n = fnrefnum() # Footnote number, or 0 if no footnote on line 201 | g(fnurl(n)) # crashes if no footnote on line 202 | 203 | def grfx(): 204 | 'Get and Render web page at next Footnote eXtracted from current line.' 205 | gfx() 206 | render.r(url) # gfx assigns global url 207 | 208 | def clear_webpages(): 209 | 'Delete all webpages, buffers whose filename starts with http' 210 | ed.clear_buffers('all web pages', 211 | discard=(lambda buf: buf['filename'].startswith('http'))) 212 | 213 | # Add keycodes for browser operations to keymap 214 | dmacs.keymap[key.M_g] = gx # get page at URL on current line in buffer 215 | # FIXME? Overrides M_g: edsel.graffiti in dmacs 216 | dmacs.keymap[key.M_r] = render.r # render html from current buf. to .txt .buf 217 | dmacs.keymap[key.M_ret] = grx # get and render page at URL on current line 218 | dmacs.keymap[key.M_s] = grfx # get and render page at next footnote ref on line. 219 | 220 | -------------------------------------------------------------------------------- /threads/threads_1.txt: -------------------------------------------------------------------------------- 1 | threads_1.txt 2 | 3 | Experiments with Python threading using the pmacs editor with 4 | timers.py and writer.py 5 | 6 | The name of the timers module is plural, but the name of 7 | its timer function is singular. So we can say 'from timers import 8 | timer' then 'reload timers' without a name clash. 9 | 10 | To run these experiments, follow along in this file and type 11 | each command. To begin, at the system command prompt, type this 12 | command to define the PYTHONPATH so the commands work in any directory. 13 | Note the initial lone dot . at the start of the command: 14 | 15 | ...S . ~/Piety/bin/paths 16 | 17 | Then type this command to start the pmacs editor. 18 | 19 | ...% python3 -im pm 20 | 21 | An empty window appears with the cursor in it, and an empty Python REPL 22 | region below. 23 | 24 | Move the cursor to the Python REPL: type M-x ('meta x') by holding down 25 | the alt key and while you type the x key. Now you can type Python 26 | statments at the REPL prompt >>> 27 | 28 | The timer function repeats printing a timestamp message, with a given 29 | delay between messages. It has this signature: 30 | 31 | def timer(n=1, delay=1.0, label=''): ... 32 | 33 | The first example runs code in the REPL without threading. Type these 34 | commands at the Python REPL prompt: 35 | 36 | >>> from timers import timer 37 | 38 | >>> timer() 39 | 1 2024-01-12 21:06:40.986959 40 | 41 | The optional label argument can distinguish messages from different threads. 42 | 43 | >>> timer(3,1,'A') 44 | A 1 2024-01-12 21:06:51.511963 45 | A 2 2024-01-12 21:06:52.514235 46 | A 3 2024-01-12 21:06:53.516400 47 | 48 | You might want to print the messages to an editor buffer instead of the 49 | REPL. The writer module contains a write function that appends a string 50 | to the editor current buffer and displays the buffer in the editor focus 51 | window. To print the messages in the focus window, use redirect_stdout: 52 | 53 | >>> import writer 54 | 55 | >>> from contextlib import redirect_stdout 56 | 57 | >>> with redirect_stdout(writer) as buf: timer(3,1,'A') 58 | 59 | (messages appear in editor focus window) 60 | 61 | Now let's make two threads that execute concurrently, printing 62 | alternating lines of output. In this example ta and tb are threads that 63 | call timer 3 times, after 5 sec delay, with given label A or B 64 | 65 | >>> from threading import Thread 66 | 67 | Create the thread objects but don't start them: 68 | 69 | >>> ta = Thread(target=timer,args=(3,5,'A')) 70 | >>> tb = Thread(target=timer,args=(3,5,'B')) 71 | 72 | To interleave printing messages in the scrolling REPL type these 73 | commands at the REPL prompt. Use the Thread start() method to run them 74 | in the background. 75 | 76 | >>> ta.start() 77 | >>> tb.start() 78 | >>> A 1 2024-01-12 22:12:14.708146 79 | B 1 2024-01-12 22:12:17.277168 80 | A 2 2024-01-12 22:12:19.714243 81 | B 2 2024-01-12 22:12:22.283257 82 | A 3 2024-01-12 22:12:24.719974 83 | B 3 2024-01-12 22:12:27.289073 84 | 85 | We pass the long 5 sec delay to the timers here to give us time to type 86 | both start commands before messages start to appear in the REPL. 87 | 88 | You can only call start() on a thread once. To run the threads again, 89 | you have to make new thread objects by repeating the ta = ... statement etc. 90 | 91 | Alternatively, you don't have to first create a thread object 92 | at all, you can just create and start a thread in one statement: 93 | 94 | >>> Thread(target=timer,args=(3,5,'A')).start() 95 | >>> Thread(target=timer,args=(3,5,'B')).start() 96 | ... 97 | ... messages interleave 98 | ... 99 | 100 | Be sure to use the Thread start method, not the run method. 101 | Apparently the thread run() method runs the thread in the foreground and 102 | blocks the REPL until the thread exits. The thread start() method runs 103 | the thread in the background and returns to the REPL right away. 104 | 105 | Unfortunately redirect_stdout does not work with threads, so 106 | we can't use it to print interleaving messages in an editor buffer. 107 | We did some experiments that confirmed this. It is explained here: 108 | 109 | From https://docs.python.org/3/library/contextlib.html: 110 | 111 | "contextlib.redirect_stdout(new_target) 112 | Context manager for temporarily redirecting sys.stdout to another file 113 | or file-like object. ... 114 | 115 | Note that the global side effect on sys.stdout means that this context 116 | manager is not suitable for use in library code and most threaded 117 | applications. It also has no effect on the output of subprocesses. 118 | 119 | However, it is still a useful approach for many utility scripts." 120 | 121 | We can't use redirect_stdout with threads, but we can still write 122 | code that sends output from different threads to different destinations. 123 | 124 | The ptimer function has a destination keyword argument that can be used 125 | to send its timestamp messages to any editor buffer, not just the 126 | current buffer. Its signature is: 127 | 128 | def ptimer(n=1, delay=1.0, label='', destination=sys.stdout): ... 129 | 130 | The default destination argument sends the messages to the REPL. 131 | 132 | To send messages to a different buffer, we must create an instance of 133 | the Writer class from the writer module to collect and forward the messages 134 | (see writer.txt for more explanation). 135 | 136 | >>> from writer import Writer 137 | 138 | >>> abuf = Writer('a.txt') 139 | 140 | Create a buffer a.txt in the focus window. 141 | 142 | >>> e('a.txt') 143 | a.txt, 0 lines 144 | 145 | Then to send the messages to a.txt without threading: 146 | 147 | >>> from timers import ptimer 148 | 149 | >>> ptimer(3,5,'A',abuf) 150 | 151 | (a.txt updates in focus window) 152 | 153 | To send messages from two threads to a.txt, showing alternating messages, 154 | show a.txt in the focus window and type these commands: 155 | 156 | >>> Thread(target=ptimer,args=(5,10,'A', abuf)).start() 157 | >>> Thread(target=ptimer,args=(5,10,'B', abuf)).start() 158 | 159 | (A and B messages alternate in a.txt) 160 | 161 | We give ourselves a long 10 second delay so we can type the second 162 | Thread command before messages start appearing. During that 163 | delay, press the up-arrow key to restore the previous Thread command 164 | and use the inline REPL editor to change the A to B, then type 165 | return to start the second thread. 166 | 167 | We can update different windows from different threads. Create 168 | a buffer b.txt and show windows for both a.txt and b.txt. Create 169 | a Writer for b.txt: 170 | 171 | >>> o2() 172 | 173 | >>> e('b.txt') 174 | b.txt, 0 lines 175 | 176 | >>> bbuf = Writer('b.txt') 177 | 178 | Then issue these two commands. Note the second command prints to bbuf: 179 | 180 | >>> Thread(target=ptimer,args=(5,10,'A', abuf)).start() 181 | >>> Thread(target=ptimer,args=(5,10,'B', bbuf)).start() 182 | 183 | We can try shorter delays to see if the display can keep up without 184 | getting scrambled. 185 | 186 | The pm script starts the pmacs editor with the standard Python REPL, so 187 | it is not possible to type commands while the display is updating 188 | rapidly. It is not possible to restore the cursor to the correct location 189 | in the standard Python REPL command line after writing a message in a 190 | display window. 191 | 192 | However, we can work around that. We can type commands that edit one 193 | window at a time from a single thread, then retrieve those same commands 194 | to update two windows from two threads. 195 | 196 | First type this command, which writes 100 messages to a.txt in 10 seconds: 197 | 198 | >>> Thread(target=ptimer,args=(100,0.1,'A', abuf)).start() 199 | 200 | Then, after all the messages are printed, type this command, which does 201 | the same in b.txt: 202 | 203 | >>> Thread(target=ptimer, args=(100,0.1,'B',bbuf)).start() 204 | 205 | After all those messages are printed, retrieve the first command (for 206 | the a.txt buffer) by typing up-arrow in the REPL, then press RETURN to 207 | run it. While it is running, press the up-arrow again to retrieve the 208 | command for b.txt, then type RETURN to run that. Both buffers update 209 | in two windows. 210 | 211 | It is difficult to start the second thread when the first window is 212 | updating at 10 messages/sec because the cursor resets to the first column. 213 | Press up-arrow and when you see a B in the fragment of text on the line, 214 | press RET. 215 | 216 | With the 0.1 sec delay, both buffers update in two windows and no text 217 | is scrambled. We did not code any explicit synchronization between 218 | the threads and the pmacs editor, so it appears that the Python 219 | threading mechanism itself must allow each thread to always finish 220 | printing a complete line before it switches to another thread. 221 | 222 | The function threading.enumerate() lists the active threads. 223 | 224 | There is no simple way to kill a thread in Python. The thread has 225 | to be coded in a particular way to make this possible. 226 | 227 | Recall that the pm script starts the pmacs editor with the standard 228 | Python REPL, so so it is not possible to type commands while the display 229 | is updating rapidly because is not always possible to restore the cursor 230 | to the correct location in the standard Python REPL command line after 231 | writing a message in a display window. 232 | 233 | This is a serious limitation which often prevents using the standard 234 | Python REPL while displaying output from other tasks. Therefore we 235 | write our own custom Python REPL named pysh (rhymes with fish) that 236 | provides more control over the cursor in the Python command line. 237 | The experiments described in threads_2.txt and threads_3.txt demonstrate 238 | using our custom pysh REPL while other tasks rapidly update the display. 239 | 240 | Revised May 2024 241 | 242 | --------------------------------------------------------------------------------