├── README.md
├── hypothesis
    ├── Makefile
    ├── Pipfile
    ├── Pipfile.lock
    └── tests.py
├── vex
└── vexlib
    ├── cli.py
    ├── commands.py
    ├── errors.py
    ├── fs.py
    ├── project.py
    └── rson.py


/README.md:
--------------------------------------------------------------------------------
  1 | # `vex`: a database for files and directories.
  2 | 
  3 | Note: This is a work-in-progress, Please don't link to this project yet, it isn't ready for an audience yet. Thank you!
  4 | 
  5 | `vex` is a command line tool for tracking and sharing changes, like `hg`, `git`, or `svn`. Unlike other source
  6 | control systems, vex comes with `vex undo`, and `vex redo`.
  7 | 
  8 | ## Undo and Redo are faster than re-downloading and crying.
  9 | 
 10 | Added the wrong file? Run: `vex undo`
 11 | 
 12 | ```
 13 | $ vex add wrong_file.txt
 14 | $ vex undo
 15 | ```
 16 | 
 17 | Deleted an important file? Run `vex undo`
 18 | 
 19 | ```
 20 | $ vex remove important.txt
 21 | $ vex undo
 22 | ```
 23 | 
 24 | Restored the wrong file? Run `vex undo`
 25 | 
 26 | ```
 27 | $ vex restore file_with_changes.txt
 28 | $ vex undo
 29 | ```
 30 | 
 31 | Committed on the wrong branch? Yes, `vex undo`
 32 | 
 33 | ```
 34 | $ vex commit 
 35 | $ vex undo
 36 | $ vex branch:save_as testing
 37 | ```
 38 | 
 39 | Created a project with the wrong settings? ... `vex undo`!
 40 | 
 41 | ```
 42 | $ vex init . --include='*.py'
 43 | $ vex undo
 44 | $ vex init . --include='*.rb' --include='Gemspec'
 45 | ```
 46 | 
 47 | Undid something by accident? `vex redo`!
 48 | 
 49 | ```
 50 | $ vex undo
 51 | $ vex redo
 52 | ```
 53 | 
 54 | ## What makes vex different:
 55 | 
 56 | - Undo for almost everything!
 57 | - Commands are named after their purpose, rather than implemention.
 58 | - Tab Completion built in!
 59 | - Empty Directories are tracked too.
 60 | - You can work on a subdirectory, rather than the project as a whole.
 61 | - Output goes through `less` if it's too big for your terminal.
 62 | - Changed files do not need to be `vex add`'d before commit, just new ones
 63 | - You can change branches without having to commit or remove unsaved changes.
 64 | - Branches can have multiple sessions attached, with different changes (committed & uncommited).
 65 | 
 66 | ... and if there's a bug in `vex`, it tries its best to leave the project in a working state too!
 67 | 
 68 | ## Quick Install
 69 | 
 70 | Install python 3.6, I recommend using the wonderful "Homebrew" tool. Then, enable tab completion:
 71 | 
 72 | ```
 73 | $ alias vex=/path/to/project/vex # or `pwd`/vex if you're inside the project
 74 | $ complete -o nospace -O vex vex
 75 | ```
 76 | 
 77 | ### Command Cheatsheet
 78 | 
 79 | Every vex command looks like this: 
 80 | 
 81 | `vex <path:to:command> <--arguments> <--arguments=value> <value> ...`
 82 | 
 83 | `vex help <command>` will show the manual, and `vex <command> --help` shows usage.
 84 | 
 85 | Commands can be one or more names seperated by `:`, like `vex commit` or `vex undo:list`
 86 | 
 87 | ### Undo/Redo
 88 | 
 89 | | `vex` | `hg` | `git` |
 90 | | ----- | ----- | ----- |
 91 | | `vex undo`		| `hg rollback` for commits	| `git reset --hard HEAD~1` for commits |
 92 | | `vex redo`		| ...             	        | ... 	|
 93 | | `vex undo:list`	| ...             	        | ... 	|
 94 | | `vex redo:list`	| ...             	        | ... 	|
 95 | 
 96 | ### General
 97 | 
 98 | | `vex` | `hg` | `git` |
 99 | | ----- | ----- | ----- |
100 | | `vex init`		| `hg init`		    | `git init` 	|
101 | | `vex status`		| `hg status`	    | `git add -u && git status` 	|
102 | | `vex log`	    	| `hg log`		    | `git log --first-parent` 	|
103 | | `vex diff`	       	| `hg diff`	    | `git diff --cached` 	|
104 | | `vex diff:branch`    	| `hg diff`   	| `git diff @{upstream}` 	|
105 | 
106 | ### Files
107 | 
108 | | `vex` | `hg` | `git` |
109 | | ----- | ----- | ----- |
110 | | `vex add`	    	| `hg add`	    	| `git add` |
111 | | `vex forget`		| `hg forget`   	| `git remove --cached (-r)` 	|
112 | | `vex remove`		| `hg remove`   	| `git remove (-r)` 	|
113 | | `vex restore`		| `hg revert`   	| `git checkout -- <file>` |
114 | | `vex switch`		|  N/A | N/A |
115 | 
116 | ### Commits
117 | 
118 | | `vex` | `hg` | `git` |
119 | | ----- | ----- | ----- |
120 | | `vex id`		        | `hg id`   	| `git rev-parse HEAD` 	|
121 | | `vex commit`		    | `hg commit`	| `git commit -a` 	|
122 | | `vex commit:amend`	| ...        	| `git commit --amend` 	|
123 | | `vex message:edit` | ... | ... |
124 | | `vex message:get` | ... | ... |
125 | | `vex message:amend` | ... | ... |
126 | 
127 | ### Branches
128 | 
129 | | `vex` | `hg` | `git` |
130 | | ----- | ----- | ----- |
131 | | `vex branch:new`                  | `hg bookmark -i`, `hg update -r` | `git checkout -b new old`|
132 | | `vex branch:open`                 | `hg update -r <name>` | `git checkout` |
133 | | `vex branch:saveas`               | `hg bookmark <name>` | `git checkout -b new`|
134 | | `vex branch:rename`               | `hg bookmark --rename` | `git branch -m new`|
135 | | `vex branch:swap`                 | ... | ... |
136 | | `vex branches` / `branch:list`    | `hg bookmark` | `git branch --list` |
137 | 
138 | ### Options/Arguments
139 | 
140 | The argument to a command can take one the following form:
141 | 
142 | - Boolean: `--name`, `--name=true`, `--name=false`
143 | - Single value `--name=...`
144 | - Multiple values `--name=... --name=...`
145 | - Positional `vex command <value> <value>`
146 | 
147 | There are no single letter flags like, `-x`. Tab completion works, and vex opts for a new command over a flag to change behaviour.
148 | 
149 | 
150 | 
151 | ## Workflows
152 | 
153 | ### Creating a project
154 | 
155 | `vex init` or `vex init <directory>`. 
156 | 
157 | ```
158 | $ vex init . --include="*.py" --ignore="*.py?"
159 | $ vex add
160 | $ vex status
161 | ... # *.py files added and waiting for commit
162 | ```
163 | 
164 | By default, `vex init name` creates a repository with a `/name` directory inside. 
165 | 
166 | (Note: `vex undo`/`vex redo` will undo `vex init`, but leave the `/.vex` directory intact)
167 | 
168 | You can create a git backed repo with `vex init --git`, or `vex git:init`. The latter command creates a bare, empty git repository, but the first one will include settings files inside the first commit. `vex git:init` also defaults to a checkout prefix of `/` and a branch called `master`. `vex init --git` uses the same defaults as `vex init`, a prefix that matches the name of the working directory, and a branch called `latest`.
169 | 
170 | ### Undoing changes
171 | 
172 | - `vex undo` undoes the last command that changed something
173 | - `vex undo:list` shows the list of commands that can be undone
174 | - `vex undo` can be run more than once,
175 | - `vex redo` redoes the last undone command
176 | - `vex redo:list` shows the potential options, `vex redo --choice=<n>` to pick one
177 | - `vex redo` can be run more than once, and redone more than once
178 | 
179 | ### Moving around the project
180 | 
181 | - `vex switch` (change repo subtree checkout)
182 | 
183 | ### Adding/removing files from the project
184 | 
185 | - `vex add`
186 | - `vex forget` 
187 | - `vex ignore` 
188 | - `vex include`
189 | - `vex remove` 
190 | - `vex restore` 
191 | 
192 | TODO
193 | 
194 | - `vex restore --pick` *
195 | 
196 | ### File properties
197 | 
198 | - `vex fileprops` / `vex properties`
199 | - `vex fileprops:set`
200 | 
201 | ### Inspecting changes
202 | 
203 | - `vex log`
204 | - `vex status`
205 | - `vex diff <file> <file...>`
206 | - `vex diff:branch <branch>`
207 | 
208 | ### Saving changes
209 | 
210 | - `vex message` edit message
211 | - `vex message:get` print message
212 | - `vex commit <files>`
213 | - `vex commit:prepare` / 'vex commit:prepared'
214 | 
215 | TODO 
216 | 
217 | - `vex commit:amend` *
218 | - `vex commit --pick` * 
219 | - `vex commit:rollback` *
220 | - `vex commit:revert` *
221 | - `vex commit:squash` * (flatten a branch)
222 | 
223 | ### Working on a branch
224 | 
225 | - `vex branch` what branch are you on
226 | - `vex branch:new` create a new branch
227 | - `vex branch:open` open existing branch
228 | - `vex branch:saveas`  (see `git checkout -b`)
229 | - `vex branch:swap`  
230 | - `vex branch:rename` 
231 | - `vex branches` list all branches
232 | 
233 | TODO
234 | 
235 | - `vex branch:close` *
236 | 
237 | ### Working on an anonymous branch
238 | 
239 | - `vex session` (see all open branches/anonymous branches)
240 | - `vex sessions`
241 | 
242 | TODO
243 | 
244 | - `vex session:new` *
245 | - `vex session:open` *
246 | - `vex session:detach` *
247 | - `vex session:attach` *
248 | - `vex rewind`  * (like git checkout)
249 | 
250 | ## TODO
251 | 
252 | ### Applying changes from a branch *
253 | 
254 | When applying changes from another branch, vex creates new commits with new timestamps
255 | 
256 | - `vex commit:apply <branch>` *
257 | - `vex commit:append <branch>` *
258 |    create a new commit for each change in the other branch, as-is
259 | - `vex commit:replay <branch>` *
260 |    create a new commit for each change in the branch, reapplying the changes
261 | - `vex commit:apply --squash` *
262 |    create a new commit with the changes from the other branch
263 | - `vex commit:apply --pick`  *
264 | 
265 | ### Rebuilding a branch with upsteam changes *
266 | 
267 | - `vex update` * (rebase, --all affecting downstream branches too) 
268 | - `vex update --manual` * (handle conflcts)
269 | 
270 | ### Subprojects *
271 | 
272 | - `vex project:init name` *
273 | - `vex mount <branch/remote> <path>` *
274 | 
275 | ### Sharing changes *
276 | 
277 | - `vex export` * 
278 | - `vex import` *
279 | - `vex pull` *
280 | - `vex push` *
281 | - `vex sync` * (pull, update, push)
282 | - `vex remotes` *
283 | - `vex remotes:add` *
284 | - `vex serve` *
285 | - `vex clone` *
286 | 
287 | ### Cleaning the changelog
288 | 
289 | - `vex purge` *
290 | - `vex truncate` *
291 | 
292 | ### Project settings *
293 | 
294 | - `vex project:settings` * 
295 | - `vex project:authors`  *
296 | - `vex project:set author.name ...` *
297 | 
298 | ### Branch Settings *
299 | 
300 | - `vex baranch:lock <branch>` * (sets a `/.vex/settings/policy` file inside a branch with a `{branch-name: policy } ` entry
301 | - `vex branch:set ..` *
302 | 
303 | ### Subcommands *
304 | 
305 | Store some environment variables and entry points in a settings file, and run those commands
306 | 
307 | - `vex env` *
308 | - `vex exec` / `vex run` *
309 | - `vex exec:make` *
310 | - `vex exec:test` *
311 | 
312 | The directory `/.vex/settings/bin/` inside working copy is tracked, and added to the `PATH`
313 | 
314 | ### Scripting
315 | 
316 | - `vex --rson/--json/--yaml` * (note: subset of yaml)
317 | - `vex debug`
318 | - `vex debug:cat` *
319 | - `vex debug:ls` *
320 | 
321 | ### Versioning
322 | 
323 | - `vex commit --major/--minor/--patch`  *
324 | - /.vex/setting/features *
325 | 
326 | ## Debugging
327 | 
328 | If `vex` crashes with a known exception, it will not print the stack trace by default. Additionally, after a crash, `vex` will try to roll back any unfinished changes. Use `vex debug <command>` to always print the stack trace, but this will leave the repo in a broken state, so use `vex debug:rollback` to rollback manually.
329 | 
330 | Note: some things just break horribly, and a `rollback` isn't enough. Use `vex fake <command>` to not break things and see what would be changed (for some commands, they rely on changes being written and so `fake` gives weird results, sorry).
331 | 
332 | 


--------------------------------------------------------------------------------
/hypothesis/Makefile:
--------------------------------------------------------------------------------
1 | 
2 | all:
3 | 	true
4 | install: 
5 | 	pipenv install
6 | test:
7 | 	pipenv run pytest tests.py
8 | 


--------------------------------------------------------------------------------
/hypothesis/Pipfile:
--------------------------------------------------------------------------------
 1 | [[source]]
 2 | 
 3 | url = "https://pypi.python.org/simple"
 4 | verify_ssl = true
 5 | name = "pypi"
 6 | 
 7 | 
 8 | [packages]
 9 | 
10 | hypothesis = "*"
11 | pytest = "*"
12 | 
13 | [dev-packages]
14 | 
15 | [requires]
16 | 
17 | python_version = "3.6"
18 | 


--------------------------------------------------------------------------------
/hypothesis/Pipfile.lock:
--------------------------------------------------------------------------------
  1 | {
  2 |     "_meta": {
  3 |         "hash": {
  4 |             "sha256": "64a237dd64f0a5893ffe9df6e518c8321c80181631fe4b55adc2fe08fff9e756"
  5 |         },
  6 |         "host-environment-markers": {
  7 |             "implementation_name": "cpython",
  8 |             "implementation_version": "3.6.4",
  9 |             "os_name": "posix",
 10 |             "platform_machine": "x86_64",
 11 |             "platform_python_implementation": "CPython",
 12 |             "platform_release": "13.4.0",
 13 |             "platform_system": "Darwin",
 14 |             "platform_version": "Darwin Kernel Version 13.4.0: Wed Mar 18 16:20:14 PDT 2015; root:xnu-2422.115.14~1/RELEASE_X86_64",
 15 |             "python_full_version": "3.6.4",
 16 |             "python_version": "3.6",
 17 |             "sys_platform": "darwin"
 18 |         },
 19 |         "pipfile-spec": 6,
 20 |         "requires": {
 21 |             "python_version": "3.6"
 22 |         },
 23 |         "sources": [
 24 |             {
 25 |                 "name": "pypi",
 26 |                 "url": "https://pypi.python.org/simple",
 27 |                 "verify_ssl": true
 28 |             }
 29 |         ]
 30 |     },
 31 |     "default": {
 32 |         "atomicwrites": {
 33 |             "hashes": [
 34 |                 "sha256:a24da68318b08ac9c9c45029f4a10371ab5b20e4226738e150e6e7c571630ae6",
 35 |                 "sha256:240831ea22da9ab882b551b31d4225591e5e447a68c5e188db5b89ca1d487585"
 36 |             ],
 37 |             "version": "==1.1.5"
 38 |         },
 39 |         "attrs": {
 40 |             "hashes": [
 41 |                 "sha256:4b90b09eeeb9b88c35bc642cbac057e45a5fd85367b985bd2809c62b7b939265",
 42 |                 "sha256:e0d0eb91441a3b53dab4d9b743eafc1ac44476296a2053b6ca3af0b139faf87b"
 43 |             ],
 44 |             "version": "==18.1.0"
 45 |         },
 46 |         "coverage": {
 47 |             "hashes": [
 48 |                 "sha256:7608a3dd5d73cb06c531b8925e0ef8d3de31fed2544a7de6c63960a1e73ea4bc",
 49 |                 "sha256:3a2184c6d797a125dca8367878d3b9a178b6fdd05fdc2d35d758c3006a1cd694",
 50 |                 "sha256:f3f501f345f24383c0000395b26b726e46758b71393267aeae0bd36f8b3ade80",
 51 |                 "sha256:0b136648de27201056c1869a6c0d4e23f464750fd9a9ba9750b8336a244429ed",
 52 |                 "sha256:337ded681dd2ef9ca04ef5d93cfc87e52e09db2594c296b4a0a3662cb1b41249",
 53 |                 "sha256:3eb42bf89a6be7deb64116dd1cc4b08171734d721e7a7e57ad64cc4ef29ed2f1",
 54 |                 "sha256:be6cfcd8053d13f5f5eeb284aa8a814220c3da1b0078fa859011c7fffd86dab9",
 55 |                 "sha256:69bf008a06b76619d3c3f3b1983f5145c75a305a0fea513aca094cae5c40a8f5",
 56 |                 "sha256:2eb564bbf7816a9d68dd3369a510be3327f1c618d2357fa6b1216994c2e3d508",
 57 |                 "sha256:9d6dd10d49e01571bf6e147d3b505141ffc093a06756c60b053a859cb2128b1f",
 58 |                 "sha256:701cd6093d63e6b8ad7009d8a92425428bc4d6e7ab8d75efbb665c806c1d79ba",
 59 |                 "sha256:5a13ea7911ff5e1796b6d5e4fbbf6952381a611209b736d48e675c2756f3f74e",
 60 |                 "sha256:c1bb572fab8208c400adaf06a8133ac0712179a334c09224fb11393e920abcdd",
 61 |                 "sha256:03481e81d558d30d230bc12999e3edffe392d244349a90f4ef9b88425fac74ba",
 62 |                 "sha256:28b2191e7283f4f3568962e373b47ef7f0392993bb6660d079c62bd50fe9d162",
 63 |                 "sha256:de4418dadaa1c01d497e539210cb6baa015965526ff5afc078c57ca69160108d",
 64 |                 "sha256:8c3cb8c35ec4d9506979b4cf90ee9918bc2e49f84189d9bf5c36c0c1119c6558",
 65 |                 "sha256:7e1fe19bd6dce69d9fd159d8e4a80a8f52101380d5d3a4d374b6d3eae0e5de9c",
 66 |                 "sha256:6bc583dc18d5979dc0f6cec26a8603129de0304d5ae1f17e57a12834e7235062",
 67 |                 "sha256:198626739a79b09fa0a2f06e083ffd12eb55449b5f8bfdbeed1df4910b2ca640",
 68 |                 "sha256:7aa36d2b844a3e4a4b356708d79fd2c260281a7390d678a10b91ca595ddc9e99",
 69 |                 "sha256:3d72c20bd105022d29b14a7d628462ebdc61de2f303322c0212a054352f3b287",
 70 |                 "sha256:4635a184d0bbe537aa185a34193898eee409332a8ccb27eea36f262566585000",
 71 |                 "sha256:e05cb4d9aad6233d67e0541caa7e511fa4047ed7750ec2510d466e806e0255d6",
 72 |                 "sha256:76ecd006d1d8f739430ec50cc872889af1f9c1b6b8f48e29941814b09b0fd3cc",
 73 |                 "sha256:7d3f553904b0c5c016d1dad058a7554c7ac4c91a789fca496e7d8347ad040653",
 74 |                 "sha256:3c79a6f7b95751cdebcd9037e4d06f8d5a9b60e4ed0cd231342aa8ad7124882a",
 75 |                 "sha256:56e448f051a201c5ebbaa86a5efd0ca90d327204d8b059ab25ad0f35fbfd79f1",
 76 |                 "sha256:ac4fef68da01116a5c117eba4dd46f2e06847a497de5ed1d64bb99a5fda1ef91",
 77 |                 "sha256:1c383d2ef13ade2acc636556fd544dba6e14fa30755f26812f54300e401f98f2",
 78 |                 "sha256:b8815995e050764c8610dbc82641807d196927c3dbed207f0a079833ffcf588d",
 79 |                 "sha256:104ab3934abaf5be871a583541e8829d6c19ce7bde2923b2751e0d3ca44db60a",
 80 |                 "sha256:9e112fcbe0148a6fa4f0a02e8d58e94470fc6cb82a5481618fea901699bf34c4",
 81 |                 "sha256:15b111b6a0f46ee1a485414a52a7ad1d703bdf984e9ed3c288a4414d3871dcbd",
 82 |                 "sha256:e4d96c07229f58cb686120f168276e434660e4358cc9cf3b0464210b04913e77",
 83 |                 "sha256:f8a923a85cb099422ad5a2e345fe877bbc89a8a8b23235824a93488150e45f6e"
 84 |             ],
 85 |             "version": "==4.5.1"
 86 |         },
 87 |         "hypothesis": {
 88 |             "hashes": [
 89 |                 "sha256:ceb4d9b582184b041adc5647121fbafe3fcf49b7fbd218195d903c3fc6bc7916"
 90 |             ],
 91 |             "version": "==3.57.0"
 92 |         },
 93 |         "more-itertools": {
 94 |             "hashes": [
 95 |                 "sha256:a18d870ef2ffca2b8463c0070ad17b5978056f403fb64e3f15fe62a52db21cc0",
 96 |                 "sha256:6703844a52d3588f951883005efcf555e49566a48afd4db4e965d69b883980d3",
 97 |                 "sha256:2b6b9893337bfd9166bee6a62c2b0c9fe7735dcf85948b387ec8cba30e85d8e8"
 98 |             ],
 99 |             "version": "==4.2.0"
100 |         },
101 |         "pluggy": {
102 |             "hashes": [
103 |                 "sha256:d345c8fe681115900d6da8d048ba67c25df42973bda370783cd58826442dcd7c",
104 |                 "sha256:e160a7fcf25762bb60efc7e171d4497ff1d8d2d75a3d0df7a21b76821ecbf5c5",
105 |                 "sha256:7f8ae7f5bdf75671a718d2daf0a64b7885f74510bcd98b1a0bb420eb9a9d0cff"
106 |             ],
107 |             "version": "==0.6.0"
108 |         },
109 |         "py": {
110 |             "hashes": [
111 |                 "sha256:983f77f3331356039fdd792e9220b7b8ee1aa6bd2b25f567a963ff1de5a64f6a",
112 |                 "sha256:29c9fab495d7528e80ba1e343b958684f4ace687327e6f789a94bf3d1915f881"
113 |             ],
114 |             "version": "==1.5.3"
115 |         },
116 |         "pytest": {
117 |             "hashes": [
118 |                 "sha256:c76e93f3145a44812955e8d46cdd302d8a45fbfc7bf22be24fe231f9d8d8853a",
119 |                 "sha256:39555d023af3200d004d09e51b4dd9fdd828baa863cded3fd6ba2f29f757ae2d"
120 |             ],
121 |             "version": "==3.6.0"
122 |         },
123 |         "six": {
124 |             "hashes": [
125 |                 "sha256:832dc0e10feb1aa2c68dcc57dbb658f1c7e65b9b61af69048abc87a2db00a0eb",
126 |                 "sha256:70e8a77beed4562e7f14fe23a786b54f6296e34344c23bc42f07b15018ff98e9"
127 |             ],
128 |             "version": "==1.11.0"
129 |         }
130 |     },
131 |     "develop": {}
132 | }
133 | 


--------------------------------------------------------------------------------
/hypothesis/tests.py:
--------------------------------------------------------------------------------
 1 | import tempfile
 2 | import shutil
 3 | import os
 4 | import os.path
 5 | import sys
 6 | import subprocess
 7 | 
 8 | from hypothesis import given
 9 | from hypothesis.strategies import text, integers, lists
10 | from hypothesis.stateful import rule, precondition, RuleBasedStateMachine, Bundle
11 | 
12 | 
13 | vexpath = os.path.normpath(os.path.join(os.path.split(os.path.abspath(__file__))[0], "..", "vex"))
14 | 
15 | class Vex:
16 |     def __init__(self, path, dir, command=()):
17 |         self.path = path
18 |         self.dir=dir
19 |         self.command = command
20 | 
21 |     def __getattr__(self, name):
22 |         return self.__class__(self.path, self.dir, command=self.command+(name,))
23 | 
24 |     def __call__(self, *args, **kwargs):
25 |         cmd = [sys.executable]
26 |         cmd.append(self.path)
27 |         if self.command:
28 |             cmd.append(":".join(self.command))
29 |         for name, value in kwargs.items():
30 |             if value is True:
31 |                 cmd.append("--{}=true".format(name))
32 |             elif value is False:
33 |                 cmd.append("--{}=false".format(name))
34 |             elif isinstance(value, (list, tuple)):
35 |                 for v in value:
36 |                     cmd.append("--{}={}".format(name, v))
37 |             else:
38 |                 cmd.append("--{}={}".format(name, value))
39 |         for value in args:
40 |             cmd.append(value)
41 | 
42 |         p=  subprocess.run(cmd, stdout=subprocess.PIPE, cwd=self.dir)
43 |         if p.returncode:
44 |             sys.stdout.buffer.write(p.stdout)
45 |             raise Exception('Error')
46 |         return p.stdout
47 | 
48 | class VexMachine(RuleBasedStateMachine):
49 |     def __init__(self):
50 |         self.tempd = tempfile.mkdtemp()
51 |         self.vex= Vex(vexpath, self.tempd)
52 |         self.vex.init(prefix='/')
53 |         RuleBasedStateMachine.__init__(self)
54 | 
55 |     def teardown(self):
56 |         shutil.rmtree(self.tempd)
57 |         RuleBasedStateMachine.teardown(self)
58 | 
59 |     @rule()
60 |     def status(self):
61 |         self.vex.status()
62 | 
63 | 
64 | TestHeap = VexMachine.TestCase
65 | 


--------------------------------------------------------------------------------
/vex:
--------------------------------------------------------------------------------
 1 | #!/usr/bin/env python3 
 2 | """
 3 | 
 4 | vex is a command line program for saving changes to a project, switching
 5 | between different versions, and sharing those changes.
 6 | 
 7 | vex supports bash completion: run `complete -o nospace -C vex vex`
 8 | 
 9 | """
10 | import os, os.path
11 | import sys
12 | import subprocess
13 | import tempfile
14 | 
15 | if sys.version_info.major < 3 or (sys.version_info.minor < 6)  or (sys.version_info.minor == 6  and sys.implementation.name != 'cpython'):
16 |     print('Minimum Python 3.7 (or CPython 3.6')
17 |     sys.exit(-1)
18 | sys.path.append(os.path.split(os.path.abspath(__file__))[0])
19 | 
20 | from vexlib.commands import vex_cmd
21 | 
22 | vex_cmd.main(__name__)
23 | 


--------------------------------------------------------------------------------
/vexlib/cli.py:
--------------------------------------------------------------------------------
  1 | """ abandon all faith all ye who enter here
  2 | 
  3 | A program consists of `cli.Commands()`, chained together, used to 
  4 | decorate functions to dispatch:
  5 | 
  6 | ```
  7 | cmd = cli.Command('florb','florb the morps')
  8 | @cmd.on_run("one [two] [three...]")
  9 | def cmd_run(one, two, three):
 10 |     print([one, two, three])
 11 | ```
 12 | 
 13 | gives this output:
 14 | 
 15 | ```
 16 | $ florb one 
 17 | ["one", None, []]
 18 | 
 19 | $ florb one two three four
 20 | ["one", "two", ["three", "four]]
 21 | ```
 22 | 
 23 | If needed, these options can be passed *entirely* as flags: 
 24 | 
 25 | ```
 26 | $ florb --one=one --two=two --three=three --three=four
 27 | ["one", "two", ["three", "four]]
 28 | ```
 29 | 
 30 | ## Subcommands
 31 | 
 32 | `Command` can be nested, giving a `cmd one <args>` `cmd two <args>` like interface:
 33 | ``` 
 34 | root = cli.Command('example', 'my program')
 35 | 
 36 | subcommand = cli.subcommand('one', 'example subcommand')
 37 | 
 38 | @subcommand.on_run(...)
 39 | def subcommand_run(...):
 40 |     ...
 41 | ```
 42 | 
 43 | `cmd help one` `cmd one --help`, `cmd help two` `cmd two --help` will print out the manual and usage for `one` and `two` respectively.
 44 | 
 45 | The parameter to `on_run()`, is called an argspec.
 46 | 
 47 | ## Argspec
 48 | 
 49 | An argspec is a string that describes how to turn CLI arguments into a dictionary of name, value pairs. For example:
 50 | 
 51 | - "x y z" given "1 2 3" gives {"x":1, "y":2, "z":3}
 52 | - "[x...]" given "a b c" gives {"x":["a","b","c"]}
 53 | 
 54 | This is used to call the function underneath, so every value in the function must be present in the argspec. When no argspec is provided, `textfree86` defaults to a string of argument names, i.e `foo(x,y,z)` gets `"x y z"`. 
 55 | 
 56 | The dictionary passed will contain a value for every name in the argspec. An argspec resembles a usage string, albeit with a standard formatting for flags or other command line options:
 57 | 
 58 | - `--name?` describes a switch, which defaults to `False`, but when present, is set to `True`, additionally, `--name=true` and `--name=false` both work.
 59 | 
 60 | - `--name` describes a normal flag, which defaults to `None`, and on the CLI `--name=value` sets it.
 61 | 
 62 | - `--name...` describes a list flag, which defaults to `[]`, and on the CLI `--name=value` appends to it
 63 | 
 64 | - `name` describes a positional argument. It must come after any flags and before any optional positional arguments.
 65 | 
 66 | - `[name]` describes an optional positional argument. If one arg is given for two optional positional args, like `[x] [y]`, then the values are assigned left to right.
 67 | 
 68 | - `[name...]` describes a tail positonal argument. It defaults to `[]`, and all remaining arguments are appended to it.
 69 | 
 70 | A short argspec has four parts, `<flags> <positional> [<optional positional>]* [<tail positional>...]`
 71 | 
 72 | ### Long Argspec
 73 | 
 74 | Passing a multi-line string allows you to pass in short descriptions of the arguments, using `# ...` at the end of each line.
 75 | 
 76 | ```
 77 | demo = cli.Command('demo', 'cli example programs')
 78 | @demo.on_run('''
 79 |     --switch?       # a demo switch
 80 |     --value:str     # pass with --value=...
 81 |     --bucket:int... # a list of numbers
 82 |     pos1            # positional
 83 |     [opt1]          # optional 1
 84 |     [opt2]          # optional 2
 85 |     [tail...]       # tail arg
 86 | ''')
 87 | def run(switch, value, bucket, pos1, opt1, opt2, tail):
 88 |     '''a demo command that shows all the types of options'''
 89 |     return [switch, value, bucket, pos1, opt1, opt2, tail]
 90 | ```
 91 | 
 92 | ### Argument Types
 93 | 
 94 | A field can contain a parsing instruction, `x:string` or `x:int`
 95 | 
 96 | - `int`, `integer`
 97 | - `float`, `num`, `number`
 98 | - `str`, `string`
 99 | - `bool`, `boolean` (accepts 'true', 'false')
100 | 
101 | Example `arg1:str arg2:int` describes a progrm that would accept `foo 123` as input, passing `{'arg1': 'foo', 'arg2': 123}` to the function.
102 | 
103 | A scalar field tries to convert the argument to an integer or floating point number, losslessly, and if successful, uses that.
104 | 
105 | The default is 'string'
106 | 
107 | """
108 | 
109 | import io
110 | import os
111 | import sys
112 | import time
113 | import types
114 | import itertools
115 | import subprocess
116 | import traceback
117 | 
118 | 
119 | ### Argspec / Parsing
120 | 
121 | class BadArg(Exception):
122 |     def action(self, path):
123 |         return Action("error", path, {}, errors=self.args)
124 | 
125 | class BadDefinition(Exception):
126 |     pass
127 | 
128 | class Complete:
129 |     def __init__(self, prefix, name, argtype):
130 |         self.prefix = prefix
131 |         self.name = name
132 |         self.argtype = argtype
133 | 
134 | class Argspec:
135 |     def __init__(self, switches, flags, lists, positional, optional, tail, argtypes, descriptions):
136 |         self.switches = switches
137 |         self.flags = flags
138 |         self.lists = lists
139 |         self.positional = positional
140 |         self.optional = optional 
141 |         self.tail = tail
142 |         self.argtypes = argtypes
143 |         self.descriptions = descriptions
144 | 
145 | ARGTYPES=[x.strip() for x in """
146 |     bool boolean
147 |     int integer
148 |     float num number
149 |     str string
150 |     scalar
151 |     path 
152 |     branch
153 |     commit
154 | """.split() if x]
155 | #   stretch goals: rwfile jsonfile textfile
156 | 
157 | def parse_argspec(argspec):
158 |     """
159 |         argspec is a short description of a command's expected args:
160 |         "x y z"         three args (x,y,z) in that order
161 |         "x [y] [z]"       three args, where the second two are optional. 
162 |                         "arg1 arg2" is x=arg1, y=arg2, z=null
163 |         "x y [z...]"      three args (x,y,z) where the final arg can be repeated 
164 |                         "arg1 arg2 arg3 arg4" is z = [arg3, arg4]
165 | 
166 |         an argspec comes in the following format, and order
167 |             <flags> <positional> <optional> <tail>
168 | 
169 |         for a option named 'foo', a:
170 |             switch is '--foo?'
171 |                 `cmd --foo` foo is True
172 |                 `cmd`       foo is False
173 |             flag is `--foo`
174 |                 `cmd --foo=x` foo is 'x'
175 |             list is `--foo...`
176 |                 `cmd` foo is []
177 |                 `cmd --foo=1 --foo=2` foo is [1,2]
178 |             positional is `foo`
179 |                 `cmd x`, foo is `x`
180 |             optional is `[foo]`
181 |                 `cmd` foo is null
182 |                 `cmd x` foo is x 
183 |             tail is `[foo...]` 
184 |                 `cmd` foo is []
185 |                 `cmd 1 2 3` foo is [1,2,3] 
186 |     """
187 |     positional = []
188 |     optional = []
189 |     tail = None
190 |     flags = []
191 |     lists = []
192 |     switches = []
193 |     descriptions = {}
194 | 
195 |     if '\n' in argspec:
196 |         args = [line for line in argspec.split('\n') if line]
197 |     else:
198 |         if argspec.count('#') > 0:
199 |             raise BadDefinition('BUG: comment in single line argspec')
200 |         args = [x for x in argspec.split()]
201 | 
202 |     argtypes = {}
203 |     argnames = set()
204 | 
205 |     def argdesc(arg):
206 |         if '#' in arg:
207 |             arg, desc = arg.split('#', 1)
208 |             return arg.strip(), desc.strip()
209 |         else:
210 |             return arg.strip(), None
211 | 
212 |     def argname(arg, desc, argtype=None):
213 |         if not arg:
214 |             return arg
215 |         if ':' in arg:
216 |             name, atype = arg.split(':')
217 |             if atype not in ARGTYPES:
218 |                 raise BadDefinition("BUG: option {} has unrecognized type {}".format(name, atype))
219 |             argtypes[name] = atype 
220 |         else:
221 |             name = arg
222 |             if argtype:
223 |                 argtypes[name] = argtype 
224 |         if name in argnames:
225 |             raise BadDefinition('BUG: duplicate arg name {}'.format(name))
226 |         argnames.add(name)
227 |         if desc:
228 |             descriptions[name] = desc
229 |         return name
230 | 
231 |     nargs = len(args) 
232 |     while args: # flags
233 |         arg, desc = argdesc(args[0])
234 |         if not arg.startswith('--'): 
235 |             break
236 |         else:
237 |             args.pop(0)
238 |         if arg.endswith('?'):
239 |             if ':' in arg:
240 |                 raise BadDefinition('BUG: boolean switches cant have types: {!r}'.format(arg))
241 |             switches.append(argname(arg[2:-1], desc, 'boolean'))
242 |         elif arg.endswith('...'):
243 |             lists.append(argname(arg[2:-3], desc))
244 |         elif arg.endswith('='):
245 |             flags.append(argname(arg[2:-1], desc))
246 |         else:
247 |             flags.append(argname(arg[2:],desc))
248 | 
249 |     while args: # positional
250 |         arg, desc = argdesc(args[0])
251 |         if arg.startswith('--'): raise BadDefinition('positional arguments must come after flags')
252 | 
253 |         if arg.endswith(('...]', ']', '...')) : 
254 |             break
255 |         else:
256 |             args.pop(0)
257 | 
258 |         positional.append(argname(arg, desc))
259 | 
260 |     if args and args[0].endswith('...'):
261 |         arg, desc = argdesc(args.pop(0))
262 |         if arg.startswith('--'): raise BadDefinition('flags must come before positional args')
263 |         if arg.startswith('['): raise BadDefinition('badarg')
264 |         tail = argname(arg[:-3], desc)
265 |     elif args:
266 |         while args: # optional
267 |             arg, desc = argdesc(args[0])
268 |             if arg.startswith('--'): raise BadDefinition('badarg')
269 |             if arg.endswith('...]'): 
270 |                 break
271 |             else:
272 |                 args.pop(0)
273 |             if not (arg.startswith('[') and arg.endswith(']')): raise BadDefinition('badarg')
274 | 
275 |             optional.append(argname(arg[1:-1], desc))
276 | 
277 |         if args: # tail
278 |             arg, desc = argdesc(args.pop(0))
279 |             if arg.startswith('--'): raise BadDefinition('badarg')
280 |             if not arg.startswith('['): raise BadDefinition('badarg')
281 |             if not arg.endswith('...]'): raise BadDefinition('badarg')
282 |             tail = argname(arg[1:-4], desc)
283 | 
284 |     if args:
285 |         raise BadDefinition('bad argspec')
286 |     
287 |     # check names are valid identifiers
288 | 
289 |     return nargs, Argspec(
290 |             switches = switches,
291 |             flags = flags,
292 |             lists = lists,
293 |             positional = positional, 
294 |             optional = optional , 
295 |             tail = tail, 
296 |             argtypes = argtypes,
297 |             descriptions = descriptions,
298 |     )
299 | 
300 | def parse_err(error, pos, argv):
301 |     message = ["{}, at argument #{}:".format(error, pos)]
302 |     if pos-3 >0:
303 |             message.append(" ... {} args before".format(pos-3))
304 |     for i, a in enumerate(argv[pos-3:pos+3],pos-3):
305 |         if i == pos:
306 |             message.append(">>> {}".format(a))
307 |         else:
308 |             message.append("  > {}".format(a))
309 |     if pos+3 < len(argv)-1:
310 |             message.append(" ... {} args after".format(pos-3))
311 |     message = "\n".join(message)
312 |     return BadArg(message)
313 | 
314 | def parse_args(argspec, argv, environ):
315 |     options = []
316 |     flags = {}
317 |     args = {}
318 | 
319 |     for pos, arg in enumerate(argv):
320 |         if arg.startswith('--'):
321 |             if '=' in arg:
322 |                 key, value = arg[2:].split('=',1)
323 |             else:
324 |                 key, value = arg[2:], None
325 |             if key not in flags:
326 |                 flags[key] = []
327 |             flags[key].append((pos,value))
328 |         else:
329 |             options.append((pos,arg))
330 | 
331 |     for name in argspec.switches:
332 |         args[name] = False
333 |         if name not in flags:
334 |             continue
335 | 
336 |         values = flags.pop(name)
337 | 
338 |         if not values: 
339 |             raise parse_err("value given for switch flag {}".format(name), pos, argv)
340 |         if len(values) > 1:
341 |             raise parse_err("duplicate switch flag for: {}".format(name, ", ".join(repr(v) for v in values)), pos, argv)
342 | 
343 |         pos, value = values[0]
344 | 
345 |         if value is None:
346 |             args[name] = True
347 |         else:
348 |             args[name] = try_parse(name, value, "boolean", pos, argv)
349 | 
350 |     for name in argspec.flags:
351 |         args[name] = None
352 |         if name not in flags:
353 |             continue
354 | 
355 |         values = flags.pop(name)
356 |         if not values or values[0] is None:
357 |             raise parse_err("missing value for option flag {}".format(name), pos, argv)
358 |         if len(values) > 1:
359 |             raise parse_err("duplicate option flag for: {}".format(name, ", ".join(repr(v) for v in values)), pos, argv)
360 | 
361 |         pos, value = values[0]
362 |         args[name] = try_parse(name, value, argspec.argtypes.get(name), pos, argv)
363 | 
364 |     for name in argspec.lists:
365 |         args[name] = []
366 |         if name not in flags:
367 |             continue
368 | 
369 |         values = flags.pop(name)
370 |         if not values or None in values:
371 |             raise parse_err("missing value for list flag {}".format(name), pos, argv)
372 | 
373 |         for pos, value in values:
374 |             args[name].append(try_parse(name, value, argspec.argtypes.get(name), pos, argv))
375 | 
376 |     named_args = False
377 |     if flags:
378 |         for name in argspec.positional:
379 |             if name in flags:
380 |                 named_args = True
381 |                 break
382 |         for name in argspec.optional:
383 |             if name in flags:
384 |                 named_args = True
385 |                 break
386 |         if argspec.tail in flags:
387 |             named_args = True
388 |                 
389 |     if named_args:
390 |         for name in argspec.positional:
391 |             args[name] = None
392 |             if name not in flags:
393 |                 raise parse_err("missing named option: {}".format(name), pos, argv)
394 | 
395 |             values = flags.pop(name)
396 |             if not values or values[0] is None:
397 |                 raise parse_err("missing value for named option {}".format(name), pos, argv)
398 |             if len(values) > 1:
399 |                 raise parse_err("duplicate named option for: {}".format(name, ", ".join(repr(v) for v in values)), pos, argv)
400 | 
401 |             pos, value = values[0]
402 |             args[name] = try_parse(name, value, argspec.argtypes.get(name), pos, argv)
403 | 
404 |         for name in argspec.optional:
405 |             args[name] = None
406 |             if name not in flags:
407 |                 continue
408 | 
409 |             pos, values = flags.pop(name)
410 |             if not values or values[0] is None:
411 |                 raise parse_err("missing value for named option {}".format(name), pos, argv)
412 |             if len(values) > 1:
413 |                 raise parse_err("duplicate named option for: {}".format(name, ", ".join(repr(v) for v in values)), pos, argv)
414 | 
415 |             pos, value = values[0]
416 |             args[name] = try_parse(name, value, argspec.argtypes.get(name), pos, argv)
417 | 
418 |         name = argspec.tail
419 |         if name and name in flags:
420 |             args[name] = []
421 | 
422 |             pos, values = flags.pop(name)
423 |             if not values or None in values:
424 |                 raise parse_err("missing value for named option  {}".format(name), pos, argv)
425 | 
426 |             for pos, value in values:
427 |                 args[name].append(try_parse(name, value, argspec.argtypes.get(name), pos, argv))
428 |     else:
429 |         if flags:
430 |             raise parse_err("unknown option flags: --{}".format("".join(flags)), pos, argv)
431 | 
432 |         if argspec.positional:
433 |             for name in argspec.positional:
434 |                 if not options: 
435 |                     raise parse_err("missing option: {}".format(name), len(argv), argv)
436 |                 pos, value= options.pop(0)
437 | 
438 |                 args[name] = try_parse(name, value, argspec.argtypes.get(name), pos, argv)
439 | 
440 |         if argspec.optional:
441 |             for name in argspec.optional:
442 |                 if not options: 
443 |                     args[name] = None
444 |                 else:
445 |                     pos, value= options.pop(0)
446 |                     args[name] = try_parse(name, value, argspec.argtypes.get(name), pos, argv)
447 | 
448 |         if argspec.tail:
449 |             tail = []
450 |             name = argspec.tail
451 |             tailtype = argspec.argtypes.get(name)
452 |             while options:
453 |                 pos, value= options.pop(0)
454 |                 tail.append(try_parse(name, value, tailtype, pos, argv))
455 | 
456 |             args[name] = tail
457 | 
458 |     if options and named_args:
459 |         raise parse_err("unnamed options given {!r}".format(" ".join(arg for pos,arg in options)), pos, argv)
460 |     if options:
461 |         raise parse_err("unrecognised option: {!r}".format(" ".join(arg for pos,arg in options)), pos, argv)
462 |     return args
463 | 
464 | def try_parse(name, arg, argtype, pos, argv):
465 |     if argtype in (None, "str", "string"):
466 |         return arg
467 |     elif argtype in ("branch", "commit"):
468 |         return arg
469 |     elif argtype in ("path"):
470 |         return os.path.normpath(os.path.join(os.getcwd(), arg))
471 | 
472 |     elif argtype in ("int","integer"):
473 |         try:
474 |             i = int(arg)
475 |             if str(i) == arg: return i
476 |         except:
477 |             pass
478 |         raise parse_err('{} expects an integer, got {}'.format(name, arg), pos, argv)
479 | 
480 |     elif argtype in ("float","num", "number"):
481 |         try:
482 |             i = float(arg)
483 |             if str(i) == arg: return i
484 |         except:
485 |             pass
486 |         raise parse_err('{} expects an floating-point number, got {}'.format(name, arg), pos, argv)
487 |     elif argtype in ("bool", "boolean"):
488 |         if arg == "true":
489 |             return True
490 |         elif arg == "false":
491 |             return False
492 |         raise parse_err('{} expects either true or false, got {}'.format(name, arg), pos, argv)
493 |     elif argtype == "scalar":
494 |         try:
495 |             i = int(arg)
496 |             if str(i) == arg: return i
497 |         except:
498 |             pass
499 |         try:
500 |             f = float(arg)
501 |             if str(f) == arg: return f
502 |         except:
503 |             pass
504 |         return arg
505 |     else:
506 |         raise parse_err("Don't know how to parse option {}, of unknown type {}".format(name, argtype), pos, argv)
507 | 
508 | class CommandDescription:
509 |     def __init__(self, prefix, name, subcommands, subaliases, groups, short, long, argspec):
510 |         self.prefix = prefix
511 |         self.name = name
512 |         self.subcommands = subcommands
513 |         self.subaliases = subaliases
514 |         self.groups = groups
515 |         self.short, self.long = short, long
516 |         self.argspec = argspec
517 | 
518 |     def version(self):
519 |         return "<None>"
520 | 
521 | 
522 | class Action:
523 |     def __init__(self, mode, command, argv, errors=()):
524 |         self.mode = mode
525 |         self.path = command
526 |         self.argv = argv
527 |         self.errors = errors
528 | 
529 | class Error(Exception):
530 |     def __init__(self, exit_code, value) :
531 |         self.exit_code = exit_code
532 |         self.value = value
533 |         Exception.__init__(self)
534 | 
535 | class Group:
536 |     def __init__(self, name, command):
537 |         self.name = name
538 |         self.command = command
539 |         
540 |     def subcommand(self, name, short=None, long=None, aliases=()):
541 |         return self.command.subcommand(name, short=short, long=long, aliases=aliases, group=self.name)
542 | 
543 | class Command:
544 |     def __init__(self, name, short=None,*, long=None, aliases=(), prefixes=()):
545 |         self.name = name
546 |         self.prefix = [] 
547 |         self.subcommands = {}
548 |         self.groups = {None:[]}
549 |         self.subaliases = {}
550 |         self.run_fn = None
551 |         self.aliases=aliases
552 |         self.argspec = None
553 |         self.nargs = 0
554 |         self.call_fn = None
555 |         self.complete_fn= None
556 |         self.prefixes=prefixes
557 |         self.set_desc(short, long)
558 | 
559 |     def set_desc(self, short, long):
560 |         if long:
561 |             out = []
562 |             for para in long.strip().split('\n\n'):
563 |                 para = " ".join(x for x in para.split() if x)
564 |                 out.append(para)
565 |             long = "\n\n".join(out)
566 |         else:
567 |             long = None
568 |         if long and '\n' in long and short is None:
569 |             self.short, self.long = long.split('\n',1)
570 |             self.long = self.long.strip()
571 |         elif long and short is None:
572 |             self.long = long.strip()
573 |             self.short = self.long
574 |         else:
575 |             self.short = short
576 |             self.long = long
577 | 
578 |     def main(self, name):
579 |         if name == '__main__':
580 |             argv = sys.argv[1:]
581 |             environ = os.environ
582 |             code = main(self, argv, environ)
583 |             sys.exit(code)
584 | 
585 |     # -- builder methods
586 | 
587 |     def group(self, name):
588 |         self.groups[name] = []
589 |         return Group(name, self)
590 | 
591 |     def subcommand(self, name, short=None, long=None, aliases=(), group=None):
592 |         #if self.argspec:
593 |         #    raise Exception('bad')
594 |         if name in self.subaliases or name in self.subcommands:
595 |             raise BadDefinition('Duplicate {}'.format(name))
596 |         for a in aliases:
597 |             if a in self.subaliases or a in self.subcommands:
598 |                 raise BadDefinition('Duplicate {}'.format(a))
599 |         cmd = Command(name, short)
600 |         for a in aliases:
601 |             self.subaliases[a] = name
602 |         cmd.prefix.extend(self.prefix)
603 |         cmd.prefix.append(self.name)
604 |         self.subcommands[name] = cmd
605 |         self.groups[group].append(name)
606 |         return cmd
607 | 
608 |     def on_complete(self):
609 |         def _decorator(fn):
610 |             self.complete_fn = fn
611 |             return fn
612 |         return _decorator
613 | 
614 |     def on_call(self):
615 |         def _decorator(fn):
616 |             self.call_fn = fn
617 |             return fn
618 |         return _decorator
619 | 
620 | 
621 |     def on_run(self):
622 |         """A decorator for setting the function to be run"""
623 |         if self.run_fn:
624 |             raise BadDefinition('double definition')
625 | 
626 |         #if self.subcommands:
627 |         #    raise BadDefinition('bad')
628 | 
629 |         def decorator(fn):
630 |             self.run_fn = fn
631 |             if not self.long:
632 |                 self.set_desc(self.short, fn.__doc__)
633 | 
634 |             args = list(self.run_fn.__code__.co_varnames[:self.run_fn.__code__.co_argcount])
635 |             args = [a for a in args if not a.startswith('_')]
636 | 
637 |             if hasattr(fn, 'argspec'):
638 |                 self.nargs, self.argspec = fn.nargs, fn.argspec
639 |             else:
640 |                 self.nargs, self.argspec = parse_argspec(" ".join(args))
641 | 
642 |         
643 |             if self.nargs != len(args):
644 |                 raise BadDefinition('bad option definition')
645 | 
646 |             return fn
647 |         return decorator
648 | 
649 |     def handler(self, path):
650 |         handler = None
651 |         if path and path[0] in self.subcommands:
652 |             handler = self.subcommands[path[0]].handler(path[1:])
653 |         if not handler:
654 |             handler = self.call_fn
655 |         return handler
656 | 
657 |     # -- end of builder methods
658 | 
659 |     def bind(self, path, argv):
660 |         if path and path[0] in self.subcommands:
661 |             return self.subcommands[path[0]].bind(path[1:], argv)
662 |         elif self.run_fn:
663 |             if len(argv) == self.nargs:
664 |                 return lambda: self.run_fn(**argv)
665 |             else:
666 |                 raise Error(-1, "bad options")
667 |         else:
668 |             if len(argv) == 0:
669 |                 return lambda: (self.manual())
670 |             else:
671 |                 raise Error(-1, self.usage())
672 | 
673 |     def complete_path(self, route, path):
674 |         if path:
675 |             output = []
676 |             if len(path) > 0:
677 |                 path0 = path[0]
678 |                 if path0 in self.subaliases:
679 |                     path0 = self.subaliases[path0]
680 |                 if path0 in self.subcommands:
681 |                     output.extend(self.subcommands[path0].complete_path(route+[path[0]], path[1:]))
682 |                 if len(path) > 1:
683 |                     return output
684 |             prefix = ""
685 |             for name,cmd in self.subcommands.items():
686 |                 if not path[0] or name.startswith(path[0]):
687 |                     if name == path[0]: continue
688 |                     if cmd.subcommands and cmd.argspec:
689 |                         output.append("{}{}".format(prefix, name))
690 |                     elif cmd.subcommands and not cmd.argspec:
691 |                         output.append("{}{}:".format(prefix, name))
692 |                     else:
693 |                         output.append("{}{} ".format(prefix, name))
694 |             for name,cmd in self.subaliases.items():
695 |                 cmd = self.subcommands[cmd]
696 |                 if path[0] and name.startswith(path[0]):
697 |                     if cmd.subcommands and cmd.argspec:
698 |                         output.append("{}{}".format(prefix, name))
699 |                     elif cmd.subcommands and not cmd.argspec:
700 |                         output.append("{}{}:".format(prefix, name))
701 |                     else:
702 |                         output.append("{}{} ".format(prefix, name))
703 |             return output
704 |         elif route:
705 |             output = []
706 |             prefix = route[-1]
707 |             if self.subcommands:
708 |                 for name in self.groups[None]:
709 |                     output.append("{}:{}".format(prefix, name))
710 |                 if self.argspec:
711 |                     output.append("{} ".format(prefix))
712 |             else:
713 |                 output.append("{} ".format(prefix))
714 |             return output
715 |         return ()
716 | 
717 |     def parse_args(self, path, argv, environ, route):
718 |         if self.subcommands and path:
719 |             if path[0] in self.subaliases:
720 |                 path[0] = self.subaliases[path[0]]
721 | 
722 |             if path[0] in self.subcommands:
723 |                 return self.subcommands[path[0]].parse_args(path[1:], argv, environ, route+[path[0]])
724 |             else:
725 |                 if route:
726 |                     error="unknown subcommand {} for {}".format(path[0],":".join(route))
727 |                     return Action("error", route, {}, errors=(error,))
728 |                 return Action("error", route, {}, errors=("an unknown command: {}".format(path[0]),))
729 | 
730 |             # no argspec, print usage
731 |         elif not self.argspec:
732 |             if argv and argv[0]:
733 |                 if "--help" in argv:
734 |                     return Action("usage", route, {})
735 |                 return Action("error", route, {}, errors=("unknown option: {}".format(argv[0]),))
736 | 
737 |             return Action("help", route, {})
738 |         else:
739 |             if '--help' in argv:
740 |                 return Action("usage", route, {})
741 |             try:
742 |                 args = parse_args(self.argspec, argv, environ)
743 |                 return Action("call", route, args)
744 |             except BadArg as e:
745 |                 return e.action(route)
746 | 
747 |     def help(self, path, *, usage=False):
748 |         if path and path[0] in self.subcommands:
749 |             return self.subcommands[path[0]].help(path[1:], usage=usage)
750 |         else:
751 |             if usage:
752 |                 return self.usage()
753 |             return self.manual()
754 | 
755 |     def complete_arg(self, path, prefix, text):
756 |         if path: 
757 |             if path[0] in self.subaliases:
758 |                 path[0] = self.subaliases[path[0]]
759 |             if path[0] in self.subcommands:
760 |                 return self.subcommands[path[0]].complete_arg(path[1:], prefix, text)
761 |         else:
762 |             if text.startswith('--'):
763 |                 return self.complete_flag(text[2:])
764 |             elif text.startswith('-'):
765 |                 return self.complete_flag(text[1:])
766 |             elif self.argspec:
767 |                 n = len([p for p in prefix if p and not p.startswith('--')])
768 |                 field = None
769 |                 if n < len(self.argspec.positional):
770 |                     for i, name in enumerate(self.argspec.positional):
771 |                         if i == n:
772 |                             field = name
773 |                 else:
774 |                     n-=len(self.argspec.positional)
775 |                     if n < len(self.argspec.optional):
776 |                         for i, name in enumerate(self.argspec.optional):
777 |                             if i == n:
778 |                                 field = name
779 |                     elif self.argspec.tail:
780 |                         field = self.argspec.tail
781 |                 if not field:
782 |                     return ()
783 | 
784 |                 argtype = self.argspec.argtypes.get(field)
785 |                 return Complete(text, field, argtype)
786 |         return ()
787 | 
788 |     def complete_flag(self, prefix):
789 |         if '=' in prefix:
790 |             field, prefix = prefix.split('=', 1)
791 |             argtype = self.argspec.argtypes.get(field)
792 |             return Complete(prefix, field, argtype)
793 |         elif self.argspec:
794 |             out = []
795 |             out.extend("--{} ".format(x) for x in self.argspec.switches if x.startswith(prefix))
796 |             out.extend("--{}=".format(x) for x in self.argspec.flags if x.startswith(prefix))
797 |             out.extend("--{}=".format(x) for x in self.argspec.lists if x.startswith(prefix))
798 |             out.extend("--{}=".format(x) for x in self.argspec.positional if x.startswith(prefix))
799 |             out.extend("--{}=".format(x) for x in self.argspec.optional if x.startswith(prefix))
800 |             out.extend("--{}=".format(x) for x in (self.argspec.tail,) if x and x.startswith(prefix))
801 |             return out
802 |         else:
803 |             return ()
804 |             
805 |         
806 |     def manual(self):
807 |         output = []
808 |         full_name = list(self.prefix)
809 |         full_name.append(self.name)
810 |         full_name = "{}{}{}".format(full_name[0], (" " if full_name[1:] else ""), ":".join(full_name[1:]))
811 |         output.append("Name: {}{}{}".format(full_name, (" -- " if self.short else ""), self.short or ""))
812 | 
813 |         output.append("")
814 | 
815 |         output.append(self.usage(group=None))
816 |         output.append("")
817 | 
818 |         if self.argspec and self.argspec.descriptions:
819 |             output.append('Options:')
820 |             for name, desc in self.argspec.descriptions.items():
821 |                 output.append('  --{}\t{}'.format(name, desc))
822 |             output.append('')
823 | 
824 |         if self.long:
825 |             output.append('Description: {}'.format(self.long))
826 |             output.append("")
827 | 
828 |         if self.subcommands:
829 |             output.append("Commands:") 
830 |             for group, subcommands in self.groups.items():
831 |                 for name in subcommands:
832 |                     if name.startswith((" ", "_",)): continue
833 |                     cmd = self.subcommands[name]
834 |                     output.append("  {.name:10}  {}".format(cmd, cmd.short or ""))
835 |                 output.append("")
836 |         return "\n".join(output)
837 | 
838 |     def usage(self, group=None):
839 |         output = []
840 |         args = []
841 |         full_name = list(self.prefix)
842 |         full_name.append(self.name)
843 |         help_full_name = "{} [help]{}{}".format(full_name[0], (" " if full_name[1:] else ""), ":".join(full_name[1:]))
844 |         full_name = "{}{}{}".format(full_name[0], (" " if full_name[1:] else ""), ":".join(full_name[1:]))
845 |         if self.argspec:
846 |             if self.argspec.switches:
847 |                 args.extend("[--{0}]".format(o) for o in self.argspec.switches)
848 |             if self.argspec.flags:
849 |                 args.extend("[--{0}=<{0}>]".format(o) for o in self.argspec.flags)
850 |             if self.argspec.lists:
851 |                 args.extend("[--{0}=<{0}>...]".format(o) for o in self.argspec.lists)
852 |             if self.argspec.positional:
853 |                 args.extend("<{}>".format(o) for o in self.argspec.positional)
854 |             if self.argspec.optional:
855 |                 args.extend("[<{}>]".format(o) for o in self.argspec.optional)
856 |             if self.argspec.tail:
857 |                 args.append("[<{}>...]".format(self.argspec.tail))
858 | 
859 | 
860 |             output.append("Usage: {0} {1}".format(full_name, " ".join(args)))
861 |         subcommands = self.groups[group]
862 |         subcommands = "|".join(subcommands)
863 |         if group is None and len(self.groups) > 1:
864 |             subcommands += "|..."
865 |         if not self.prefix and subcommands:
866 |             output.append("Usage: {0} [help] <{1}> [--help]".format(self.name, subcommands))
867 |         elif subcommands:
868 |             output.append("Usage: {0}:<{1}> [--help]".format(help_full_name, subcommands))
869 |         return "\n".join(output)
870 | 
871 | 
872 | def main(root, argv, environ):
873 | 
874 |     if 'COMP_LINE' in environ and 'COMP_POINT' in environ:
875 |         arg, offset =  environ['COMP_LINE'], int(environ['COMP_POINT'])
876 |         prefix, arg = arg[:offset].rsplit(' ', 1)
877 |         tmp = prefix.lstrip().split(' ', 1)
878 |         if len(tmp) > 1:
879 |             path = tmp[1].split(' ')
880 |             if path[0] in root.prefixes or path[0] in ('help', 'debug'):
881 |                 if len(path) > 1:
882 |                     path = path[1].split(':') 
883 |                     result = root.complete_arg(path, path[2:], arg)
884 |                 else:
885 |                     result = root.complete_path([], arg.split(':'))
886 |             else:
887 |                 path0 = path[0].split(':')
888 |                 result = root.complete_arg(path0, path[1:], arg)
889 |         else:
890 |             result = root.complete_path([], arg.split(':'))
891 |         if isinstance(result, Complete):
892 |             result = root.complete_fn(result.prefix, result.name, result.argtype)
893 |         for line in result:
894 |             print(line)
895 |         return 0
896 | 
897 | 
898 |     if argv and argv[0] == "help":
899 |         argv.pop(0)
900 |         path = []
901 |         if argv and not argv[0].startswith('--'):
902 |             path = argv.pop(0).strip().split(':')
903 |         action = root.parse_args(path, argv, environ, [])
904 |         action = Action("help", action.path, {})
905 |     elif argv and (argv[0] in ('debug', 'help') or argv[0] in root.prefixes) and any(argv[1:]):
906 |         mode = argv.pop(0)
907 |         path = []
908 |         if argv and not argv[0].startswith('--'):
909 |             path = argv.pop(0).strip().split(':')
910 |         action = root.parse_args(path, argv, environ, [])
911 |         if action.path == []:
912 |             action = Action(action.mode, [mode], action.argv)
913 |         elif action.mode == "call":
914 |             action = Action(mode, action.path, action.argv)
915 |     elif argv and argv[0] == '--version':
916 |         action = Action("version", [], {})
917 |     elif argv and argv[0] == '--help':
918 |         action = Action("usage", [], {})
919 |     else:
920 |         path = []
921 |         if argv and not argv[0].startswith('--'):
922 |             path = argv.pop(0).strip().split(':')
923 |         action = root.parse_args(path, argv, environ, [])
924 | 
925 | 
926 |     try:
927 |         if action.mode == "error":
928 |             if action.path:
929 |                 print("Error: {} {}, {}".format(root.name, ":".join(action.path), ", ".join(action.errors)))
930 |             else:
931 |                 print("Error: {}, {}".format(root.name, ", ".join(action.errors)))
932 |             print(root.help(action.path, usage=True))
933 |             return -1
934 |         elif action.mode == "version":
935 |             result = root.version()
936 |             callback = lambda:result
937 |         elif action.mode == "usage":
938 |             result = root.help(action.path, usage=True)
939 |             callback = lambda:result
940 |         elif action.mode == "help":
941 |             result = root.help(action.path, usage=False)
942 |             callback = lambda:result
943 |         elif action.mode in ("call", "debug") or action.mode in root.prefixes:
944 |             callback =  root.bind(action.path, action.argv)
945 |         else:
946 |             raise Error('what')
947 | 
948 |         handler = root.handler(action.path)
949 |         if not handler:
950 |             try:
951 |                 result = callback()
952 |                 if isinstance(result, types.GeneratorType):
953 |                     for line in result:
954 |                         print(line)
955 |                 else:
956 |                     print(result)
957 |                 return 0
958 |             except Exception as e:
959 |                 if action.mode == "debug":
960 |                     raise
961 |                 result= "".join(traceback.format_exception(*sys.exc_info()))
962 |                 print(result)
963 |                 return -1
964 |         else:
965 |             return handler(action.mode, action.path, action.argv, callback)
966 |     except Error as e:
967 |         print()
968 |         print(e.value)
969 |         return e.exit_code
970 | 
971 | 
972 | def argspec(spec=None):
973 |     def decorator(fn):
974 |         nonlocal spec
975 |         args = list(fn.__code__.co_varnames[:fn.__code__.co_argcount])
976 |         args = [a for a in args if not a.startswith('_')]
977 |         
978 |         if spec is None:
979 |             nargs, spec = parse_argspec(" ".join(args))
980 |         else:
981 |             nargs, spec = parse_argspec(spec)
982 | 
983 |         if nargs != len(args):
984 |             raise BadDefinition('bad option definition')
985 | 
986 |         fn.nargs = nargs
987 |         fn.argspec = spec
988 | 
989 |         return fn
990 |     return decorator
991 | 
992 | 


--------------------------------------------------------------------------------
/vexlib/commands.py:
--------------------------------------------------------------------------------
   1 | #!/usr/bin/env python3
   2 | """a database for files and directories
   3 | 
   4 | vex is a command line program for saving changes to a project, switching
   5 | between different versions, and sharing those changes.
   6 | 
   7 | vex supports bash completion: run `complete -o nospace -C vex vex`
   8 | 
   9 | """
  10 | import os
  11 | import sys
  12 | import time
  13 | import types
  14 | import os.path
  15 | import traceback
  16 | import subprocess
  17 | import tempfile
  18 | 
  19 | from contextlib import contextmanager
  20 | 
  21 | from vexlib.cli import Command, argspec
  22 | from vexlib.project import Project
  23 | from vexlib.errors import VexBug, VexNoProject, VexNoHistory, VexUnclean, VexError, VexArgument, VexUnimplemented
  24 | 
  25 | DEFAULT_CONFIG_DIR = ".vex"
  26 | DEFAULT_INCLUDE = ["*"] 
  27 | DEFAULT_IGNORE =  [".*", DEFAULT_CONFIG_DIR, ".DS_Store", "*~", "*.swp", "__*__"]
  28 | 
  29 | fake = False
  30 | 
  31 | # CLI bits. Should handle environs, cwd, etc
  32 | vex_cmd = Command('vex', long=__doc__, prefixes=['fake', 'time', 'profile'])
  33 | vex_init = vex_cmd.subcommand('init')
  34 | 
  35 | vex_undo = vex_cmd.subcommand('undo')
  36 | vex_undo_list = vex_undo.subcommand('list')
  37 | vex_redo = vex_cmd.subcommand('redo')
  38 | vex_redo_list = vex_redo.subcommand('list')
  39 | 
  40 | vex_status = vex_cmd.subcommand('status')
  41 | vex_log = vex_cmd.subcommand('log', aliases=['changelog'])
  42 | vex_diff = vex_cmd.subcommand('diff')
  43 | vex_diff_file = vex_diff.subcommand('file')
  44 | 
  45 | vex_switch = vex_cmd.subcommand('switch')
  46 | 
  47 | vex_cmd_files = vex_cmd.group('files')
  48 | 
  49 | vex_add = vex_cmd_files.subcommand('add')
  50 | vex_forget = vex_cmd_files.subcommand('forget')
  51 | vex_remove = vex_cmd_files.subcommand('remove')
  52 | vex_restore = vex_cmd_files.subcommand('restore')
  53 | vex_missing = vex_cmd_files.subcommand('missing', aliases=['untracked'])
  54 | 
  55 | vex_cmd_commit = vex_cmd.group("commit")
  56 | 
  57 | vex_id = vex_cmd_commit.subcommand('id')
  58 | vex_commit = vex_cmd_commit.subcommand('commit')
  59 | vex_prepare = vex_commit.subcommand('prepare', aliases=['save'])
  60 | vex_commit_prepared = vex_commit.subcommand('prepared')
  61 | vex_amend = vex_commit.subcommand('amend')
  62 | vex_message = vex_cmd_commit.subcommand('message')
  63 | vex_message_edit = vex_message.subcommand('edit')
  64 | vex_message_get = vex_message.subcommand('get')
  65 | vex_message_amend = vex_message.subcommand('amend')
  66 | vex_message_filename = vex_message.subcommand('filename', aliases=['path'])
  67 | vex_message_set = vex_message.subcommand('set')
  68 | vex_template = vex_message.subcommand('template')
  69 | vex_template_edit = vex_template.subcommand('edit')
  70 | vex_template_get = vex_template.subcommand('get')
  71 | vex_template_filename = vex_template.subcommand('filename', aliases=['path'])
  72 | vex_template_set = vex_template.subcommand('set')
  73 | vex_commit_apply = vex_commit.subcommand('apply')
  74 | vex_commit_append = vex_commit.subcommand('append')
  75 | vex_commit_replay = vex_commit.subcommand('replay')
  76 | vex_commit_squash = vex_commit.subcommand('squash')
  77 | vex_rollback = vex_commit.subcommand('rollback')
  78 | vex_revert = vex_commit.subcommand('revert')
  79 | vex_rewind = vex_cmd_commit.subcommand('rewind')
  80 | vex_update = vex_cmd_commit.subcommand('update')
  81 | vex_cmd_branch = vex_cmd.group('branch')
  82 | 
  83 | vex_branch = vex_cmd_branch.subcommand('branch')
  84 | vex_branch_list = vex_branch.subcommand('list')
  85 | vex_branches = vex_cmd_branch.subcommand('branches')
  86 | vex_branch_get = vex_branch.subcommand('get', aliases=["show", "info"])
  87 | vex_branch_open = vex_branch.subcommand('open')
  88 | vex_branch_new = vex_branch.subcommand('new')
  89 | vex_branch_saveas = vex_branch.subcommand('saveas')
  90 | vex_branch_rename = vex_branch.subcommand('rename')
  91 | vex_branch_swap = vex_branch.subcommand('swap')
  92 | 
  93 | vex_diff_branch = vex_diff.subcommand('branch')
  94 | vex_branch_diff = vex_branch.subcommand('diff')
  95 | 
  96 | vex_session = vex_cmd_branch.subcommand('session')
  97 | vex_sessions = vex_cmd_branch.subcommand('sessions')
  98 | vex_ignore = vex_cmd_files.subcommand('ignore')
  99 | vex_ignore_add = vex_ignore.subcommand('add')
 100 | vex_include = vex_cmd_files.subcommand('include')
 101 | vex_include_add = vex_include.subcommand('add')
 102 | props_cmd = vex_cmd_files.subcommand('fileprops', aliases=['props', 'properties', 'property'])
 103 | props_list_cmd = props_cmd.subcommand('get')
 104 | props_set_cmd = props_cmd.subcommand('set')
 105 | 
 106 | vex_cmd_debug = vex_cmd.group('debug')
 107 | vex_debug = vex_cmd_debug.subcommand('debug', 'internal: run a command without capturing exceptions, or repairing errors')
 108 | 
 109 | vex_cmd_git = vex_cmd.group('git')
 110 | vex_git = vex_cmd_git.subcommand('git', short="* interact with a git repository")
 111 | 
 112 | def get_project():
 113 |     working_dir = os.getcwd()
 114 |     while True:
 115 |         config_dir = os.path.join(working_dir,  DEFAULT_CONFIG_DIR)
 116 |         if os.path.exists(config_dir):
 117 |             break
 118 |         new_working_dir = os.path.split(working_dir)[0]
 119 |         if new_working_dir == working_dir:
 120 |             return None
 121 |         working_dir = new_working_dir
 122 |     git = os.path.exists(os.path.join(config_dir, "git"))
 123 |     return Project(config_dir, working_dir, fake=fake, git=git)
 124 | 
 125 | def open_project(allow_empty=False):
 126 |     p = get_project()
 127 |     if not p:
 128 |         raise VexNoProject('no vex project found in {}'.format(os.getcwd()))
 129 |     if not allow_empty and p.history_isempty():
 130 |         raise VexNoHistory('Vex project exists, but `vex init` has not been run (or has been undone)')
 131 |     elif not p.clean_state():
 132 |         raise VexUnclean('Another change is already in progress. Try `vex debug:status`')
 133 |     return p
 134 |     
 135 | @vex_cmd.on_complete()
 136 | def Complete(prefix, field, argtype):
 137 |     out = []
 138 |     if argtype == 'path':
 139 |         if prefix:
 140 |             out.extend("{} ".format(p) for p in os.listdir() if p.startswith(prefix))
 141 |         else:
 142 |             out.extend("{} ".format(p) for p in os.listdir() if not p.startswith('.'))
 143 |     elif argtype in ('bool', 'boolean'):
 144 |         vals = ('true ','false ')
 145 |         if prefix:
 146 |             out.extend(p for p in vals if p.startswith(prefix))
 147 |         else:
 148 |             out.extend(vals)
 149 | 
 150 |     elif argtype == 'branch':
 151 |         p = open_project()
 152 |         if p:
 153 |             vals = p.list_branches()
 154 |             if prefix:
 155 |                 out.extend("{} ".format(name) for name, uuid in vals if name and name.startswith(prefix))
 156 |             else:
 157 |                 out.extend("{} ".format(name) for name, uuid in vals if name)
 158 | 
 159 |     return out
 160 | 
 161 | @vex_cmd.on_call()
 162 | def Call(mode, path, args, callback):
 163 |     """ calling vex foo:bar args, calls this function with 'call', ['foo', 'bar'], args, and
 164 |         a callback that is the right function to call
 165 |     """
 166 |     global fake # so sue me
 167 |     if mode == 'fake':
 168 |         fake = True
 169 |     def do(pager=True):
 170 |         try:
 171 |             result = callback()
 172 |             if pager and sys.stderr.isatty() and sys.stdout.isatty():
 173 |                 env = {}
 174 |                 env.update(os.environ)
 175 |                 env["LESS"] = "FRX"
 176 |                 env["LV"] = "-c"
 177 |                 p = subprocess.Popen('less', env=env, stdin=subprocess.PIPE, encoding='utf8')
 178 | 
 179 |                 if isinstance(result, types.GeneratorType):
 180 |                     for line in result:
 181 |                         if line is not None:
 182 |                             print(line, file=p.stdin)
 183 |                 elif result is not None:
 184 |                     print(result, file=p.stdin)
 185 |                 p.stdin.close()
 186 |                 while p.poll() is None:
 187 |                     try:
 188 |                         p.wait()
 189 |                     except KeyboardInterrupt:
 190 |                         pass
 191 |             elif isinstance(result, types.GeneratorType):
 192 |                 for line in result:
 193 |                     if line is not None:
 194 |                         print(line)
 195 |             elif result is not None:
 196 |                 print(result)
 197 |             return 0
 198 | 
 199 |         except Exception as e:
 200 |             if mode =="debug":
 201 |                 raise
 202 |             result= "".join(traceback.format_exception(*sys.exc_info()))
 203 |             message = str(e)
 204 |             if not message: message = e.__class__.__name__
 205 |             vex_error = isinstance(e, VexError)
 206 | 
 207 |         if path:
 208 |             print("Error: vex {}, {}".format(':'.join(path), message))
 209 |         else:
 210 |             print("Error: vex {}".format(message))
 211 | 
 212 |         if not vex_error:
 213 |             print("\nWorse still, it's an error vex doesn't recognize yet. A python traceback follows:\n")
 214 |             print(result)
 215 | 
 216 |         p = get_project()
 217 | 
 218 |         if p and p.exists() and not p.clean_state():
 219 |             with p.lock('rollback') as p:
 220 |                 p.rollback_new_action()
 221 | 
 222 |                 if not p.clean_state():
 223 |                     print('This is bad: The project history is corrupt, try `vex debug:status` for more information')
 224 |                 else:
 225 |                     print('Good news: The changes that were attempted have been undone')
 226 |         return -1
 227 |     if mode == 'time':
 228 |         now = time.monotonic()
 229 |         ret = do(pager=False)
 230 |         end = time.monotonic()
 231 |         print()
 232 |         print("{}".format(end-now))
 233 | 
 234 |         return ret
 235 |     elif mode == 'profile':
 236 |         import cProfile
 237 |         pr = cProfile.Profile()
 238 |         pr.enable()
 239 |         ret = do(pager=False)
 240 |         pr.disable()
 241 |         pr.print_stats(sort='time')
 242 |         return ret
 243 |     else:
 244 |         return do()
 245 | 
 246 | 
 247 | @contextmanager
 248 | def watcher():
 249 |     p = subprocess.run('fswatch --version', stdout=subprocess.DEVNULL, shell=True)
 250 |     if p.returncode: raise VexBug('fswatch is not installed')
 251 | 
 252 |     p = subprocess.Popen('fswatch .', shell=True, stdout=subprocess.PIPE)
 253 |     def watch_files():
 254 |         line = None
 255 |         while True:
 256 |             try:
 257 |                 line = p.stdout.readline()
 258 |                 if not line: break
 259 |                 yield line.decode('utf-8').rstrip()
 260 |             except KeyboardInterrupt:
 261 |                 break
 262 |     try:
 263 |         yield watch_files
 264 |     finally:
 265 |         p.terminate()
 266 | 
 267 | 
 268 | 
 269 | @vex_init.on_run()
 270 | @argspec('''
 271 |     --working:path    # Working directory, where files are edited/changed
 272 |     --config:path     # Normally /working_dir/.vex if not given 
 273 |     --prefix:path     # Subdirectory to check out of the repository, normally the working directory name
 274 |     --include:str... # files to include whe using vex add, can be passed multiple times 
 275 |     --ignore:str...  # files to ignore when using vex add, can be passed multiple times
 276 |     --git? # git backed
 277 |     [directory]  #
 278 | ''')
 279 | def Init(directory, working, config, prefix, include, ignore, git):
 280 |     """
 281 |         Create a new vex project in a given directory. 
 282 | 
 283 |         - If no directory given, it is assumed to be the current directory.
 284 |         - Inside that directory, a `.vex` directory is created to store the project history.
 285 |         - An initial empty commit is added.
 286 |         - The subtree checkout defaults to `/directory_name`.
 287 |         
 288 |         i.e a `vex init` in `/a/b` creates a `/a/b/.vex` directory, an empty commit, and checks
 289 |         out `/b` in the repo, into `/a/b` on the filesystem.`
 290 | 
 291 |         If you make a mistake, `vex undo` will undo the initial commit, but not remove
 292 |         the `.vex` directory. 
 293 | 
 294 |         `init` takes multiple `--include=<file>` and `--ignore=<file>` arguments, 
 295 |         defaulting to `--include='*' --ignore='.vex' --ignore='.*'`
 296 | 
 297 |         `--include`, `--ignore`, can be passed multiple times, and work the 
 298 |         same as `vex include 'pattern'` and `vex ignore 'pattern'`
 299 | 
 300 |     """
 301 | 
 302 |     working_dir = working or directory or os.getcwd()
 303 |     config_dir = config or os.path.join(working_dir,  DEFAULT_CONFIG_DIR)
 304 |     prefix = prefix or os.path.split(working_dir)[1] or ''
 305 |     prefix = os.path.join('/', prefix)
 306 | 
 307 |     include = include or DEFAULT_INCLUDE
 308 |     ignore = ignore or DEFAULT_IGNORE
 309 | 
 310 |     p = Project(config_dir, working_dir, fake, git=git)
 311 | 
 312 |     if p.exists() and not p.clean_state():
 313 |         yield ('This vex project is unwell. Try `vex debug:status`')
 314 |     elif p.exists():
 315 |         if not p.history_isempty():
 316 |             raise VexError("A vex project already exists here")
 317 |         else:
 318 |             yield ('A empty project was round, re-creating project in "{}"...'.format(os.path.relpath(config_dir)))
 319 |             with p.lock('init') as p:
 320 |                 p.init(prefix, include, ignore)
 321 |     else:
 322 |         p.makedirs()
 323 |         p.makelock()
 324 |         with p.lock('init') as p:
 325 |             yield ('Creating vex project in "{}"...'.format(working_dir))
 326 |             p.init(prefix, include, ignore)
 327 | 
 328 | 
 329 | @vex_undo.on_run()
 330 | def Undo():
 331 |     """
 332 |         Undo the last command.
 333 | 
 334 |         `vex undo` will return the project to how it was before the last command changed 
 335 |         things. running `vex undo:list` will show the list of commands that can be undone.
 336 | 
 337 |         for example:
 338 | 
 339 |         - `vex diff` / `vex status` / `vex log` and some other commands do not do anything
 340 |         and so cannot be undone.
 341 | 
 342 |         - calling `vex undo` after `vex commit` will not change the working copy, but will 
 343 |         remove the commit from the list of changes to the project
 344 | 
 345 |         - calling `vex undo` after calling `vex switch` to change which directory inside the
 346 |         repo to work on, will change the directory back. Edits to files are not undone.
 347 | 
 348 |         - calling `vex undo` after creating a branch with `vex new` will switch back
 349 |         to the old branch, but save the existing local changes incase `vex redo` is called.
 350 | 
 351 |         `vex undo:list` shows the list of commands that have been performed,
 352 |         and the order they will be undone in.
 353 | 
 354 |         similarly `vex redo:list` shows the actions that can be redone.
 355 |     """
 356 | 
 357 |     p = open_project()
 358 | 
 359 |     with p.lock('undo') as p:
 360 |         action = p.undo()
 361 |     if action:
 362 |         yield 'undid {}'.format(action.command)
 363 | 
 364 | @vex_undo_list.on_run()
 365 | def UndoList():
 366 |     """
 367 |         List the commands that can be undone.
 368 | 
 369 |         `vex undo` will return the project to how it was before the last command changed 
 370 |         things. running `vex undo:list` will show the list of commands that can be undone.
 371 | 
 372 |         `vex undo:list` shows the list of commands that have been performed,
 373 |         and the order they will be undone in.
 374 | 
 375 |         similarly `vex redo:list` shows the actions that can be redone.
 376 |     """
 377 | 
 378 |     p = open_project()
 379 | 
 380 |     count = 0
 381 |     for entry,redos in p.list_undos():
 382 |         count -= 1
 383 |         alternative = ""
 384 |         if len(redos) == 1:
 385 |             alternative = "(then ran but undid: {})".format(redos[0].command)
 386 |         elif len(redos) > 0:
 387 |             alternative = "(then ran but undid: {}, and {} )".format(",".join(r.command for r in redos[:-1]), redos[-1].command)
 388 | 
 389 |         yield "{}: {}, ran {}\t{}".format(count, entry.time, entry.command,alternative)
 390 |         yield ""
 391 | 
 392 | @vex_redo.on_run()
 393 | @argspec('''
 394 |     --choice:int # Which command to redo. `--choice=0` means the last action uandone.     
 395 | ''')
 396 | def Redo(choice):
 397 |     """
 398 |         Redo the last undone command.
 399 | 
 400 |         `vex redo` will redo the last action undone. `vex redo --list` will show the
 401 |         list of commands to choose from.
 402 | 
 403 |         `vex redo` is not the same as re-running the command, as `vex redo` will
 404 |         repeat the original changes made, without consulting the files in the working copy,
 405 |         or restoring the files if the command is something like `vex open`. 
 406 | 
 407 |         for example, redoing a `vex commit` will not commit the current versions of the files in the project
 408 |         
 409 |         redoing a `vex new <branch_name>` will reopen a branch, restoring the working copy
 410 |         with any local changes before `vex undo` was called.
 411 |         
 412 |         similarly, calling undo and redo on a `vex switch` operation, will just change which
 413 |         directory is checked out, saving and restoring local changes to files.
 414 | 
 415 |         if you do a different action after undo, you can still undo and redo.
 416 | 
 417 |         `vex redo:list` shows the actions that can be redone and `vex redo --choice=<n>` picks one.
 418 | 
 419 |         The order of the list changes when you pick a different item to redo. 
 420 |     """
 421 |     p = open_project(allow_empty=True)
 422 | 
 423 |     with p.lock('redo') as p:
 424 |         choices = p.list_redos()
 425 | 
 426 |         if choices:
 427 |             choice = choice or 0
 428 |             action = p.redo(choice)
 429 |             if action:
 430 |                 yield 'redid {}'.format(action.command)
 431 |         else:
 432 |             yield ('Nothing to redo')
 433 | 
 434 | 
 435 | @vex_redo_list.on_run()
 436 | def RedoList():
 437 |     """
 438 |         List the commands that can be redone.
 439 | 
 440 |         `vex redo` will redo the last action undone. `vex redo:list` will show the
 441 |         list of commands to choose from.
 442 |     """
 443 |     p = open_project(allow_empty=True)
 444 | 
 445 |     with p.lock('redo') as p:
 446 |         choices = p.list_redos()
 447 | 
 448 |         if choices:
 449 |             for n, choice in enumerate(choices):
 450 |                 yield "{}: {}, {}".format(n, choice.time, choice.command)
 451 |         else:
 452 |             yield ('Nothing to redo')
 453 | 
 454 | 
 455 | @vex_status.on_run()
 456 | @argspec('''
 457 |     --all?      # Show all files inside the repo, even ones outside working copy
 458 |     --missing?  # Show untracked files
 459 | ''')
 460 | def Status(all, missing):
 461 |     """
 462 |         Show the files and directories tracked by vex.
 463 | 
 464 |         `vex status` shows the state of each visible file, `vex status --all` shows the status 
 465 |         of every file in the current session/branch.
 466 |     """
 467 |     p = open_project()
 468 |     cwd = os.getcwd()
 469 |     with p.lock('status') as p:
 470 |         files = p.status()
 471 |         for reponame in sorted(files, key=lambda p:p.split(':')):
 472 |             entry = files[reponame]
 473 |             path = os.path.relpath(reponame, p.prefix())
 474 |             if entry.working is None:
 475 |                 if all:
 476 |                     yield "hidden:{:9}\t{} ".format(entry.state, path)
 477 |             elif reponame.startswith('/.vex/') or reponame == '/.vex':
 478 |                 if all:
 479 |                     yield "{}:{:8}\t{}".format('setting', entry.state, path)
 480 |             else:
 481 |                 if all or entry.state != 'tracked':
 482 |                     yield "{:16}\t{}{}".format(entry.state, path, ('*' if entry.stash else '') )
 483 |         yield ""
 484 |         if all or missing:
 485 |             for f in p.untracked(os.getcwd()):
 486 |                 path = os.path.relpath(f)
 487 |                 yield "{:16}\t{}".format('untracked', path)
 488 | 
 489 | 
 490 | @vex_log.on_run()
 491 | @argspec('''
 492 |         --all? # Show all changes
 493 | ''')
 494 | def Log(all):
 495 |     """
 496 |         List changes made to the project.
 497 | 
 498 |         `vex changelog` or `vex log` shows the list of commits inside a branch, using 
 499 |         the current branch if none given.
 500 |     """
 501 |     
 502 |     p = open_project()
 503 |     for entry in p.log(all=all):
 504 |         yield (entry)
 505 |         yield ""
 506 | 
 507 | 
 508 | @vex_diff.on_run()
 509 | @vex_diff_file.on_run()
 510 | @argspec('''
 511 |     [file:path...] # difference between two files
 512 | ''')
 513 | def Diff(file):
 514 |     """
 515 |         Show the changes, line by line, that have been made since the last commit.
 516 | 
 517 |         `vex diff` shows the changes waiting to be committed for the given files
 518 |     """
 519 |     p = open_project()
 520 |     with p.lock('diff') as p:
 521 |         cwd = os.getcwd()
 522 |         files = file if file else None # comes in as []
 523 |         for name, diff in  p.active_diff_files(files).items():
 524 |             yield diff
 525 | 
 526 | 
 527 | @vex_add.on_run()
 528 | @argspec('''
 529 |     --include:str... # files to include whe using vex add, can be passed multiple times 
 530 |     --ignore:str...  # files to ignore when using vex add, can be passed multiple times
 531 |     [file:path...]     # filename or directory to add
 532 | ''')
 533 | def Add(include, ignore, file):
 534 |     """
 535 |         Add files to the project.
 536 | 
 537 |         `vex add` will add all files given to the project, and recurse through
 538 |         subdirectories too.
 539 | 
 540 |         it uses the settings in `vex ignore` and `vex include`
 541 | 
 542 |     """
 543 |     cwd = os.getcwd()
 544 |     if not file:
 545 |         files = [cwd]
 546 |     else:
 547 |         files = file
 548 |     missing = [f for f in file if not os.path.exists(f)]
 549 |     if missing:
 550 |         raise VexArgument('cannot find {}'.format(",".join(missing)))
 551 |     p = open_project()
 552 |     include = include if include else None
 553 |     ignore = ignore if ignore else None
 554 |     with p.lock('add') as p:
 555 |         for f in p.add(files, include=include, ignore=ignore):
 556 |             f = os.path.relpath(f)
 557 |             yield "add: {}".format(f)
 558 | 
 559 | @vex_forget.on_run()
 560 | @argspec('''
 561 |         [file:path...] # Files to remove from next commit
 562 | ''')
 563 | def Forget(file):
 564 |     """
 565 |         Remove files from the project, without deleting them.
 566 | 
 567 |         `vex forget` will instruct vex to stop tracking a file, and it will not appear
 568 |         inside the next commit.
 569 | 
 570 |         it does not delete the file from the working copy.
 571 |     """
 572 |     if not file:
 573 |         return
 574 | 
 575 |     file = [f for f in file if os.path.exists(f)]
 576 |     p = open_project()
 577 |     with p.lock('forget') as p:
 578 |         for f in p.forget(file).values():
 579 |             f = os.path.relpath(f)
 580 |             yield "forget: {}".format(f)
 581 | 
 582 | @vex_remove.on_run()
 583 | @argspec('''
 584 |         [file:path...] # Files to remove from working copy
 585 | ''')
 586 | def Remove(file):
 587 |     """ 
 588 |         Remove files from the project, deleting them.
 589 | 
 590 |         `vex remove` will instruct vex to stop tracking a file, and it will not appear
 591 |         inside the next commit.
 592 | 
 593 |         it will delete the file from the working copy.
 594 |     """
 595 |     if not file:
 596 |         return
 597 | 
 598 |     file= [f for f in file if os.path.exists(f)]
 599 |     p = open_project()
 600 |     with p.lock('remove') as p:
 601 |         for f in p.remove(file).values():
 602 |             f = os.path.relpath(f)
 603 |             yield "remove: {}".format(f)
 604 | 
 605 | @vex_restore.on_run()
 606 | @argspec('''
 607 |         [file:path...] # Files to restore to working copy
 608 | ''')
 609 | def Restore(file):
 610 |     """
 611 |         Restore files from the project, overwriting uncommited modifications 
 612 | 
 613 |         `vex restore` will change a file back to how it was .
 614 |     """
 615 |     if not file:
 616 |         return
 617 | 
 618 |     p = open_project()
 619 |     with p.lock('restore') as p:
 620 |         for f in p.restore(file).values():
 621 |             f = os.path.relpath(f)
 622 |             yield "restore: {}".format(f)
 623 | 
 624 | @vex_missing.on_run()
 625 | @argspec()
 626 | def Missing():
 627 |     """
 628 |         List files that are untracked within the working copy.
 629 | 
 630 |         `vex missing` shows missing files
 631 |     """
 632 |     p = open_project()
 633 |     for f in p.untracked(os.getcwd()):
 634 |             f = os.path.relpath(f)
 635 |             yield "missing: {}".format(f)
 636 | 
 637 | @vex_id.on_run()
 638 | def Id():
 639 |     """
 640 |         Show last commit identifier
 641 | 
 642 |         `vex id` will print the commit-id
 643 |     """
 644 | 
 645 |     p = open_project()
 646 |     return p.active().prepare
 647 | 
 648 | @vex_commit.on_run()
 649 | @argspec('''
 650 |     --add?          # Run `vex add` before commiting
 651 |     --message:str       # Set commit message
 652 |     [file:path...]  # Commit only a few changed files
 653 | ''')
 654 | def Commit(add, message, file):
 655 |     """
 656 |         Save the working copy, adding a new entry into the list of project changes.
 657 | 
 658 |         `vex commit` saves the current state of the project.
 659 | 
 660 |     """
 661 |     p = open_project()
 662 |     with p.lock('commit') as p:
 663 |         if add:
 664 |             for f in p.add([os.getcwd()]):
 665 |                 f = os.path.relpath(f)
 666 |                 yield "add: {}".format(f)
 667 | 
 668 |         cwd = os.getcwd()
 669 |         files = file if file else None
 670 | 
 671 |         if message is not None:
 672 |             p.state.set('message', message)
 673 | 
 674 |         changes = p.commit_active(files)
 675 | 
 676 |         if changes:
 677 |             for name, entries in changes.items():
 678 |                 entries = [entry.text for entry in entries]
 679 |                 name = os.path.relpath(name, p.prefix())
 680 |                 yield "commit: {}, {}".format(', '.join(entries), name)
 681 | 
 682 |         else:
 683 |             yield 'commit: Nothing to commit'
 684 | 
 685 | @vex_prepare.on_run()
 686 | @argspec('''
 687 |         --add?          # Run `vex add` before commiting
 688 |         --watch?         # Unsupported
 689 |         [file:path...] # Files to add to the commt 
 690 | ''')
 691 | def Prepare(file,watch, add):
 692 |     """
 693 |         Save current working copy, ready for commiting.
 694 | 
 695 |         `vex prepare` is like `vex commit`, except that the next commit will inherit all of the 
 696 |         changes made.
 697 | 
 698 |         preparory commits are not applied to branches.
 699 |     """
 700 |     p = open_project()
 701 |     yield ('Preparing')
 702 |     with p.lock('prepare') as p:
 703 |         if add:
 704 |             for f in p.add([os.getcwd()]):
 705 |                 f = os.path.relpath(f)
 706 |                 yield "add: {}".format(f)
 707 | 
 708 |         cwd = os.getcwd()
 709 |         files = file if file else None
 710 |         if watch:
 711 |             active = p.active()
 712 |             prefix = p.prefix()
 713 |             with watcher() as files:
 714 |                 for file in files():
 715 |                     if p.check_file(file):
 716 |                         repo = p.full_to_repo_path(prefix, file)
 717 |                         if repo in active.files:
 718 |                             p.prepare([file])
 719 |                             yield os.path.relpath(file)
 720 |         else:
 721 |             p.prepare(files)
 722 | 
 723 | @vex_commit_prepared.on_run()
 724 | def CommitPrepared():
 725 |     """
 726 |         Save any prepared changes as a new entry in the list of changes.
 727 | 
 728 |         `vex commit:prepared` transforms earlier `vex prepare` into a commit
 729 | 
 730 |     """
 731 |     p = open_project()
 732 |     with p.lock('commit:prepared') as p:
 733 |         changes = p.commit_prepared()
 734 |         if changes:
 735 |             for name, entries in changes.items():
 736 |                 entries = [entry.text for entry in entries]
 737 |                 name = os.path.relpath(name, p.prefix())
 738 |                 yield "commit: {}, {}".format(', '.join(entries), name)
 739 |         else:
 740 |             yield 'commit: Nothing to commit'
 741 | 
 742 | 
 743 | @vex_amend.on_run()
 744 | @argspec('''
 745 |         [file:path...] # files to change
 746 | ''')
 747 | def Amend(file):
 748 |     """
 749 |         Replace the last commit with the current changes in the project
 750 | 
 751 |         `vex amend` allows you to re-commit, indicating that the last commit
 752 |         was incomplete.
 753 | 
 754 |         `vex amend` is like `vex prepare`, except that it operates on the last commit, 
 755 |         instead of preparing for the next.
 756 | 
 757 |     """
 758 |     p = open_project()
 759 |     yield ('Amending')
 760 |     with p.lock('amend') as p:
 761 |         cwd = os.getcwd()
 762 |         files = file if file else None
 763 |         if p.amend(files):
 764 |             yield 'Committed'
 765 |         else:
 766 |             yield 'Nothing to commit'
 767 |         # check that session() and branch()
 768 | 
 769 | @vex_message.on_run()
 770 | @vex_message_edit.on_run()
 771 | @argspec('--editor [message]')
 772 | def EditMessage(editor, message):
 773 |     """
 774 |         Edit the commit message (No Undo/Redo)
 775 | 
 776 |         Use `--editor` to open the file in an editor
 777 |     """
 778 |     p = open_project()
 779 |     if message:
 780 |         with p.lock('message:set') as p:
 781 |             p.state.set('message', message)
 782 |             return "set"
 783 | 
 784 |     with p.lock('editor') as p:
 785 |         if not editor and p.state.exists('editor'):
 786 |             editor = p.state.get('editor')
 787 |         if not editor:
 788 |             editor = os.environ.get('EDITOR')
 789 |         if not editor:
 790 |             editor = os.environ.get('VISUAL')
 791 |         file = p.state.filename('message')
 792 |         if not editor:
 793 |             path = os.path.relpath(file)
 794 |             raise VexArgument('with what editor?, you can open ./{} directly too'.format(path))
 795 |         p.state.set('editor', editor)
 796 |     os.execvp(editor, [editor, file])
 797 | 
 798 | @vex_message_amend.on_run()
 799 | @argspec('--editor')
 800 | def AmendMessage(editor):
 801 |     """
 802 |         Amend the commit message (No undo/redo)
 803 | 
 804 |         Use `--editor` to open the file in an editor
 805 |     """
 806 |     p = open_project()
 807 | 
 808 |     with p.lock('message:amend') as p:
 809 |         if not editor and p.state.exists('editor'):
 810 |             editor = p.state.get('editor')
 811 |         if not editor:
 812 |             editor = os.environ.get('EDITOR')
 813 |         if not editor:
 814 |             editor = os.environ.get('VISUAL')
 815 |         file = p.state.filename('message')
 816 |         if not editor:
 817 |             path = os.path.relpath(file)
 818 |             raise VexArgument('with what editor?, you can open ./{} directly too'.format(path))
 819 |         p.state.set('editor', editor)
 820 | 
 821 |         old_message = p.state.get('message')
 822 |         template = p.settings.get('template')
 823 |         if old_message and old_message != template:
 824 |             raise VexArgument("would overwrite changes.... and I can't undo that. Clear message first")
 825 | 
 826 |         commit = p.get_commit(p.active().prepare)
 827 |         changeset = p.get_manifest(commit.changeset)
 828 |         message = changeset.message
 829 |         p.state.set('message', message)
 830 |     os.execvp(editor, [editor, file])
 831 | 
 832 | 
 833 | @vex_message_get.on_run()
 834 | def GetMessage():
 835 |     """ 
 836 |         Show the commit message
 837 |     """
 838 |     p = open_project()
 839 |     if p.state.exists('message'):
 840 |         return p.state.get('message')
 841 | 
 842 | @vex_message_filename.on_run()
 843 | def MessageFilename():
 844 |     """
 845 |         Show where commit message is stored
 846 |     """
 847 |     p = open_project()
 848 |     return p.state.filename('message')
 849 | 
 850 | @vex_message_set.on_run()
 851 | @argspec('message')
 852 | def SetMessage(message):
 853 |     """ Set commit message (No Undo/Redo)"""
 854 |     p = open_project()
 855 |     with p.lock('message:set') as p:
 856 |         p.state.set('message', message)
 857 |         yield "set"
 858 | 
 859 | @vex_template.on_run()
 860 | @vex_template_edit.on_run()
 861 | @argspec('--editor')
 862 | def EditTemplate(editor):
 863 |     """ Edit commit template (No Undo/Redo)"""
 864 |     p = open_project()
 865 |     with p.lock('editor') as p:
 866 |         if not editor and p.state.exists('editor'):
 867 |             editor = p.state.get('editor')
 868 |         if not editor:
 869 |             editor = os.environ.get('EDITOR')
 870 |         if not editor:
 871 |             editor = os.environ.get('VISUAL')
 872 |         file = p.settings.filename('template')
 873 |         if not editor:
 874 |             path = os.path.relpath(file)
 875 |             raise VexArgument('with what editor?, you can open ./{} directly too'.format(path))
 876 |         p.state.set('editor', editor)
 877 |     os.execvp(editor, [editor, file])
 878 | 
 879 | 
 880 | @vex_template_get.on_run()
 881 | def GetTemplate():
 882 |     """Show commit template"""
 883 |     p = open_project()
 884 |     if p.settings.exists('template'):
 885 |         return p.settings.get('template')
 886 | 
 887 | @vex_template_filename.on_run()
 888 | def TemplateFilename():
 889 |     """Show where the commit template is stored"""
 890 |     p = open_project()
 891 |     return p.settings.filename('template')
 892 | 
 893 | @vex_template_set.on_run()
 894 | @argspec('template')
 895 | def SetTemplate(template):
 896 |     """ Set the commit template (No Undo/Redo)"""
 897 |     p = open_project()
 898 |     with p.lock('template:set') as p:
 899 |         p.settings.set('template', template)
 900 |         return "set"
 901 | 
 902 | @vex_commit_apply.on_run()
 903 | @argspec('branch:branch')
 904 | def Apply(branch):
 905 |     """ Apply changes from another branch to the current session """
 906 |     p = open_project()
 907 |     with p.lock('apply') as p:
 908 |         if not p.names.exists(branch):
 909 |             raise VexArgument('{} doesn\'t exist'.format(branch))
 910 |         p.apply_changes_from_branch(branch)
 911 | 
 912 | @vex_commit_append.on_run()
 913 | @argspec('branch:branch')
 914 | def Append(branch):
 915 |     """ Append changes from another branch to the current session """
 916 |     p = open_project()
 917 |     with p.lock('append') as p:
 918 |         if not p.names.exists(branch):
 919 |             raise VexArgument('{} doesn\'t exist'.format(branch))
 920 |         changes = p.append_changes_from_branch(branch)
 921 | 
 922 |         if changes:
 923 |             for name, entries in changes.items():
 924 |                 entries = [entry.text for entry in entries]
 925 |                 name = os.path.relpath(name, p.prefix())
 926 |                 yield "append: {}, {}".format(', '.join(entries), name)
 927 | 
 928 |         else:
 929 |             yield 'append: nothing to commit'
 930 | 
 931 | @vex_commit_replay.on_run()
 932 | @argspec('branch:branch')
 933 | def Replay(branch):
 934 |     p = open_project()
 935 |     with p.lock('replay') as p:
 936 |         if not p.names.exists(branch):
 937 |             raise VexArgument('{} doesn\'t exist'.format(branch))
 938 |         p.replay_changes_from_branch(branch)
 939 | 
 940 | @vex_commit_squash.on_run()
 941 | def Squash():
 942 |     raise VexUnimplemented()
 943 | 
 944 | @vex_rollback.on_run()
 945 | def Rollback():
 946 |     raise VexUnimplemented()
 947 | 
 948 | @vex_revert.on_run()
 949 | def Revert():
 950 |     raise VexUnimplemented()
 951 | 
 952 | @vex_rewind.on_run()
 953 | def Rewind():
 954 |     """ Open up an old version of the project """
 955 |     raise VexUnimplemented()
 956 | 
 957 | @vex_update.on_run()
 958 | def Update():
 959 |     """ Update the current branch with changes from the original """
 960 |     raise VexUnimplemented()
 961 | 
 962 | # Rollback, Revert, Squash, Update,
 963 | 
 964 | @vex_branch.on_run()
 965 | @argspec('[name:branch]')
 966 | def Branch(name):
 967 |     """
 968 |         Open an old branch or create a new branch
 969 | 
 970 |     """
 971 |     p = open_project()
 972 |     with p.lock('open') as p:
 973 |         if name:
 974 |             p.open_branch(name, create=True)
 975 |         active = p.active()
 976 |         branch = p.branches.get(active.branch)
 977 |         yield branch.name
 978 | 
 979 | @vex_branch_list.on_run()
 980 | @vex_branches.on_run()
 981 | def Branches():
 982 |     """ List all active branches in project """
 983 |     p = open_project()
 984 |     with p.lock('branches') as p:
 985 |         branches = p.list_branches()
 986 |         active = p.active()
 987 |         for (name, branch) in branches:
 988 |             if branch.uuid == active.branch:
 989 |                 if name:
 990 |                     yield "{} *".format(name)
 991 |                 else:
 992 |                     yield "{} *".format(branch.uuid)
 993 |             elif name:
 994 |                 yield name
 995 |             else:
 996 |                 yield branch.uuid
 997 | 
 998 | @vex_branch_get.on_run()
 999 | @argspec('[name:branch]')
1000 | def BranchInfo(name):
1001 |     """
1002 |         show current branch name
1003 |     """
1004 |     p = open_project()
1005 |     with p.lock('branch') as p:
1006 |         if not name:
1007 |             active = p.active()
1008 |             branch = p.branches.get(active.branch)
1009 |         else:
1010 |             b = p.names.get(name)
1011 |             if b:
1012 |                 branch = p.branches.get(b)
1013 |             else:
1014 |                 raise VexArgument("{} isn't a branch".format(name)) 
1015 |         # session is ahead (in prepared? in commits?)
1016 |         # session has detached ...?
1017 |         # 
1018 |         return branch.name
1019 | 
1020 | @vex_branch_open.on_run()
1021 | @argspec('name:branch')
1022 | def OpenBranch(name):
1023 |     """
1024 |         Open a branch
1025 |     """
1026 |     p = open_project()
1027 |     with p.lock('open') as p:
1028 |         p.open_branch(name, create=False)
1029 | 
1030 | @vex_branch_new.on_run()
1031 | @argspec('name:branch')
1032 | def NewBranch(name):
1033 |     """
1034 |         Create new branch
1035 |     """
1036 |     p = open_project()
1037 |     with p.lock('new') as p:
1038 |         if p.names.exists(name):
1039 |             raise VexArgument('{} exists'.format(name))
1040 |         p.new_branch(name)
1041 | 
1042 | @vex_branch_saveas.on_run()
1043 | @argspec('name:branch')
1044 | def SaveAsBranch(name):
1045 |     """
1046 |         Save working copy as a different branch
1047 |     """
1048 |     p = open_project()
1049 |     with p.lock('saveas') as p:
1050 |         if p.names.get(name):
1051 |             raise VexArgument('{} exists'.format(name))
1052 |         p.save_as(name)
1053 |         return name
1054 | 
1055 | @vex_branch_rename.on_run()
1056 | @argspec('name:branch')
1057 | def RenameBranch(name):
1058 |     """
1059 |         Rename the current branch
1060 |     """
1061 |     p = open_project()
1062 |     with p.lock('rename') as p:
1063 |         if p.names.get(name):
1064 |             raise VexArgument('{} exists'.format(name))
1065 |         p.rename_branch(name)
1066 | 
1067 | @vex_branch_swap.on_run()
1068 | @argspec('name:branch')
1069 | def SwapBranch(name):
1070 |     """
1071 |         Rename the current branch, swapping names with another
1072 |     """
1073 |     p = open_project()
1074 |     with p.lock('swap') as p:
1075 |         if not p.names.get(name):
1076 |             raise VexArgument("{} doesn't exist".format(name))
1077 |         p.swap_branch(name)
1078 | 
1079 | 
1080 | @vex_diff_branch.on_run()
1081 | @vex_branch_diff.on_run()
1082 | @argspec('''
1083 |         [branch:branch] # name of branch to check, defaults to current branch
1084 | ''')
1085 | def DiffBranch(branch):
1086 |     """
1087 |         `vex diff:branch` shows the changes bewtween working copy and a branch
1088 |     """
1089 |     p = open_project()
1090 |     with p.lock('diff') as p:
1091 |         if not branch:
1092 |             branch = p.active().branch
1093 |             branch = p.get_branch(branch)
1094 |             commit = branch.base
1095 |         else:
1096 |             branch = p.get_branch_uuid(branch)
1097 |             branch = p.get_branch(branch)
1098 |             commit = branch.head
1099 | 
1100 | 
1101 |         for name, diff in  p.active_diff_commit(commit).items():
1102 |             yield diff
1103 | 
1104 | @vex_switch.on_run()
1105 | @argspec('[prefix]')
1106 | def Switch(prefix):
1107 |     """
1108 |         Change which directory of the project is mapped to the working copy
1109 | 
1110 |         A project inside `/code/my_project` will normally have a "/my_project" directory inside the repo,
1111 |         mapped to the working copy.
1112 | 
1113 |     """
1114 |     p = open_project()
1115 |     if os.getcwd() != p.working_dir:
1116 |         raise VexArgument("it's best if you don't call this while in a subdirectory")
1117 |     if prefix:
1118 |         with p.lock('switch') as p:
1119 |             prefix = os.path.normpath(os.path.join(p.prefix(), prefix))
1120 |             p.switch(prefix)
1121 |     else:
1122 |         return p.prefix()
1123 | 
1124 | @vex_session.on_run()
1125 | def Session():
1126 |     """ Show active session for current branch"""
1127 |     p = open_project()
1128 |     with p.lock('sessions') as p:
1129 |         sessions = p.list_sessions()
1130 |         active = p.active()
1131 |     return active.uuid
1132 | 
1133 | @vex_sessions.on_run()
1134 | def Sessions():
1135 |     """ Show open sessions for current branch"""
1136 |     p = open_project()
1137 |     with p.lock('sessions') as p:
1138 |         sessions = p.list_sessions()
1139 |         active = p.active()
1140 |         for s in sessions:
1141 |             if s.uuid == active.uuid:
1142 |                 yield "{} *".format(s.uuid)
1143 |             else:
1144 |                 yield s.uuid
1145 | 
1146 | # XXX: vex session:open session:new session:attach session:detach session:remove
1147 | 
1148 | @vex_ignore.on_run()
1149 | @vex_ignore_add.on_run()
1150 | @argspec('[file...]')
1151 | def AddIgnore(file):
1152 |     """Add a pattern to the list of ignored files"""
1153 |     p = open_project()
1154 |     if file:
1155 |         with p.lock('ignore:add') as p:
1156 |             old = p.settings.get('ignore')
1157 |             old.extend(file)
1158 |             p.settings.set('ignore', old)
1159 |     else:
1160 |         for entry in p.settings.get('ignore'):
1161 |             yield entry
1162 | 
1163 | 
1164 | @vex_include.on_run()
1165 | @vex_include_add.on_run()
1166 | @argspec('[file...]')
1167 | def AddInclude(file):
1168 |     """ Add pattern to list for what is included by vex add automatically """
1169 |     p = open_project()
1170 |     if file:
1171 |         with p.lock('include:add') as p:
1172 |             old = p.settings.get('include')
1173 |             old.extend(file)
1174 |             p.settings.set('include', old)
1175 |     else:
1176 |         for entry in p.settings.get('include'):
1177 |             yield entry
1178 | 
1179 | 
1180 | @props_cmd.on_run()
1181 | @props_list_cmd.on_run()
1182 | @argspec('file')
1183 | def ListProps(file):
1184 |     """ List properties of file """
1185 |     p = open_project()
1186 |     with p.lock('fileprops:list') as p:
1187 |         for key,value in p.get_fileprops(file).items():
1188 |             file = os.path.relpath(file)
1189 |             yield "{}:{}:{}".format(file, key,value)
1190 | 
1191 | @props_set_cmd.on_run()
1192 | @argspec('file name value:scalar')
1193 | def SetProp(file, name, value):
1194 |     """Set file property"""
1195 |     p = open_project()
1196 |     with p.lock('fileprops:list') as p:
1197 |         p.set_fileprop(filename, name, value)
1198 | 
1199 | # Debug
1200 | 
1201 | debug_status = vex_debug.subcommand('status')
1202 | debug_restart = vex_debug.subcommand('restart')
1203 | debug_rollback = vex_debug.subcommand('rollback')
1204 | debug_test = vex_debug.subcommand('test', short="self test")
1205 | debug_soak = vex_debug.subcommand('soak', short="soak test")
1206 | debug_argparse = vex_debug.subcommand('args')
1207 | 
1208 | 
1209 | @vex_debug.on_run()
1210 | def Debug():
1211 |     """
1212 |     `vex debug commit` calls `vex commit`, but will always print a full traceback
1213 |     and never attempt to recover from incomplete changes.
1214 | 
1215 |     use with care.
1216 |     """
1217 |     yield ('Use vex debug <cmd> to run <cmd>, or use `vex debug:status`')
1218 | 
1219 | 
1220 | @debug_status.on_run()
1221 | def DebugStatus():
1222 |     p = open_project(check=False)
1223 |     with p.lock('debug:status') as p:
1224 |         yield ("Clean history", p.clean_state())
1225 |         head = p.active()
1226 |         out = []
1227 |         if head:
1228 | 
1229 |             out.append("head: {}".format(head.uuid))
1230 |             out.append("at {}, started at {}".format(head.prepare, head.commit))
1231 | 
1232 |             branch = p.branches.get(head.branch)
1233 |             out.append("commiting to branch {}".format(branch.uuid))
1234 | 
1235 |             commit = p.get_commit(head.prepare)
1236 |             out.append("last commit: {}".format(commit.__class__.__name__))
1237 |         else:
1238 |             if p.history_isempty():
1239 |                 out.append("you undid the creation. try vex redo")
1240 |             else:
1241 |                 out.append("no active head, but history, weird")
1242 |         out.append("")
1243 |         return "\n".join(out)
1244 | 
1245 | 
1246 | @debug_restart.on_run()
1247 | def DebugRestart():
1248 |     p = get_project()
1249 |     with p.lock('debug:restart') as p:
1250 |         if p.clean_state():
1251 |             yield ('There is no change in progress to restart')
1252 |             return
1253 |         yield ('Restarting current action...')
1254 |         p.restart_new_action()
1255 |         if p.clean_state():
1256 |             yield ('Project has recovered')
1257 |         else:
1258 |             yield ('Oh dear')
1259 | 
1260 | @debug_rollback.on_run()
1261 | def DebugRollback():
1262 |     p = get_project()
1263 |     with p.lock('debug:rollback') as p:
1264 |         if p.clean_state():
1265 |             yield ('There is no change in progress to rollback')
1266 |             return
1267 |         yield ('Rolling back current action...')
1268 |         p.rollback_new_action()
1269 |         if p.clean_state():
1270 |             yield ('Project has recovered')
1271 |         else:
1272 |             yield ('Oh dear')
1273 | 
1274 | 
1275 | class Vex:
1276 |     def __init__(self, path, dir, command=()):
1277 |         self.path = path
1278 |         self.dir=dir
1279 |         self.command = command
1280 | 
1281 |     def __getattr__(self, name):
1282 |         return self.__class__(self.path, self.dir, command=self.command+(name,))
1283 | 
1284 |     def __call__(self, *args, **kwargs):
1285 |         cmd = [sys.executable]
1286 |         cmd.append(self.path)
1287 |         if self.command:
1288 |             cmd.append(":".join(self.command))
1289 |         for name, value in kwargs.items():
1290 |             if value is True:
1291 |                 cmd.append("--{}=true".format(name))
1292 |             elif value is False:
1293 |                 cmd.append("--{}=false".format(name))
1294 |             elif isinstance(value, (list, tuple)):
1295 |                 for v in value:
1296 |                     cmd.append("--{}={}".format(name, v))
1297 |             else:
1298 |                 cmd.append("--{}={}".format(name, value))
1299 |         for value in args:
1300 |             cmd.append(value)
1301 | 
1302 |         p=  subprocess.run(cmd, stdout=subprocess.PIPE, cwd=self.dir)
1303 |         if p.returncode:
1304 |             sys.stdout.buffer.write(p.stdout)
1305 |             raise Exception('Error')
1306 |         print("vex {}".format(" ".join(cmd[1:])))
1307 |         for line in p.stdout.splitlines():
1308 |             print(">  ", line.decode('utf-8'))
1309 | 
1310 | @debug_test.on_run()
1311 | @argspec('--git? [dir]')
1312 | def DebugTest(git, dir):
1313 | 
1314 | 
1315 |     def do(dir):
1316 |         dir = os.path.join(dir, 'repo')
1317 |         os.makedirs(dir, exist_ok=True)
1318 |         print("Using:", dir)
1319 |         def shell(args):
1320 |             print('shell:', args)
1321 |             p= subprocess.run(args, stdout=subprocess.PIPE, shell=True, cwd=dir)
1322 |             if p.returncode:
1323 |                 sys.stdout.write(p.stdout)
1324 |                 raise Exception('error')
1325 |             return p.stdout
1326 |         vex = Vex(os.path.normpath(os.path.join(os.path.split(os.path.abspath(__file__))[0], "..", "vex")), dir)
1327 | 
1328 |         vex.init(git=git)
1329 | 
1330 |         shell('date >> date')
1331 |         shell('mkdir -p dir1 dir2 dir3/dir3.1 dir3/dir3.2')
1332 |         shell('echo yes >> dir1/a')
1333 |         shell('echo yes >> dir1/b')
1334 |         shell('echo yes >> dir1/c')
1335 | 
1336 |         vex.add()
1337 |         vex.commit()
1338 | 
1339 |         vex.undo()
1340 |         vex.status(all=True)
1341 |         vex.commit.prepare()
1342 |         vex.status(all=True)
1343 |         vex.commit.prepared()
1344 |         vex.status(all=True)
1345 | 
1346 |         vex.undo()
1347 |         vex.undo()
1348 |         vex.redo(choice=1)
1349 |         vex.log()
1350 |         shell('date >> date')
1351 |         vex.status(all=True)
1352 |         vex.switch()
1353 |         vex.switch('dir1')
1354 |         shell('rm a')
1355 |         shell('mkdir a')
1356 |         vex.switch('/repo')
1357 |         vex.undo()
1358 |         vex.redo()
1359 |         vex.status(all=True)
1360 |         vex.commit()
1361 |         vex.status(all=True)
1362 |         shell('rmdir dir2')
1363 |         shell('date >> dir2')
1364 |         vex.status(all=True)
1365 |         vex.commit()
1366 |         vex.undo()
1367 |         vex.branch.saveas('other')
1368 |         vex.branch('latest')
1369 |         vex.undo()
1370 |         vex.commit()
1371 |         vex.branch('latest')
1372 |         vex.status(all=True)
1373 |         vex.id()
1374 |     if not dir:
1375 |         with tempfile.TemporaryDirectory() as dir:
1376 |             do(dir)
1377 |     else:
1378 |         dir = os.path.join(os.getcwd(),dir)
1379 |         os.makedirs(dir, exist_ok=True)
1380 |         print(dir)
1381 |         do(dir)
1382 | 
1383 |     
1384 | @debug_soak.on_run()
1385 | def DebugSoak():
1386 |     pass
1387 | 
1388 | @debug_argparse.on_run()
1389 | @argspec('''
1390 |     --switch?       # a demo switch
1391 |     --value:str     # pass with --value=...
1392 |     --bucket:int... # a list of numbers
1393 |     pos1            # positional
1394 |     [opt1]          # optional 1
1395 |     [opt2]          # optional 2
1396 |     [tail...]       # tail arg
1397 | ''')
1398 | def run(switch, value, bucket, pos1, opt1, opt2, tail):
1399 |     """a demo command that shows all the types of options"""
1400 |     return [switch, value, bucket, pos1, opt1, opt2, tail]
1401 | 
1402 | # Git
1403 | 
1404 | vex_git_push = vex_git.subcommand('push')
1405 | vex_git_init = vex_git.subcommand('init')
1406 | vex_git_clone = vex_git.subcommand('clone')
1407 | vex_git_cat = vex_git.subcommand('cat')
1408 | 
1409 | @vex_git_push.on_run()
1410 | @argspec('url [remote_branch]')
1411 | def GitPush(url, remote_branch):
1412 |     p = open_project()
1413 |     if not p.git:
1414 |         raise VexArgument('no')
1415 |     with p.lock('git-push') as p:
1416 |         active = p.active()
1417 |         branch = p.branches.get(active.branch)
1418 |         remote_branch = remote_branch or branch.name
1419 |         return p.repo.push(url, remote_branch, branch.head)
1420 | 
1421 | @vex_git_cat.on_run()
1422 | @argspec('commit')
1423 | def GitCat(commit):
1424 |     p = open_project()
1425 |     if commit.startswith('git'):
1426 |         return p.repo.cat_file(commit).decode('utf-8')
1427 | 
1428 | 
1429 | @vex_git_clone.on_run()
1430 | @argspec('''
1431 |     --working:path    # Working directory, where files are edited/changed
1432 |     --config:path     # Normally /working_dir/.vex if not given 
1433 |     --prefix:path     # Subdirectory to check out of the repository, normally the working directory name
1434 |     --include:str... # files to include whe using vex add, can be passed multiple times 
1435 |     --ignore:str...  # files to ignore when using vex add, can be passed multiple times
1436 |     --author:str    #
1437 |     --email:str     #
1438 |     url              # 
1439 |     [directory]      #
1440 | ''')
1441 | def GitClone(url, directory, working, config, prefix, include, ignore, author, email):
1442 |     """
1443 |         Create a new vex project in a given directory from the given git url
1444 | 
1445 |         - If no directory given, it is assumed to be the current directory.
1446 |         - Inside that directory, a `.vex` directory is created to store the project history.
1447 |         - An initial empty commit is added.
1448 |         - The subtree checkout defaults to `/directory_name`.
1449 |         
1450 |         i.e a `vex init` in `/a/b` creates a `/a/b/.vex` directory, an empty commit, and checks
1451 |         out `/b` in the repo, into `/a/b` on the filesystem.`
1452 | 
1453 |         If you make a mistake, `vex undo` will undo the initial commit, but not remove
1454 |         the `.vex` directory. 
1455 | 
1456 |         `init` takes multiple `--include=<file>` and `--ignore=<file>` arguments, 
1457 |         defaulting to `--include='*' --ignore='.vex' --ignore='.*'`
1458 | 
1459 |         `--include`, `--ignore`, can be passed multiple times, and work the 
1460 |         same as `vex include 'pattern'` and `vex ignore 'pattern'`
1461 | 
1462 |     """
1463 | 
1464 |     name = url.rsplit('/',1)[1].rsplit('.git',1)[0]
1465 | 
1466 |     working_dir = working or directory or os.path.join(os.getcwd(), name)
1467 |     config_dir = config or os.path.join(working_dir,  DEFAULT_CONFIG_DIR)
1468 |     prefix = prefix or '/'
1469 |     prefix = os.path.join('/', prefix)
1470 | 
1471 |     include = include or DEFAULT_INCLUDE
1472 |     ignore = ignore or DEFAULT_IGNORE
1473 | 
1474 |     if not author:
1475 |         author = subprocess.run(['git','config','--global', 'user.name'], encoding='utf-8', check=True, stdout=subprocess.PIPE).stdout.strip()
1476 | 
1477 |     if not email:
1478 |         email = subprocess.run(['git','config','--global', 'user.email'], encoding='utf-8', check=True, stdout=subprocess.PIPE).stdout.strip()
1479 |     p = Project(config_dir, working_dir, fake, git=True)
1480 | 
1481 |     if p.exists() and not p.clean_state():
1482 |         yield ('This vex project is unwell. Try `vex debug:status`')
1483 |     elif p.exists():
1484 |         raise VexError("A vex project already exists here")
1485 |     else:
1486 |         yield ('Creating vex project in "{}"...'.format(working_dir))
1487 |         yield p.repo.clone(url)
1488 |         p.makedirs()
1489 |         p.makelock()
1490 |         with p.lock('git:clone') as p:
1491 |             yield ('Creating working env')
1492 |             p.init_from_git_clone(prefix, include, ignore, author, email)
1493 | 
1494 | @vex_git_init.on_run()
1495 | @argspec('''
1496 |     --working:path    # Working directory, where files are edited/changed
1497 |     --config:path     # Normally /working_dir/.vex if not given 
1498 |     --prefix:path     # Subdirectory to check out of the repository, normally the working directory name
1499 |     --include:str... # files to include whe using vex add, can be passed multiple times 
1500 |     --ignore:str...  # files to ignore when using vex add, can be passed multiple times
1501 |     --author:str
1502 |     --email:str
1503 |     [directory]      # defaults to '.'
1504 | ''')
1505 | def GitInit(directory, working, config, prefix, include, ignore, author, email):
1506 |     """
1507 |         Create a new vex project in a given directory from the given git url
1508 | 
1509 |         - If no directory given, it is assumed to be the current directory.
1510 |         - Inside that directory, a `.vex` directory is created to store the project history.
1511 |         - An initial empty commit is added.
1512 |         - The subtree checkout defaults to `/directory_name`.
1513 |         
1514 |         i.e a `vex init` in `/a/b` creates a `/a/b/.vex` directory, an empty commit, and checks
1515 |         out `/b` in the repo, into `/a/b` on the filesystem.`
1516 | 
1517 |         If you make a mistake, `vex undo` will undo the initial commit, but not remove
1518 |         the `.vex` directory. 
1519 | 
1520 |         `init` takes multiple `--include=<file>` and `--ignore=<file>` arguments, 
1521 |         defaulting to `--include='*' --ignore='.vex' --ignore='.*'`
1522 | 
1523 |         `--include`, `--ignore`, can be passed multiple times, and work the 
1524 |         same as `vex include 'pattern'` and `vex ignore 'pattern'`
1525 | 
1526 |     """
1527 | 
1528 |     working_dir = working or directory or os.getcwd()
1529 |     config_dir = config or os.path.join(working_dir,  DEFAULT_CONFIG_DIR)
1530 |     prefix = prefix or '/'
1531 |     prefix = os.path.join('/', prefix)
1532 | 
1533 |     include = include or DEFAULT_INCLUDE
1534 |     ignore = ignore or DEFAULT_IGNORE
1535 | 
1536 |     if not author:
1537 |         author = subprocess.run(['git','config','--global', 'user.name'], encoding='utf-8', check=True, stdout=subprocess.PIPE).stdout.strip()
1538 | 
1539 |     if not email:
1540 |         email = subprocess.run(['git','config','--global', 'user.email'], encoding='utf-8', check=True, stdout=subprocess.PIPE).stdout.strip()
1541 |     p = Project(config_dir, working_dir, fake, git=True)
1542 | 
1543 |     if p.exists() and not p.clean_state():
1544 |         yield ('This vex project is unwell. Try `vex debug:status`')
1545 |     elif p.exists():
1546 |         raise VexError("A vex project already exists here")
1547 |     else:
1548 |         yield ('Creating vex project in "{}"...'.format(working_dir))
1549 |         p.makedirs()
1550 |         p.makelock()
1551 |         with p.lock('git:init') as p:
1552 |             yield ('Creating working env')
1553 |             p.init_from_git_init(prefix, include, ignore, author, email)
1554 | 
1555 | 
1556 | 
1557 | vex_cmd.main(__name__)
1558 | 


--------------------------------------------------------------------------------
/vexlib/errors.py:
--------------------------------------------------------------------------------
 1 | """
 2 |     hooray for exceptions
 3 | """
 4 | class VexError(Exception): pass
 5 | 
 6 | # Should not happen: bad state reached internally
 7 | # always throws exception
 8 | 
 9 | class VexBug(Exception): pass
10 | class VexCorrupt(Exception): pass
11 | 
12 | # Can happen: bad environment/arguments
13 | class VexLock(Exception): pass
14 | class VexArgument(VexError): pass
15 | 
16 | # Recoverable State Errors
17 | class VexNoProject(VexError): pass
18 | class VexNoHistory(VexError): pass
19 | class VexUnclean(VexError): pass
20 | 
21 | class VexUnimplemented(VexError): pass
22 | 
23 | 


--------------------------------------------------------------------------------
/vexlib/fs.py:
--------------------------------------------------------------------------------
  1 | """
  2 | 
  3 |     - filename matching to gitignore style globs
  4 | 
  5 |     - listing directories recursively using said patterns
  6 | 
  7 |     - blob store (content addressable thingy)
  8 | 
  9 |     - file store (read write objects to named files, using rson to ser/deser)
 10 |     
 11 |     - shelling out to diff
 12 | 
 13 | """
 14 | import fnmatch
 15 | import hashlib
 16 | import subprocess
 17 | import os
 18 | import os.path
 19 | import sys
 20 | import shutil
 21 | import sqlite3
 22 | 
 23 | 
 24 | from contextlib import contextmanager
 25 | from uuid import uuid4
 26 | from datetime import datetime, timezone
 27 | 
 28 | def UUID(): return str(uuid4())
 29 | def NOW(): return datetime.now(timezone.utc)
 30 | 
 31 | 
 32 | try:
 33 |     import fcntl
 34 | 
 35 | 
 36 |     class LockFile:
 37 |         def __init__(self, lockfile):
 38 |             self.lockfile = lockfile
 39 | 
 40 |         def makelock(self):
 41 |             with open(self.lockfile, 'xb') as fh:
 42 |                 fh.write(b'# created by %d at %a\n'%(os.getpid(), str(NOW())))
 43 | 
 44 |         @contextmanager
 45 |         def __call__(self, command):
 46 |             try:
 47 |                 fh = open(self.lockfile, 'wb')
 48 |                 fcntl.flock(fh, fcntl.LOCK_EX | fcntl.LOCK_NB)
 49 |             except (IOError, FileNotFoundError):
 50 |                 raise VexLock('Cannot open project lockfile: {}'.format(self.lockfile))
 51 |             try:
 52 |                 fh.truncate(0)
 53 |                 fh.write(b'# locked by %d at %a\n'%(os.getpid(), str(NOW())))
 54 |                 fh.write("{}".format(command).encode('utf-8'))
 55 |                 fh.write(b'\n')
 56 |                 fh.flush()
 57 |                 yield self
 58 |                 fh.write(b'# released by %d %a\n'%(os.getpid(), str(NOW())))
 59 |             finally:
 60 |                 fh.close()
 61 | except ImportError:
 62 |     import msvcrt
 63 |     class LockFile:
 64 |         def __init__(self, lockfile):
 65 |             self.lockfile = lockfile
 66 | 
 67 |         def makelock(self):
 68 |             with open(self.lockfile, 'xb') as fh:
 69 |                 fh.write(b'# created by %d at %a\n'%(os.getpid(), str(NOW())))
 70 | 
 71 |         @contextmanager
 72 |         def __call__(self, command):
 73 |             try:
 74 |                 fh = open(self.lockfile, 'wb')
 75 |                 msvcrt.locking(fh.fileno(), msvcrt.LK_NBLCK, 1)
 76 |             except (OSError, IOError, FileNotFoundError):
 77 |                 raise VexLock('Cannot open project lockfile: {}'.format(self.lockfile))
 78 |             try:
 79 |                 yield self
 80 |             finally:
 81 |                 msvcrt.locking(fh.fileno(), msvcrt.LK_UNLCK, 1)
 82 | 
 83 | 
 84 | 
 85 | # History: Used to track undo/redo and changes to repository state
 86 | 
 87 | class HistoryStore:
 88 |     def __init__(self, file, codec):
 89 |         self.file = file
 90 |         self._db = None
 91 |         self.codec = codec
 92 |     
 93 |     @property
 94 |     def db(self):
 95 |         if self._db:
 96 |             return self._db
 97 |         self._db = sqlite3.connect(self.file)
 98 |         return self._db
 99 | 
100 |     def makedirs(self):
101 |         c = self.db.cursor()
102 |         c.execute('''
103 |             create table if not exists dos(
104 |                 addr text primary key,
105 |                 prev text not null, 
106 |                 timestamp datetime default current_timestamp,
107 |                 action text)
108 |         ''')
109 |         c.execute('''
110 |             create table if not exists redos(
111 |                 addr text primary key,
112 |                 dos text not null)
113 |         ''')
114 |         c.execute('''
115 |             create table if not exists next(
116 |                 id INTEGER PRIMARY KEY CHECK (id = 0) default 0,
117 |                 mode text not null,
118 |                 value text not null,
119 |                 current text)
120 |         ''')
121 |         c.execute('''
122 |             create table if not exists current (
123 |                 id INTEGER PRIMARY KEY CHECK (id = 0) default 0,
124 |                 value text not null)
125 |         ''')
126 |         self.db.commit()
127 | 
128 |     def exists(self):
129 |         if os.path.exists(self.file):
130 |             c = self.db.cursor()
131 |             c.execute('''SELECT name FROM sqlite_master WHERE name='current' ''')
132 |             if bool(c.fetchone()):
133 |                 return bool(self.current())
134 | 
135 |     def current(self):
136 |         c = self.db.cursor() 
137 |         c.execute('select value from current where id = 0')
138 |         row = c.fetchone()
139 |         if row:
140 |             return self.codec.parse(row[0])
141 | 
142 |     def next(self):
143 |         c = self.db.cursor() 
144 |         c.execute('select mode, value, current from next where id = 0')
145 |         row = c.fetchone()
146 |         if row:
147 |             return str(row[0]), str(row[1]), str(row[2])
148 | 
149 |     def set_current(self, value):
150 |         if not value: raise Exception()
151 |         c = self.db.cursor() 
152 |         buf = self.codec.dump(value)
153 |         c.execute('update current set value = ? where id = 0', [buf])
154 |         c.execute('insert or ignore into current (id,value) values (0,?)',[buf])
155 |         self.db.commit()
156 | 
157 |     def set_next(self, mode, value, current):
158 |         c = self.db.cursor() 
159 |         c.execute('update next set mode = ?, value=?, current = ? where id = 0', [mode, value, current])
160 |         c.execute('insert or ignore into next (id,mode, value,current) values (0,?,?,?)',[mode, value, current])
161 |         self.db.commit()
162 | 
163 |     def get_entry(self, addr):
164 |         c = self.db.cursor() 
165 |         c.execute('select prev, action from dos where addr = ?', [addr])
166 |         row = c.fetchone()
167 |         if row:
168 |             return (str(row[0]),self.codec.parse(row[1]))
169 | 
170 |     def put_entry(self, prev, obj):
171 |         c=self.db.cursor()
172 |         buf = self.codec.dump(obj)
173 |         addr = UUID().split('-',1)[0]
174 |         c.execute('insert into dos (addr, prev, action) values (?,?,?)',[addr, prev, buf])
175 |         self.db.commit()
176 |         return addr
177 | 
178 |     def get_redos(self, addr):
179 |         c = self.db.cursor() 
180 |         c.execute('select dos from redos where addr = ?', [addr])
181 |         row = c.fetchone()
182 |         if row and row[0]:
183 |             row = str(row[0]).split(",")
184 |             return row
185 |         return []
186 | 
187 |     def set_redos(self, addr, redo):
188 |         c = self.db.cursor() 
189 |         buf = ",".join(redo) if redo else ""
190 |         c.execute('update redos set dos = ? where addr = ?', [buf, addr])
191 |         self.db.commit()
192 |         c.execute('insert or ignore into redos (addr,dos) values (?,?)',[addr, buf])
193 |         self.db.commit()
194 | 
195 | # Filename patterns
196 | 
197 | from .errors import *
198 | 
199 | def match_filename(path, name, ignore, include):
200 |     if ignore:
201 |         if isinstance(ignore, str): ignore = ignore,
202 |         for rule in ignore:
203 |             if '**' in rule:
204 |                 raise VexUnimplemented()
205 |             elif rule.startswith('/'):
206 |                 if rule == path:
207 |                     return False
208 |             elif fnmatch.fnmatch(name, rule):
209 |                 return False
210 | 
211 |     if include:
212 |         if isinstance(include, str): include = include,
213 |         for rule in include:
214 |             if '**' in rule:
215 |                 raise VexUnimplemented()
216 |             elif rule.startswith('/'):
217 |                 if rule == path:
218 |                     return True
219 |             elif fnmatch.fnmatch(name, rule):
220 |                 return True
221 | def file_diff(name, old, new):
222 |     # XXX Pass properties
223 |     a,b = os.path.join('./a',name[1:]), os.path.join('./b', name[1:])
224 |     p = subprocess.run(["diff", '-u', '--label', a, '--label', b, old, new], stdout=subprocess.PIPE, encoding='utf8')
225 |     return p.stdout
226 | 
227 | 
228 | def list_dir(dir, ignore, include):
229 |     output = []
230 |     scan = [dir]
231 |     while scan:
232 |         dir = scan.pop()
233 |         with os.scandir(dir) as ls:
234 |             for f in ls:
235 |                 p = f.path
236 |                 if not match_filename(p, f.name, ignore, include): continue
237 |                 if f.is_dir():
238 |                     output.append(p)
239 |                     scan.append(p)
240 |                 elif f.is_file():
241 |                     output.append(p)
242 |     return output
243 | # Stores
244 | 
245 | class FileStore:
246 |     def __init__(self, dir, codec, rawkeys=()):
247 |         self.codec = codec
248 |         self.dir = dir
249 |         self.rawkeys = rawkeys
250 | 
251 |     def makedirs(self):
252 |         os.makedirs(self.dir, exist_ok=True)
253 |     def filename(self, name):
254 |         return os.path.join(self.dir, name)
255 |     def list(self):
256 |         for name in os.listdir(self.dir):
257 |             if os.path.isfile(self.filename(name)):
258 |                 yield name
259 |     def exists(self, addr):
260 |         return os.path.exists(self.filename(addr))
261 |     def get(self, name):
262 |         if not self.exists(name):
263 |             if name in self.rawkeys:
264 |                 return ""
265 |             return None
266 |         with open(self.filename(name), 'rb') as fh:
267 |             return self.parse(name, fh.read())
268 |     def set(self, name, value):
269 |         with open(self.filename(name),'w+b') as fh:
270 |             fh.write(self.dump(name, value))
271 |     def parse(self, name, value):
272 |         if name in self.rawkeys:
273 |             return value.decode('utf-8')
274 |         else:
275 |             return self.codec.parse(value)
276 |     def dump(self, name, value):
277 |         if name in self.rawkeys:
278 |             if value:
279 |                 return value.encode('utf-8')
280 |             else:
281 |                 return b""
282 |         else:
283 |             return self.codec.dump(value)
284 | 
285 | class BlobStore:
286 |     prefix = "vex:"
287 |     def __init__(self, dir, codec):
288 |         self.dir = dir
289 |         self.codec = codec
290 | 
291 |     def hashlib(self):
292 |         return hashlib.shake_256()
293 | 
294 |     def prefixed_addr(self, hash):
295 |         return "{}{}".format(self.prefix, hash.hexdigest(20))
296 | 
297 |     def makedirs(self):
298 |         os.makedirs(self.dir, exist_ok=True)
299 | 
300 |     def copy_from(self, other, addr):
301 |         if other.exists(addr) and not self.exists(addr):
302 |             src, dest = other.filename(addr), self.filename(addr)
303 |             os.makedirs(os.path.split(dest)[0], exist_ok=True)
304 |             shutil.copyfile(src, dest)
305 |         elif not self.exists(addr):
306 |             raise VexCorrupt('Missing file {}'.format(other.filename(addr)))
307 | 
308 |     def move_from(self, other, addr):
309 |         if other.exists(addr) and not self.exists(addr):
310 |             src, dest = other.filename(addr), self.filename(addr)
311 |             os.makedirs(os.path.split(dest)[0], exist_ok=True)
312 |             os.rename(src, dest)
313 |         elif not self.exists(addr):
314 |             raise VexCorrupt('Missing file {}'.format(other.filename(addr)))
315 | 
316 |     def make_copy(self, addr, dest):
317 |         filename = self.filename(addr)
318 |         shutil.copyfile(filename, dest)
319 | 
320 |     def addr_for_file(self, file):
321 |         hash = self.hashlib()
322 |         with open(file,'rb') as fh:
323 |             buf = fh.read(4096)
324 |             while buf:
325 |                 hash.update(buf)
326 |                 buf = fh.read(4096)
327 |         return self.prefixed_addr(hash)
328 | 
329 | 
330 |     def inside(self, file):
331 |         return os.path.commonpath((file, self.dir)) == self.dir
332 | 
333 |     def addr_for_buf(self, buf):
334 |         hash = self.hashlib()
335 |         hash.update(buf)
336 |         return self.prefixed_addr(hash)
337 | 
338 |     def filename(self, addr):
339 |         if not addr.startswith(self.prefix):
340 |             raise VexBug('bug')
341 |         addr = addr[len(self.prefix):]
342 |         return os.path.join(self.dir, addr[:2], addr[2:])
343 | 
344 |     def exists(self, addr):
345 |         return os.path.exists(self.filename(addr))
346 | 
347 |     def put_file(self, file, addr=None):
348 |         addr = addr or self.addr_for_file(file)
349 |         if not self.exists(addr):
350 |             filename = self.filename(addr)
351 |             os.makedirs(os.path.split(filename)[0], exist_ok=True)
352 |             shutil.copyfile(file, filename)
353 |         return addr
354 | 
355 |     def put_buf(self, buf, addr=None):
356 |         addr = addr or self.addr_for_buf(buf)
357 |         if not self.exists(addr):
358 |             filename = self.filename(addr)
359 |             os.makedirs(os.path.split(filename)[0], exist_ok=True)
360 |             with open(filename, 'xb') as fh:
361 |                 fh.write(buf)
362 |         return addr
363 | 
364 |     def put_obj(self, obj):
365 |         buf = self.codec.dump(obj)
366 |         return self.put_buf(buf)
367 | 
368 |     def get_file(self, addr):
369 |         return self.filename(addr)
370 | 
371 |     def get_obj(self, addr):
372 |         with open(self.filename(addr), 'rb') as fh:
373 |             return self.codec.parse(fh.read())
374 | 
375 | class Repo:
376 |     def __init__(self, config_dir, codec):
377 |         self.dir = config_dir
378 |         self.commits =   BlobStore(os.path.join(config_dir, 'objects', 'commits'), codec)
379 |         self.manifests = BlobStore(os.path.join(config_dir, 'objects', 'manifests'), codec)
380 |         self.files =     BlobStore(os.path.join(config_dir, 'objects', 'files'), codec)
381 |         self.scratch =   BlobStore(os.path.join(config_dir, 'objects', 'scratch'), codec)
382 | 
383 |     def makedirs(self):
384 |         os.makedirs(self.dir, exist_ok=True)
385 |         self.commits.makedirs()
386 |         self.manifests.makedirs()
387 |         self.files.makedirs()
388 |         self.scratch.makedirs()
389 | 
390 |     def addr_for_file(self, path):
391 |         return self.scratch.addr_for_file(path)
392 | 
393 |     def get_commit(self, addr):
394 |         return self.commits.get_obj(addr)
395 | 
396 |     def get_manifest(self, addr):
397 |         return self.manifests.get_obj(addr)
398 | 
399 |     def get_file(self, addr):
400 |         return self.files.get_file(addr)
401 | 
402 |     def get_scratch_file(self, addr):
403 |         return self.scratch.get_file(addr)
404 | 
405 |     def get_scratch_commit(self,addr):
406 |         return self.scratch.get_obj(addr)
407 | 
408 |     def get_scratch_manifest(self, addr):
409 |         return self.scratch.get_obj(addr)
410 | 
411 |     def put_scratch_file(self, value, addr=None):
412 |         return self.scratch.put_file(value, addr)
413 | 
414 |     def put_scratch_commit(self, value):
415 |         return self.scratch.put_obj(value)
416 | 
417 |     def put_scratch_manifest(self, value):
418 |         return self.scratch.put_obj(value)
419 | 
420 |     def add_commit_from_scratch(self, addr):
421 |         self.commits.copy_from(self.scratch, addr)
422 | 
423 |     def add_manifest_from_scratch(self, addr):
424 |         self.manifests.copy_from(self.scratch, addr)
425 | 
426 |     def add_file_from_scratch(self, addr):
427 |         self.files.copy_from(self.scratch, addr)
428 | 
429 |     def get_file_path(self, addr):
430 |         # diff
431 |         return self.files.filename(addr)
432 | 
433 |     def copy_from_scratch(self, addr, path):
434 |         self.scratch.make_copy(addr, path)
435 | 
436 |     def copy_from_file(self, addr, path):
437 |         self.files.make_copy(addr, path)
438 | 
439 |     def copy_from_any(self, addr, path):
440 |         if self.files.exists(addr):
441 |             return self.files.make_copy(addr, path)
442 | 
443 |         self.scratch.make_copy(addr, path)
444 | 
445 | 
446 | class GitRepo:
447 |     def __init__(self, config_dir, codec):
448 |         self.dir = os.path.join(config_dir, 'git')
449 |         self.codec = codec
450 |         self.env = {'GIT_DIR':self.dir}
451 | 
452 |     def makedirs(self):
453 |         os.makedirs(self.dir, exist_ok=True)
454 |         subprocess.run(['git', 'init', '-q', '--bare', self.dir])
455 |     
456 |     def clone(self, url):
457 |         os.makedirs(self.dir, exist_ok=True)
458 |         p = subprocess.run(['git', 'clone',  '-q', '--bare', url, self.dir])
459 |         return p.stdout
460 | 
461 |     def branches(self):
462 |         dir = os.path.join(self.dir, 'refs', 'heads')
463 |         branches = {}
464 |         for name in os.listdir(dir):
465 |             with open(os.path.join(dir, name)) as fh:
466 |                 branches[name] = fh.read().strip()
467 |         with open(os.path.join(self.dir, 'packed-refs')) as fh:
468 |             for line in fh.readlines():
469 |                 line = line.split(' ',1)
470 |                 if len(line) != 2:
471 |                     continue
472 |                 value, name = line
473 |                 name = name.strip()
474 |                 if name.startswith('refs/heads/'):
475 |                     branches[name.rsplit('/',1)[1]] = value
476 | 
477 |         return branches
478 |                 
479 |     def head(self):
480 |         with open(os.path.join(self.dir, 'HEAD')) as fh:
481 |             head = fh.read().strip()
482 |             if head.startswith('ref: refs/heads/'):
483 |                 return head.rsplit('/',1)[1]
484 |             raise VexBug('welp')
485 | 
486 | 
487 |     def push(self, url, remote_branch, commit):
488 |         p = subprocess.run(['git', 'push', url, '{}:refs/heads/{}'.format(commit[4:], remote_branch)], stdout=subprocess.PIPE, encoding='utf-8', env=self.env)
489 |         return p.stdout
490 |         
491 | 
492 |     def addr_for_file(self, path):
493 |         p = subprocess.run(['git', 'hash-object','-t','blob', path], stdout=subprocess.PIPE, encoding='utf-8', env=self.env)
494 |         return "git:{}".format(p.stdout.strip())
495 | 
496 |     def diff(self,old, new):
497 |         cmd = ['git', 'diff', old[4:], new[4:]]
498 |         p = subprocess.run(cmd, stdout=subprocess.PIPE, encoding='utf8')
499 |         return p.stdout
500 | 
501 |     def cat_file(self, addr):
502 |         p = subprocess.run(['git', 'cat-file', '-p', addr[4:]], stdout=subprocess.PIPE, stderr=subprocess.PIPE, env=self.env)
503 |         return p.stdout
504 | 
505 |     def get_commit(self, addr):
506 |         out = self.codec.parse_git_inline(addr)
507 |         if out is not None: return out
508 |         p = subprocess.run(['git', 'cat-file',  'commit', addr[4:]], stdout=subprocess.PIPE, stderr=subprocess.PIPE, env=self.env)
509 |         return self.codec.parse_git_commit(p.stdout)
510 | 
511 |     def get_manifest(self, addr):
512 |         out = self.codec.parse_git_inline(addr)
513 |         if out is not None: return out
514 |         p = subprocess.run(['git', 'cat-file', 'tree', addr[4:]], stdout=subprocess.PIPE,stderr=subprocess.PIPE, env=self.env)
515 |         return self.codec.parse_git_tree(p.stdout)
516 | 
517 |     def copy_from_any(self, addr, path):
518 |         with open(path, 'xb') as fh:
519 |             p = subprocess.run(['git', 'cat-file', 'blob', addr[4:]], stdout=fh, stderr=subprocess.PIPE, env=self.env)
520 | 
521 |     def put_scratch_file(self, value, addr=None):
522 |         p = subprocess.run(['git', 'hash-object', '-w','-t', 'blob',  value], stdout=subprocess.PIPE, stderr=subprocess.PIPE, encoding='utf-8', env=self.env)
523 |         o = p.stdout.strip()
524 |         return "git:{}".format(o)
525 | 
526 |     def put_scratch_commit(self, value):
527 |         out = self.codec.dump_git_inline(value)
528 |         if out: return out
529 | 
530 |         buf = self.codec.dump_git_commit(value) # -t commit
531 |         p = subprocess.Popen(['git', 'hash-object', '-t', 'commit', '-w', '--stdin'], stdout=subprocess.PIPE, stdin=subprocess.PIPE, stderr=subprocess.PIPE, env=self.env)
532 |         p.stdin.write(buf)
533 |         p.stdin.close()
534 |         o= p.stdout.read().decode('utf-8').strip()
535 |         return "git:{}".format(o)
536 | 
537 |     def put_scratch_manifest(self, value):
538 |         out = self.codec.dump_git_inline(value)
539 |         if out: return out
540 |         ### inlining changelog objects
541 |         buf = self.codec.dump_git_tree(value) # -t blob
542 |         p = subprocess.Popen(['git', 'hash-object', '-w', '-t', 'tree', '--stdin'], stdout=subprocess.PIPE, stdin=subprocess.PIPE,  stderr=subprocess.PIPE, env=self.env)
543 |         p.stdin.write(buf)
544 |         p.stdin.close()
545 |         o= p.stdout.read().decode('utf-8').strip()
546 |         return "git:{}".format(o)
547 |         
548 |     def get_scratch_commit(self, addr):
549 |         return self.get_commit(addr)
550 | 
551 |     def get_scratch_manifest(self, addr):
552 |         return self.get_manifest(addr)
553 | 
554 |     def add_commit_from_scratch(self, addr):
555 |         return
556 | 
557 |     def add_manifest_from_scratch(self, addr):
558 |         return
559 | 
560 |     def add_file_from_scratch(self, addr):
561 |         return
562 | 
563 |     def copy_from_scratch(self, addr, path):
564 |         return self.copy_from_any(addr, path)
565 | 
566 |     def copy_from_file(self, addr, path):
567 |         return self.copy_from_any(addr, path)
568 | 
569 |     def get_file(self, addr):
570 |         raise Exception() # Unused
571 | 
572 |     def get_scratch_file(self, addr):
573 |         raise Exception() # Unused
574 | 
575 |     def get_file_path(self, addr):
576 |         raise Exception('nope')
577 | 
578 | 


--------------------------------------------------------------------------------
/vexlib/rson.py:
--------------------------------------------------------------------------------
   1 | #!/usr/bin/env python3
   2 | r"""
   3 | # RSON: Restructured Object Notation
   4 | 
   5 | RSON is JSON, with a little bit of sugar: Comments, Commas, and Tags.
   6 | 
   7 | For example:
   8 | 
   9 | ```
  10 | {
  11 |     "numbers": +0123.0,       # Can have leading zeros
  12 |     "octal": 0o10,            # Oh, and comments too
  13 |     "hex": 0xFF,              #
  14 |     "binary": 0b1000_0001,     # Number literals can have _'s 
  15 | 
  16 |     "lists": [1,2,3],         # Lists can have trailing commas
  17 | 
  18 |     "strings": "At least \x61 \u0061 and \U00000061 work now",
  19 |     "or": 'a string',          # both "" and '' work.
  20 | 
  21 |     "records": {
  22 |         "a": 1,               # Must have unique keys
  23 |         "b": 2,               # and the order must be kept
  24 |     },
  25 | }
  26 | ```
  27 | 
  28 | Along with some sugar atop JSON, RSON supports tagging literals to represent types outside of JSON:
  29 | 
  30 | - `@datetime "2017-11-22T23:32:07.100497Z"`, a tagged RFC 3339 datestamp
  31 | - `@duration 60` (a duration in seconds, float or int)
  32 | - `@base64 "...=="`, a base64 encoded bytestring
  33 | - `@set`, `@dict`, `@complex`, `@bytestring`
  34 | 
  35 | 
  36 | ## JSON in a nutshell:
  37 | 
  38 |  - A unicode text file, without a Byte Order Mark
  39 |  - Whitespace is `\t`, `\r`, `\n`, `\x20`
  40 |  - JSON document is either list, or object
  41 |  - Lists are `[]`, `[obj]`, `[ obj, obj ]`, ...
  42 |  - Objects: `{ "key": value}`, only string keys
  43 |  - Built-ins: `true`, `false`, `null`
  44 |  - `"unicode strings"` with escapes `\" \\ \/ \b \f \n \r \t \uFFFF`, and no control codes unecaped.
  45 |  - int/float numbers (unary minus, no leading zeros, except for `0.xxx`)
  46 |  - No Comments, No Trailing commas
  47 | 
  48 | ## RSON in a Nutshell
  49 | 
  50 |  - File MUST be utf-8, not cesu-8/utf-16/utf-32, without surrogate pairs.
  51 |  - Use `#.... <end of line>` for comments
  52 |  - Byte Order Mark is treated as whitespace (along with `\x09`, `\x0a`, `\x0d`, `\x20`)
  53 |  - RSON Document is any RSON Object, (i.e `1` is a valid RSON file).
  54 |  - Lists are `[]`, `[obj]`, `[obj,]`, `[obj, obj]` ... (trailing comma optional)
  55 |  - Records are `{ "key": value}`, keys must be unique, order must be preserved. 
  56 |  - Built-ins: `true`, `false`, `null`
  57 |  - `"unicode strings"` with escapes `\" \\ \/ \b \f \n \r \t \uFFFF`, no control codes unecaped, and `''` can be used instead of `""`.
  58 |  - int/float numbers (unary plus or minus, allowleading zeros, hex, octal, and binary integer liters)
  59 |  - Tagged literals: `@name [1,2,3]` for any other type of value.
  60 | 
  61 | 
  62 | # RSON Object Model and Syntax
  63 | 
  64 | RSON has the following types of literals:
  65 | 
  66 |  - `null`, `true`, `false`
  67 |  - Integers (decimal, binary, octal, hex)
  68 |  - Floating Point
  69 |  - Strings (using single or double quotes)
  70 |  - Lists
  71 |  - Records (a JSON object with ordering and without duplicate keys)
  72 |  - Tagged Literal
  73 | 
  74 | RSON has a number of built-in tags:
  75 |  - `@object`, `@bool`, `@int`, `@float`, `@string`, `@list`, `@record`
  76 | 
  77 | As well as optional tags for other types:
  78 |  - `@bytestring`, or `@base64` for bytestrings
  79 |  - `@float "0x0p0"`, for C99 Hex Floating Point Literals
  80 |  - `@dict` for unordered key-value maps
  81 |  - `@set` for sets, `@complex` for complex numbers
  82 |  - `@datetime`, `@duration` for time as point or measurement.
  83 | 
  84 | ## RSON strings: 
  85 | 
  86 |  - use ''s or ""s
  87 |  - json escapes, and `\xFF` (as `\u00FF`), `\UFFFFFFFF`  `\'` too
  88 |  - `\` at end of line is continuation
  89 |  - no surrogate pairs
  90 | 
  91 | ## RSON numbers:
  92 | 
  93 |  - allow unary minus, plus
  94 |  - allow leading zero
  95 |  - allow underscores (except leading digits)
  96 |  - binary ints: `0b1010`
  97 |  - octal ints `0o777`
  98 |  - hex ints: `0xFF` 
  99 | 
 100 | ## RSON lists:
 101 | 
 102 |  - allow trailing commas
 103 | 
 104 | ## RSON records (aka, JSON objects):
 105 | 
 106 |  - no duplicate keys
 107 |  - insertion order must be preserved
 108 |  - allow trailing commas
 109 |  - implementations MUST support string keys
 110 | 
 111 | ## RSON tagged objects:
 112 | 
 113 |  - `@foo.foo {"foo":1}` name is any unicode letter/digit, `_`or a `.`
 114 |  - `@int 1`, `@string "two"` are just `1` and `"two"`
 115 |  - do not nest,
 116 |  - whitespace between tag name and object is *mandatory*
 117 |  - every type has a reserved tag name
 118 |  - parsers MAY reject unknown, or return a wrapped object 
 119 | 
 120 | ### RSON C99 float strings (optional):
 121 | 
 122 |  - `@float "0x0p0"` C99 style, sprintf('%a') format
 123 |  - `@float "NaN"` or nan,Inf,inf,+Inf,-Inf,+inf,-inf
 124 |  -  no underscores allowed
 125 | 
 126 | ### RSON sets (optional):
 127 | 
 128 |  - `@set [1,2,3]`
 129 |  - always a tagged list
 130 |  - no duplicate items
 131 | 
 132 | ### RSON dicts (optional):
 133 | 
 134 |  - `@dict {"a":1}` 
 135 |  - keys must be in lexical order, must round trip in same order.
 136 |  - no duplicate items
 137 |  - keys must be comparable, hashable, parser MAY reject if not
 138 | 
 139 | ### RSON datetimes/periods (optional):
 140 | 
 141 |  - RFC 3339 format in UTC, (i.e 'Zulu time')
 142 |  - `@datetime "2017-11-22T23:32:07.100497Z"`
 143 |  - `@duration 60` (in seconds, float or int)
 144 |  - UTC MUST be supported, using `Z` suffix
 145 |  - implementations should support subset of RFC 3339
 146 | 
 147 | ### RSON bytestrings (optional):
 148 | 
 149 |  - `@bytestring "....\xff"` 
 150 |  - `@base64 "...=="`
 151 |  - returns a bytestring if possible
 152 |  - can't have `\u` `\U` escapes > 0xFF
 153 |  - all non printable ascii characters must be escaped: `\xFF`
 154 | 
 155 | ### RSON complex numbers: (optional)
 156 | 
 157 |  - `@complex [0,1]` (real, imaginary)
 158 | 
 159 | ### Builtin RSON Tags:
 160 | 
 161 | Pass throughs (i.e `@foo bar` is `bar`):
 162 | 
 163 |  - `@object` on any 
 164 |  - `@bool` on true, or false
 165 |  - `@int` on ints
 166 |  - `@float` on ints or floats
 167 |  - `@string` on strings
 168 |  - `@list` on lists
 169 |  - `@record` on records
 170 | 
 171 | Tags that transform the literal:
 172 | 
 173 |  - @float on strings (for C99 hex floats, including NaN, -Inf, +Inf)
 174 |  - @duration on numbers (seconds)
 175 |  - @datetime on strings (utc timestamp)
 176 |  - @base64 on strings (into a bytesting)
 177 |  - @bytestring on strings (into a bytestring)
 178 |  - @set on lists 
 179 |  - @complex on lists
 180 |  - @dict on records
 181 | 
 182 | Reserved:
 183 | 
 184 |  - `@unknown`
 185 | 
 186 | Any other use of a builtin tag is an error and MUST be rejected.
 187 | 
 188 | # RSON Test Vectors
 189 | 
 190 | ## MUST parse
 191 | ```
 192 | @object null
 193 | @bool true
 194 | false
 195 | 0
 196 | @float 0.0
 197 | -0.0
 198 | "test-\x32-\u0032-\U00000032"
 199 | 'test \" \''
 200 | []
 201 | [1,]
 202 | {"a":"b",}
 203 | ```
 204 | 
 205 | ## MUST not parse
 206 | 
 207 | ```
 208 | _1
 209 | 0b0123
 210 | 0o999
 211 | 0xGHij
 212 | @set {}
 213 | @dict []
 214 | [,]
 215 | {"a"}
 216 | {"a":1, "a":2}
 217 | @object @object {}
 218 | "\uD800\uDD01"
 219 | ```
 220 | 
 221 | # Alternate Encodings
 222 | 
 223 | ## Binary RSON
 224 | 
 225 | Note: this is a work-in-progress
 226 | 
 227 | This is a simple Type-Length-Value style encoding, similar to bencoding or netstrings:
 228 | 
 229 | ```
 230 | OBJECT :== TRUE | FALSE | NULL |
 231 |                 INT | FLOAT | BYTES | STRING |
 232 |                 LIST | RECORD |
 233 |                 TAG 
 234 | 
 235 | TRUE :== 'y'
 236 | FALSE :== 'n'
 237 | NULL :== 'z'
 238 | INT :== 'i' <number encoded as ascii string> '\x7f'
 239 | FLOAT :== 'f' <number encoded as hex ascii string> '\x7f'
 240 | BYTES :== 'b' <INT as n> '\x7f' <n bytes> `\x7f`
 241 | STRING :== 'u' <INT as n> '\x7f' <n bytes of utf-8 encoded string> `\x7f`
 242 | 
 243 | LIST :== 'l' <INT as n> <n OBJECTs> `\x7f`
 244 | RECORD :== 'r' <INT as n> <2n OBJECTs> `\x7f`
 245 | TAG :== 't' <STRING as tag> <OBJECT as value> `\x7f`
 246 | ```
 247 | 
 248 | If a more compact representation is needed, use compression.
 249 | 
 250 | Work in Progress:
 251 | 
 252 | - Framing (i.e encaptulating in a len-checksum header, like Gob)
 253 | - tags for unsigned int8,16,32,64, signed ints
 254 | - tags for float32, float64
 255 | - tags for ints 0..31
 256 | - tags for field/tag definitions header
 257 | - tags for [type]/fixed width types
 258 | 
 259 | Rough plan: 
 260 | ```
 261 | Tags: 'A..J' 'K..T' 'S..Z'
 262 |     unsigned 8,16,32,64, (128,256,512,1024, 2048,4096)
 263 |     negative 8,16,32,64, (128,256,512,1024, 2048,4096)
 264 |     float 16, 32         (64, 128, 256, 512)
 265 | Tags \x00-\x31:
 266 |     ints 0-31
 267 | Tags >x127:
 268 |     Either using leading bit as unary continuation bit,
 269 |     Or, UTF-8 style '10'/'11' continuation bits.
 270 | ```
 271 | 
 272 | ## Decorated JSON (RSON inside JSON)
 273 | 
 274 | - `true`, `false`, `null`, numbers, strings, lists unchanged.
 275 | - `{"a":1}` becomes `{'record': ["a", 1]}`
 276 | - `@tag {'a':1}` becomes `{'tag', ["a", 1]}`
 277 | 
 278 | Note: In this scheme, `@tag ["a",1]` and `@tag {"a":1}` encode to the same JSON, and cannot be distinguished.
 279 | """
 280 | 
 281 | import re
 282 | import io
 283 | import base64
 284 | import sys
 285 | 
 286 | if sys.version_info.minor > 6 or sys.version_info.minor == 6 and sys.implementation.name == 'cpython':
 287 |     OrderedDict = dict
 288 |     from collections import namedtuple
 289 | else:
 290 |     from collections import namedtuple, OrderedDict
 291 | 
 292 | from datetime import datetime, timedelta, timezone
 293 | 
 294 | 
 295 | CONTENT_TYPE="application/rson"
 296 | 
 297 | reserved_tags = set("""
 298 |         bool int float complex
 299 |         string bytestring base64
 300 |         duration datetime
 301 |         set list dict record
 302 |         object
 303 |         unknown
 304 | """.split())
 305 | 
 306 | whitespace = re.compile(r"(?:\ |\t|\uFEFF|\r|\n|#[^\r\n]*(?:\r?\n|$))+")
 307 | 
 308 | int_b2 = re.compile(r"0b[01][01_]*")
 309 | int_b8 = re.compile(r"0o[0-7][0-7_]*")
 310 | int_b10 = re.compile(r"\d[\d_]*")
 311 | int_b16 = re.compile(r"0x[0-9a-fA-F][0-9a-fA-F_]*")
 312 | 
 313 | flt_b10 = re.compile(r"\.[\d_]+")
 314 | exp_b10 = re.compile(r"[eE](?:\+|-)?[\d+_]")
 315 | 
 316 | string_dq = re.compile(
 317 |     r'"(?:[^"\\\n\x00-\x1F\uD800-\uDFFF]|\\(?:[\'"\\/bfnrt]|\r?\n|x[0-9a-fA-F]{2}|u[0-9a-fA-F]{4}|U[0-9a-fA-F]{8}))*"')
 318 | string_sq = re.compile(
 319 |     r"'(?:[^'\\\n\x00-\x1F\uD800-\uDFFF]|\\(?:[\"'\\/bfnrt]|\r?\n|x[0-9a-fA-F]{2}|u[0-9a-fA-F]{4}|U[0-9a-fA-F]{8}))*'")
 320 | 
 321 | tag_name = re.compile(r"@(?!\d)\w+[ ]+")
 322 | identifier = re.compile(r"(?!\d)[\w\.]+")
 323 | 
 324 | c99_flt = re.compile(
 325 |     r"NaN|nan|[-+]?Inf|[-+]?inf|[-+]?0x[0-9a-fA-F][0-9a-fA-F]*\.[0-9a-fA-F]+[pP](?:\+|-)?[\d]+")
 326 | 
 327 | str_escapes = {
 328 |     'b': '\b',
 329 |     'n': '\n',
 330 |     'f': '\f',
 331 |     'r': '\r',
 332 |     't': '\t',
 333 |     '/': '/',
 334 |     '"': '"',
 335 |     "'": "'",
 336 |     '\\': '\\',
 337 | }
 338 | 
 339 | byte_escapes = {
 340 |     'b': b'\b',
 341 |     'n': b'\n',
 342 |     'f': b'\f',
 343 |     'r': b'\r',
 344 |     't': b'\t',
 345 |     '/': b'/',
 346 |     '"': b'"',
 347 |     "'": b"'",
 348 |     '\\': b'\\',
 349 | }
 350 | 
 351 | escaped = {
 352 |     '\b': '\\b',
 353 |     '\n': '\\n',
 354 |     '\f': '\\f',
 355 |     '\r': '\\r',
 356 |     '\t': '\\t',
 357 |     '"': '\\"',
 358 |     "'": "\\'",
 359 |     '\\': '\\\\',
 360 | }
 361 | 
 362 | builtin_names = {'null': None, 'true': True, 'false': False}
 363 | builtin_values = {None: 'null', True: 'true', False: 'false'}
 364 | 
 365 | # names -> Classes (take name, value as args)
 366 | def parse_datetime(v):
 367 |     if v[-1] == 'Z':
 368 |         if '.' in v:
 369 |             return datetime.strptime(v, "%Y-%m-%dT%H:%M:%S.%fZ").replace(tzinfo=timezone.utc)
 370 |         else:
 371 |             return datetime.strptime(v, "%Y-%m-%dT%H:%M:%SZ").replace(tzinfo=timezone.utc)
 372 |     else:
 373 |         raise NotImplementedError()
 374 | 
 375 | 
 376 | def format_datetime(obj):
 377 |     obj = obj.astimezone(timezone.utc)
 378 |     return obj.strftime("%Y-%m-%dT%H:%M:%S.%fZ")
 379 | 
 380 | class ParserErr(Exception):
 381 |     def __init__(self, buf, pos, reason=None):
 382 |         self.buf = buf
 383 |         self.pos = pos
 384 |         if reason is None:
 385 |             nl = buf.rfind(' ', pos - 10, pos)
 386 |             if nl < 0:
 387 |                 nl = pos - 5
 388 |             reason = "Unknown Character {} (context: {})".format(
 389 |                 repr(buf[pos]), repr(buf[pos - 10:pos + 5]))
 390 |         Exception.__init__(self, "{} (at pos={})".format(reason, pos))
 391 | 
 392 | 
 393 | class Codec:
 394 |     content_type = CONTENT_TYPE
 395 | 
 396 |     def __init__(self, object_to_tagged, tagged_to_object):
 397 |         self.object_to_tagged = object_to_tagged
 398 |         self.tagged_to_object = tagged_to_object
 399 | 
 400 |     def parse(self, buf, transform=None):
 401 |         obj, pos = self.parse_rson(buf, 0, transform)
 402 | 
 403 |         m = whitespace.match(buf, pos)
 404 |         if m:
 405 |             pos = m.end()
 406 |             m = whitespace.match(buf, pos)
 407 | 
 408 |         if pos != len(buf):
 409 |             raise ParserErr(buf, pos, "Trailing content: {}".format(
 410 |                 repr(buf[pos:pos + 10])))
 411 | 
 412 |         return obj
 413 | 
 414 | 
 415 |     def dump(self, obj, transform=None):
 416 |         buf = io.StringIO('')
 417 |         self.dump_rson(obj, buf, transform)
 418 |         buf.write('\n')
 419 |         return buf.getvalue()
 420 | 
 421 |     def parse_rson(self, buf, pos, transform=None):
 422 |         m = whitespace.match(buf, pos)
 423 |         if m:
 424 |             pos = m.end()
 425 | 
 426 |         peek = buf[pos]
 427 |         name = None
 428 |         if peek == '@':
 429 |             m = tag_name.match(buf, pos)
 430 |             if m:
 431 |                 pos = m.end()
 432 |                 name = buf[m.start() + 1:pos].rstrip()
 433 |             else:
 434 |                 raise ParserErr(buf, pos)
 435 | 
 436 |         peek = buf[pos]
 437 | 
 438 |         if peek == '@':
 439 |             raise ParserErr(buf, pos, "Cannot nest tags")
 440 | 
 441 |         elif peek == '{':
 442 |             if name in reserved_tags:
 443 |                 if name not in ('object', 'record', 'dict'):
 444 |                     raise ParserErr(
 445 |                         buf, pos, "{} can't be used on objects".format(name))
 446 | 
 447 |             if name == 'dict':
 448 |                 out = dict()
 449 |             else:
 450 |                 out = OrderedDict()
 451 | 
 452 |             pos += 1
 453 |             m = whitespace.match(buf, pos)
 454 |             if m:
 455 |                 pos = m.end()
 456 | 
 457 |             while buf[pos] != '}':
 458 |                 key, pos = self.parse_rson(buf, pos, transform)
 459 | 
 460 |                 if key in out:
 461 |                     raise SemanticErr('duplicate key: {}, {}'.format(key, out))
 462 | 
 463 |                 m = whitespace.match(buf, pos)
 464 |                 if m:
 465 |                     pos = m.end()
 466 | 
 467 |                 peek = buf[pos]
 468 |                 if peek == ':':
 469 |                     pos += 1
 470 |                     m = whitespace.match(buf, pos)
 471 |                     if m:
 472 |                         pos = m.end()
 473 |                 else:
 474 |                     raise ParserErr(
 475 |                         buf, pos, "Expected key:value pair but found {}".format(repr(peek)))
 476 | 
 477 |                 item, pos = self.parse_rson(buf, pos, transform)
 478 | 
 479 |                 out[key] = item
 480 | 
 481 |                 peek = buf[pos]
 482 |                 if peek == ',':
 483 |                     pos += 1
 484 |                     m = whitespace.match(buf, pos)
 485 |                     if m:
 486 |                         pos = m.end()
 487 |                 elif peek != '}':
 488 |                     raise ParserErr(
 489 |                         buf, pos, "Expecting a ',', or a '{}' but found {}".format('{}',repr(peek)))
 490 |             if name not in (None, 'object', 'record', 'dict'):
 491 |                 out = self.tagged_to_object(name,  out)
 492 |             if transform is not None:
 493 |                 out = transform(out)
 494 |             return out, pos + 1
 495 | 
 496 |         elif peek == '[':
 497 |             if name in reserved_tags:
 498 |                 if name not in ('object', 'list', 'set', 'complex'):
 499 |                     raise ParserErr(
 500 |                         buf, pos, "{} can't be used on lists".format(name))
 501 | 
 502 |             if name == 'set':
 503 |                 out = set()
 504 |             else:
 505 |                 out = []
 506 | 
 507 |             pos += 1
 508 | 
 509 |             m = whitespace.match(buf, pos)
 510 |             if m:
 511 |                 pos = m.end()
 512 | 
 513 |             while buf[pos] != ']':
 514 |                 item, pos = self.parse_rson(buf, pos, transform)
 515 |                 if name == 'set':
 516 |                     if item in out:
 517 |                         raise SemanticErr('duplicate item in set: {}'.format(item))
 518 |                     else:
 519 |                         out.add(item)
 520 |                 else:
 521 |                     out.append(item)
 522 | 
 523 |                 m = whitespace.match(buf, pos)
 524 |                 if m:
 525 |                     pos = m.end()
 526 | 
 527 |                 peek = buf[pos]
 528 |                 if peek == ',':
 529 |                     pos += 1
 530 |                     m = whitespace.match(buf, pos)
 531 |                     if m:
 532 |                         pos = m.end()
 533 |                 elif peek != ']':
 534 |                     raise ParserErr(
 535 |                         buf, pos, "Expecting a ',', or a ']' but found {}".format(repr(peek)))
 536 | 
 537 |             pos += 1
 538 | 
 539 |             if name in (None, 'object', 'list', 'set'):
 540 |                 pass
 541 |             elif name == 'complex':
 542 |                 out = complex(*out)
 543 |             else:
 544 |                 out = self.tagged_to_object(name,  out)
 545 | 
 546 |             if transform is not None:
 547 |                 out = transform(out)
 548 |             return out, pos
 549 | 
 550 |         elif peek == "'" or peek == '"':
 551 |             if name in reserved_tags:
 552 |                 if name not in ('object', 'string', 'float', 'datetime', 'bytestring', 'base64'):
 553 |                     raise ParserErr(
 554 |                         buf, pos, "{} can't be used on strings".format(name))
 555 | 
 556 |             if name == 'bytestring':
 557 |                 s = bytearray()
 558 |                 ascii = True
 559 |             else:
 560 |                 s = io.StringIO()
 561 |                 ascii = False
 562 | 
 563 |             # validate string
 564 |             if peek == "'":
 565 |                 m = string_sq.match(buf, pos)
 566 |                 if m:
 567 |                     end = m.end()
 568 |                 else:
 569 |                     raise ParserErr(buf, pos, "Invalid single quoted string")
 570 |             else:
 571 |                 m = string_dq.match(buf, pos)
 572 |                 if m:
 573 |                     end = m.end()
 574 |                 else:
 575 |                     raise ParserErr(buf, pos, "Invalid double quoted string")
 576 | 
 577 |             lo = pos + 1  # skip quotes
 578 |             while lo < end - 1:
 579 |                 hi = buf.find("\\", lo, end)
 580 |                 if hi == -1:
 581 |                     if ascii:
 582 |                         s.extend(buf[lo:end - 1].encode('ascii'))
 583 |                     else:
 584 |                         s.write(buf[lo:end - 1])  # skip quote
 585 |                     break
 586 | 
 587 |                 if ascii:
 588 |                     s.extend(buf[lo:hi].encode('ascii'))
 589 |                 else:
 590 |                     s.write(buf[lo:hi])
 591 | 
 592 |                 esc = buf[hi + 1]
 593 |                 if esc in str_escapes:
 594 |                     if ascii:
 595 |                         s.extend(byte_escapes[esc])
 596 |                     else:
 597 |                         s.write(str_escapes[esc])
 598 |                     lo = hi + 2
 599 |                 elif esc == 'x':
 600 |                     n = int(buf[hi + 2:hi + 4], 16)
 601 |                     if ascii:
 602 |                         s.append(n)
 603 |                     else:
 604 |                         s.write(chr(n))
 605 |                     lo = hi + 4
 606 |                 elif esc == 'u':
 607 |                     n = int(buf[hi + 2:hi + 6], 16)
 608 |                     if ascii:
 609 |                         if n > 0xFF:
 610 |                             raise ParserErr(
 611 |                                 buf, hi, 'bytestring cannot have escape > 255')
 612 |                         s.append(n)
 613 |                     else:
 614 |                         if 0xD800 <= n <= 0xDFFF:
 615 |                             raise ParserErr(
 616 |                                 buf, hi, 'string cannot have surrogate pairs')
 617 |                         s.write(chr(n))
 618 |                     lo = hi + 6
 619 |                 elif esc == 'U':
 620 |                     n = int(buf[hi + 2:hi + 10], 16)
 621 |                     if ascii:
 622 |                         if n > 0xFF:
 623 |                             raise ParserErr(
 624 |                                 buf, hi, 'bytestring cannot have escape > 255')
 625 |                         s.append(n)
 626 |                     else:
 627 |                         if 0xD800 <= n <= 0xDFFF:
 628 |                             raise ParserErr(
 629 |                                 buf, hi, 'string cannot have surrogate pairs')
 630 |                         s.write(chr(n))
 631 |                     lo = hi + 10
 632 |                 elif esc == '\n':
 633 |                     lo = hi + 2
 634 |                 elif (buf[hi + 1:hi + 3] == '\r\n'):
 635 |                     lo = hi + 3
 636 |                 else:
 637 |                     raise ParserErr(
 638 |                         buf, hi, "Unkown escape character {}".format(repr(esc)))
 639 | 
 640 |             if name == 'bytestring':
 641 |                 out = s
 642 |             else:
 643 |                 out = s.getvalue()
 644 | 
 645 |                 if name in (None, 'string', 'object'):
 646 |                     pass
 647 |                 elif name == 'base64':
 648 |                     try:
 649 |                         out = base64.standard_b64decode(out)
 650 |                     except Exception as e:
 651 |                         raise ParserErr(buf, pos, "Invalid base64") from e
 652 |                 elif name == 'datetime':
 653 |                     try:
 654 |                         out = parse_datetime(out)
 655 |                     except Exception as e:
 656 |                         raise ParserErr(
 657 |                             buf, pos, "Invalid datetime: {}".format(repr(out))) from e
 658 |                 elif name == 'float':
 659 |                     m = c99_flt.match(out)
 660 |                     if m:
 661 |                         out = float.fromhex(out)
 662 |                     else:
 663 |                         raise ParserErr(
 664 |                             buf, pos, "invalid C99 float literal: {}".format(out))
 665 |                 else:
 666 |                     out = self.tagged_to_object(name,  out)
 667 | 
 668 |             if transform is not None:
 669 |                 out = transform(out)
 670 |             return out, end
 671 | 
 672 |         elif peek in "-+0123456789":
 673 |             if name in reserved_tags:
 674 |                 if name not in ('object', 'int', 'float', 'duration'):
 675 |                     raise ParserErr(
 676 |                         buf, pos, "{} can't be used on numbers".format(name))
 677 | 
 678 |             flt_end = None
 679 |             exp_end = None
 680 | 
 681 |             sign = +1
 682 | 
 683 |             if buf[pos] in "+-":
 684 |                 if buf[pos] == "-":
 685 |                     sign = -1
 686 |                 pos += 1
 687 |             peek = buf[pos:pos + 2]
 688 | 
 689 |             if peek in ('0x', '0o', '0b'):
 690 |                 if peek == '0x':
 691 |                     base = 16
 692 |                     m = int_b16.match(buf, pos)
 693 |                     if m:
 694 |                         end = m.end()
 695 |                     else:
 696 |                         raise ParserErr(
 697 |                             buf, pos, "Invalid hexadecimal number (0x...)")
 698 |                 elif peek == '0o':
 699 |                     base = 8
 700 |                     m = int_b8.match(buf, pos)
 701 |                     if m:
 702 |                         end = m.end()
 703 |                     else:
 704 |                         raise ParserErr(buf, pos, "Invalid octal number (0o...)")
 705 |                 elif peek == '0b':
 706 |                     base = 2
 707 |                     m = int_b2.match(buf, pos)
 708 |                     if m:
 709 |                         end = m.end()
 710 |                     else:
 711 |                         raise ParserErr(
 712 |                             buf, pos, "Invalid hexadecimal number (0x...)")
 713 | 
 714 |                 out = sign * int(buf[pos + 2:end].replace('_', ''), base)
 715 |             else:
 716 |                 m = int_b10.match(buf, pos)
 717 |                 if m:
 718 |                     int_end = m.end()
 719 |                     end = int_end
 720 |                 else:
 721 |                     raise ParserErr(buf, pos, "Invalid number")
 722 | 
 723 |                 t = flt_b10.match(buf, end)
 724 |                 if t:
 725 |                     flt_end = t.end()
 726 |                     end = flt_end
 727 | 
 728 |                 e = exp_b10.match(buf, end)
 729 |                 if e:
 730 |                     exp_end = e.end()
 731 |                     end = exp_end
 732 | 
 733 |                 if flt_end or exp_end:
 734 |                     out = sign * float(buf[pos:end].replace('_', ''))
 735 |                 else:
 736 |                     out = sign * int(buf[pos:end].replace('_', ''), 10)
 737 | 
 738 |             if name is None or name == 'object':
 739 |                 pass
 740 |             elif name == 'duration':
 741 |                 out = timedelta(seconds=out)
 742 |             elif name == 'int':
 743 |                 if flt_end or exp_end:
 744 |                     raise ParserErr(
 745 |                         buf, pos, "Can't tag floating point with @int")
 746 |             elif name == 'float':
 747 |                 if not isintance(out, float):
 748 |                     out = float(out)
 749 |             else:
 750 |                 out = self.tagged_to_object(name, out)
 751 | 
 752 |             if transform is not None:
 753 |                 out = transform(out)
 754 |             return out, end
 755 | 
 756 |         else:
 757 |             m = identifier.match(buf, pos)
 758 |             if m:
 759 |                 end = m.end()
 760 |                 item = buf[pos:end]
 761 |             else:
 762 |                 raise ParserErr(buf, pos)
 763 | 
 764 |             if item not in builtin_names:
 765 |                 raise ParserErr(
 766 |                     buf, pos, "{} is not a recognised built-in".format(repr(item)))
 767 | 
 768 |             out = builtin_names[item]
 769 | 
 770 |             if name is None or name == 'object':
 771 |                 pass
 772 |             elif name == 'bool':
 773 |                 if item not in ('true', 'false'):
 774 |                     raise ParserErr(buf, pos, '@bool can only true or false')
 775 |             elif name in reserved_tags:
 776 |                 raise ParserErr(
 777 |                     buf, pos, "{} has no meaning for {}".format(repr(name), item))
 778 |             else:
 779 |                 out = self.tagged_to_object(name,  out)
 780 | 
 781 |             if transform is not None:
 782 |                 out = transform(out)
 783 |             return out, end
 784 | 
 785 |         raise ParserErr(buf, pos)
 786 | 
 787 | 
 788 | 
 789 |     def dump_rson(self, obj, buf, transform=None):
 790 |         if transform:
 791 |             obj = transform(obj)
 792 |         if obj is True or obj is False or obj is None:
 793 |             buf.write(builtin_values[obj])
 794 |         elif isinstance(obj, str):
 795 |             buf.write('"')
 796 |             for c in obj:
 797 |                 if c in escaped:
 798 |                     buf.write(escaped[c])
 799 |                 elif ord(c) < 0x20:
 800 |                     buf.write('\\x{:02X}'.format(ord(c)))
 801 |                 else:
 802 |                     buf.write(c)
 803 |             buf.write('"')
 804 |         elif isinstance(obj, int):
 805 |             buf.write(str(obj))
 806 |         elif isinstance(obj, float):
 807 |             hex = obj.hex()
 808 |             if hex.startswith(('0', '-')):
 809 |                 buf.write(str(obj))
 810 |             else:
 811 |                 buf.write('@float "{}"'.format(hex))
 812 |         elif isinstance(obj, complex):
 813 |             buf.write("@complex [{}, {}]".format(obj.real, obj.imag))
 814 |         elif isinstance(obj, (bytes, bytearray)):
 815 |             buf.write('@base64 "')
 816 |             # assume no escaping needed
 817 |             buf.write(base64.standard_b64encode(obj).decode('ascii'))
 818 |             buf.write('"')
 819 |         elif isinstance(obj, (list, tuple)):
 820 |             buf.write('[')
 821 |             first = True
 822 |             for x in obj:
 823 |                 if first:
 824 |                     first = False
 825 |                 else:
 826 |                     buf.write(", ")
 827 |                 self.dump_rson(x, buf, transform)
 828 |             buf.write(']')
 829 |         elif isinstance(obj, set):
 830 |             buf.write('@set [')
 831 |             first = True
 832 |             for x in obj:
 833 |                 if first:
 834 |                     first = False
 835 |                 else:
 836 |                     buf.write(", ")
 837 |                 self.dump_rson(x, buf, transform)
 838 |             buf.write(']')
 839 |         elif isinstance(obj, OrderedDict): # must be before dict
 840 |             buf.write('{')
 841 |             first = True
 842 |             for k, v in obj.items():
 843 |                 if first:
 844 |                     first = False
 845 |                 else:
 846 |                     buf.write(", ")
 847 |                 self.dump_rson(k, buf, transform)
 848 |                 buf.write(": ")
 849 |                 self.dump_rson(v, buf, transform)
 850 |             buf.write('}')
 851 |         elif isinstance(obj, dict):
 852 |             buf.write('@dict {')
 853 |             first = True
 854 |             for k in sorted(obj.keys()):
 855 |                 if first:
 856 |                     first = False
 857 |                 else:
 858 |                     buf.write(", ")
 859 |                 self.dump_rson(k, buf, transform)
 860 |                 buf.write(": ")
 861 |                 self.dump_rson(obj[k], buf, transform)
 862 |             buf.write('}')
 863 |         elif isinstance(obj, datetime):
 864 |             buf.write('@datetime "{}"'.format(format_datetime(obj)))
 865 |         elif isinstance(obj, timedelta):
 866 |             buf.write('@duration {}'.format(obj.total_seconds()))
 867 |         else:
 868 |             nv = self.object_to_tagged(obj)
 869 |             name, value = nv
 870 |             if not isinstance(value, OrderedDict) and isinstance(value, dict):
 871 |                 value = OrderedDict(value)
 872 |             buf.write('@{} '.format(name))
 873 |             self.dump_rson(value, buf, transform)  # XXX: prevent @foo @foo
 874 |         
 875 | 
 876 | 
 877 | class BinaryCodec:
 878 |     """
 879 |         just enough of a type-length-value scheme to be dangerous
 880 | 
 881 |     """
 882 |     TRUE = ord("y")
 883 |     FALSE = ord("n")
 884 |     NULL = ord("z")
 885 |     INT = ord("i")
 886 |     FLOAT = ord("f")
 887 |     STRING = ord("u")
 888 |     BYTES = ord("b")
 889 |     LIST = ord("l")
 890 |     RECORD = ord("r")
 891 |     TAG = ord("t")
 892 |     END = 127
 893 | 
 894 |     def __init__(self, object_to_tagged, tagged_to_object):
 895 |         self.tags = object_to_tagged
 896 |         self.classes = tagged_to_object
 897 | 
 898 |     def parse(self, buf):
 899 |         obj, offset = self.parse_buf(buf, 0)
 900 |         return obj
 901 | 
 902 |     def dump(self, obj):
 903 |         return self.dump_buf(obj, bytearray())
 904 | 
 905 |     def parse_buf(self, buf, offset=0):
 906 |         peek = buf[offset]
 907 |         if peek == self.TRUE:
 908 |             return True, offset+1
 909 |         elif peek == self.FALSE:
 910 |             return False, offset+1
 911 |         elif peek == self.NULL:
 912 |             return None, offset+1
 913 |         elif peek == self.INT:
 914 |             end = buf.index(self.END, offset+1)
 915 |             obj = buf[offset+1:end].decode('ascii')
 916 |             return int(obj), end+1
 917 |         elif peek == self.FLOAT:
 918 |             end = buf.index(self.END, offset+1)
 919 |             obj = buf[offset+1:end].decode('ascii')
 920 |             return float.fromhex(obj), end+1
 921 |         elif peek == self.BYTES:
 922 |             size, end = self.parse_buf(buf, offset+1)
 923 |             start, end = end, end+size
 924 |             obj = buf[start:end]
 925 |             end = buf.index(self.END, end)
 926 |             return obj, end+1
 927 |         elif peek == self.STRING:
 928 |             size, end = self.parse_buf(buf, offset+1)
 929 |             start, end = end, end+size
 930 |             obj = buf[start:end].decode('utf-8')
 931 |             end = buf.index(self.END, end)
 932 |             return obj, end+1
 933 |         elif peek == self.LIST:
 934 |             size, start = self.parse_buf(buf, offset+1)
 935 |             out = []
 936 |             for _ in range(size):
 937 |                 value, start = self.parse_buf(buf, start)
 938 |                 out.append(value)
 939 |             end = buf.index(self.END, start)
 940 |             return out, end+1
 941 |         elif peek == self.RECORD:
 942 |             size, start = self.parse_buf(buf, offset+1)
 943 |             out = {}
 944 |             for _ in range(size):
 945 |                 key, start = self.parse_buf(buf, start)
 946 |                 value, start = self.parse_buf(buf, start)
 947 |                 out[key] = value
 948 | 
 949 |             end = buf.index(self.END, start)
 950 |             return out, end+1
 951 |         elif peek == self.TAG:
 952 |             tag, start = self.parse_buf(buf, offset+1)
 953 |             value, start = self.parse_buf(buf, start)
 954 |             end = buf.index(self.END, start)
 955 |             if tag == 'set':
 956 |                 out = set(value)
 957 |             elif tag == 'complex':
 958 |                 out = complex(*value)
 959 |             elif tag == 'datetime':
 960 |                 out = parse_datetime(value)
 961 |             elif tag == 'duration':
 962 |                 out = timedelta(seconds=value)
 963 |             else:
 964 |                 cls = self.classes[tag]
 965 |                 out = cls(**value)
 966 |             return out, end+1
 967 | 
 968 | 
 969 |         raise Exception('bad buf {}'.format(peek.encode('ascii')))
 970 | 
 971 | 
 972 |     def dump_buf(self, obj, buf):
 973 |         if obj is True:
 974 |             buf.append(self.TRUE)
 975 |         elif obj is False:
 976 |             buf.append(self.FALSE)
 977 |         elif obj is None:
 978 |             buf.append(self.NULL)
 979 |         elif isinstance(obj, int):
 980 |             buf.append(self.INT)
 981 |             buf.extend(str(obj).encode('ascii'))
 982 |             buf.append(self.END)
 983 |         elif isinstance(obj, float):
 984 |             buf.append(self.FLOAT)
 985 |             buf.extend(float.hex(obj).encode('ascii'))
 986 |             buf.append(self.END)
 987 |         elif isinstance(obj, (bytes,bytearray)):
 988 |             buf.append(self.BYTES)
 989 |             self.dump_buf(len(obj), buf)
 990 |             buf.extend(obj)
 991 |             buf.append(self.END)
 992 |         elif isinstance(obj, (str)):
 993 |             obj = obj.encode('utf-8')
 994 |             buf.append(self.STRING)
 995 |             self.dump_buf(len(obj), buf)
 996 |             buf.extend(obj)
 997 |             buf.append(self.END)
 998 |         elif isinstance(obj, (list, tuple)):
 999 |             buf.append(self.LIST)
1000 |             self.dump_buf(len(obj), buf)
1001 |             for x in obj:
1002 |                 self.dump_buf(x, buf)
1003 |             buf.append(self.END)
1004 |         elif isinstance(obj, (dict)):
1005 |             buf.append(self.RECORD)
1006 |             self.dump_buf(len(obj), buf)
1007 |             for k,v in obj.items():
1008 |                 self.dump_buf(k, buf)
1009 |                 self.dump_buf(v, buf)
1010 |             buf.append(self.END)
1011 |         elif isinstance(obj, (set)):
1012 |             buf.append(self.TAG)
1013 |             self.dump_buf("set", buf)
1014 |             buf.append(self.LIST)
1015 |             self.dump_buf(len(obj), buf)
1016 |             for x in obj:
1017 |                 self.dump_buf(x, buf)
1018 |             buf.append(self.END)
1019 |             buf.append(self.END)
1020 |         elif isinstance(obj, complex):
1021 |             buf.append(self.TAG)
1022 |             self.dump_buf("complex", buf)
1023 |             buf.append(self.LIST)
1024 |             self.dump_buf(2, buf)
1025 |             self.dump_buf(obj.real, buf)
1026 |             self.dump_buf(obj.imag, buf)
1027 |             buf.append(self.END)
1028 |             buf.append(self.END)
1029 |         elif isinstance(obj, datetime):
1030 |             buf.append(self.TAG)
1031 |             self.dump_buf("datetime", buf)
1032 |             self.dump_buf(format_datetime(obj), buf)
1033 |             buf.append(self.END)
1034 |         elif isinstance(obj, timedelta):
1035 |             buf.append(self.TAG)
1036 |             self.dump_buf("duration", buf)
1037 |             self.dump_buf(obj.total_seconds(), buf)
1038 |             buf.append(self.END)
1039 | 
1040 |         elif obj.__class__ in self.tags:
1041 |             tag = self.tags[obj.__class__].encode('ascii')
1042 |             buf.append(self.TAG)
1043 |             self.dump_buf(tag, buf)
1044 |             self.dump_buf(obj.__dict__, buf)
1045 |             buf.append(self.END)
1046 |         else:
1047 |             raise Exception('bad obj {!r}'.format(obj))
1048 |         return buf
1049 | 
1050 | 
1051 | if __name__ == '__main__':
1052 |     codec = Codec(None, None)
1053 |     bcodec = BinaryCodec({},{})
1054 | 
1055 |     parse = codec.parse
1056 |     dump = codec.dump
1057 | 
1058 |     bparse = bcodec.parse
1059 |     bdump = bcodec.dump
1060 | 
1061 |     def test_parse(buf, obj):
1062 |         out = parse(buf)
1063 | 
1064 |         if (obj != obj and out == out) or (obj == obj and obj != out):
1065 |             raise AssertionError('{} != {}'.format(obj, out))
1066 | 
1067 |     def test_dump(obj, buf):
1068 |         out = dump(obj)
1069 |         if buf != out:
1070 |             raise AssertionError('{} != {}'.format(buf, out))
1071 | 
1072 |     def test_parse_err(buf, exc):
1073 |         try:
1074 |             obj = parse(buf)
1075 |         except Exception as e:
1076 |             if isinstance(e, exc):
1077 |                 return
1078 |             else:
1079 |                 raise AssertionError(
1080 |                     '{} did not cause {}, but {}'.format(buf, exc, e)) from e
1081 |         else:
1082 |             raise AssertionError(
1083 |                 '{} did not cause {}, parsed:{}'.format(buf, exc, obj))
1084 | 
1085 | 
1086 |     def test_dump_err(obj, exc):
1087 |         try:
1088 |             buf = dump(obj)
1089 |         except Exception as e:
1090 |             if isinstance(e, exc):
1091 |                 return
1092 |             else:
1093 |                 raise AssertionError(
1094 |                     '{} did not cause {}, but '.format(obj, exc, e))
1095 |         else:
1096 |             raise AssertionError(
1097 |                 '{} did not cause {}, dumping: {}'.format(obj, exc, buf))
1098 | 
1099 | 
1100 |     test_parse("0", 0)
1101 |     test_parse("0x0_1_2_3", 0x123)
1102 |     test_parse("0o0_1_2_3", 0o123)
1103 |     test_parse("0b0_1_0_1", 5)
1104 |     test_parse("0 #comment", 0)
1105 |     test_parse("""
1106 | "a\\
1107 | b"        
1108 |     """, "ab")
1109 |     test_parse("0.0", 0.0)
1110 |     test_parse("-0.0", -0.0)
1111 |     test_parse("'foo'", "foo")
1112 |     test_parse(r"'fo\no'", "fo\no")
1113 |     test_parse("'\\\\'", "\\")
1114 |     test_parse(r"'\b\f\r\n\t\"\'\/'", "\b\f\r\n\t\"\'/")
1115 |     test_parse("''", "")
1116 |     test_parse(r'"\x20"', " ")
1117 |     test_parse(r'"\uF0F0"', "\uF0F0")
1118 |     test_parse(r'"\U0001F0F0"', "\U0001F0F0")
1119 |     test_parse("'\\\\'", "\\")
1120 |     test_parse("[1]", [1])
1121 |     test_parse("[1,]", [1])
1122 |     test_parse("[]", [])
1123 |     test_parse("[1 , 2 , 3 , 4 , 4 ]", [1, 2, 3, 4, 4])
1124 |     test_parse("{'a':1,'b':2}", dict(a=1, b=2))
1125 |     test_parse("@set [1,2,3,4]", set([1, 2, 3, 4]))
1126 |     test_parse("{'a':1,'b':2}", dict(a=1, b=2))
1127 |     test_parse("@complex [1,2]", 1 + 2j)
1128 |     test_parse("@bytestring 'foo'", b"foo")
1129 |     test_parse("@base64 '{}'".format(
1130 |         base64.standard_b64encode(b'foo').decode('ascii')), b"foo")
1131 |     test_parse("@float 'NaN'", float('NaN'))
1132 |     test_parse("@float '-inf'", float('-Inf'))
1133 |     obj = datetime.now().astimezone(timezone.utc)
1134 |     test_parse('@datetime "{}"'.format(
1135 |         obj.strftime("%Y-%m-%dT%H:%M:%S.%fZ")), obj)
1136 |     obj = timedelta(seconds=666)
1137 |     test_parse('@duration {}'.format(obj.total_seconds()), obj)
1138 |     test_parse("@bytestring 'fo\x20o'", b"fo o")
1139 |     test_parse("@float '{}'".format((3000000.0).hex()), 3000000.0)
1140 |     test_parse(hex(123), 123)
1141 |     test_parse('@object "foo"', "foo")
1142 |     test_parse('@object 12', 12)
1143 | 
1144 |     test_dump(1, "1")
1145 | 
1146 |     test_parse_err('"foo', ParserErr)
1147 |     test_parse_err('"\uD800\uDD01"', ParserErr)
1148 |     test_parse_err(r'"\uD800\uDD01"', ParserErr)
1149 | 
1150 |     tests = [
1151 |         0, -1, +1,
1152 |         -0.0, +0.0, 1.9,
1153 |         True, False, None,
1154 |         "str", b"bytes",
1155 |         [1, 2, 3], {"c": 3, "a": 1, "b": 2, }, set(
1156 |             [1, 2, 3]), OrderedDict(a=1, b=2),
1157 |         1 + 2j, float('NaN'),
1158 |         datetime.now().astimezone(timezone.utc),
1159 |         timedelta(seconds=666),
1160 |     ]
1161 | 
1162 |     for obj in tests:
1163 |         buf0 = dump(obj)
1164 |         obj1 = parse(buf0)
1165 |         buf1 = dump(obj1)
1166 | 
1167 |         out = parse(buf1)
1168 | 
1169 |         if obj != obj:
1170 |             if buf0 != buf1 or obj1 == obj1 or out == out:
1171 |                 raise AssertionError('{} != {}'.format(obj, out))
1172 |         else:
1173 |             if buf0 != buf1:
1174 |                 raise AssertionError(
1175 |                     'mismatched output {} != {}'.format(buf0, buf1))
1176 |             if obj != obj1:
1177 |                 raise AssertionError(
1178 |                     'failed first trip {} != {}'.format(obj, obj1))
1179 |             if obj != out:
1180 |                 raise AssertionError(
1181 |                     'failed second trip {} != {}'.format(obj, out))
1182 | 
1183 |     for obj in tests:
1184 |         buf0 = bdump(obj)
1185 |         obj1 = bparse(buf0)
1186 |         buf1 = bdump(obj1)
1187 | 
1188 |         out = bparse(buf1)
1189 | 
1190 |         if obj != obj:
1191 |             if buf0 != buf1 or obj1 == obj1 or out == out:
1192 |                 raise AssertionError('{} != {}'.format(obj, out))
1193 |         else:
1194 |             if buf0 != buf1:
1195 |                 raise AssertionError(
1196 |                     'mismatched output {} != {}'.format(buf0, buf1))
1197 |             if obj != obj1:
1198 |                 raise AssertionError(
1199 |                     'failed first trip {} != {}'.format(obj, obj1))
1200 |             if obj != out:
1201 |                 raise AssertionError(
1202 |                     'failed second trip {} != {}'.format(obj, out))
1203 |     print('tests passed')
1204 | 
1205 | 
1206 | 


--------------------------------------------------------------------------------