├── .github └── workflows │ └── go.yml ├── .gitignore ├── CONTRIBUTORS ├── LICENSE ├── Makefile ├── README.md ├── go.mod ├── go.sum ├── static ├── fsa.jpg └── fsa.tex ├── ted.go ├── ted ├── ast │ ├── ast.go │ └── expression.go ├── flags │ └── flags.go ├── lexer │ ├── lexer.go │ └── lexer_test.go ├── parser │ └── parser.go ├── runner │ ├── runner.go │ └── tape.go └── token │ └── tokens.go └── tests ├── defaults ├── capture.fsa ├── capture.in ├── capture.out ├── capture_variables.fsa ├── capture_variables.in ├── capture_variables.out ├── dountil.fsa ├── dountil.in ├── dountil.out ├── empty_line.fsa ├── empty_line.in ├── empty_line.out ├── example_1.fsa ├── example_1.in ├── example_1.out ├── example_3.fsa ├── example_3.in ├── example_3.out ├── fastforward.fsa ├── fastforward.in ├── fastforward.out ├── flags ├── rewind.fsa ├── rewind.in └── rewind.out ├── noprint ├── begin_and_end.fsa ├── begin_and_end.in ├── begin_and_end.out ├── capture.fsa ├── capture.in ├── capture.out ├── capture_groups.fsa ├── capture_groups.in ├── capture_groups.out ├── example_2.fsa ├── example_2.in ├── example_2.out ├── expressions.fsa ├── expressions.in ├── expressions.out ├── flags ├── motivation.fsa ├── motivation.in ├── motivation.out ├── motivation_all.fsa ├── motivation_all.in ├── motivation_all.out ├── motivation_reset.fsa ├── motivation_reset.in └── motivation_reset.out ├── test.zsh └── variables ├── flags ├── variables_basic.fsa ├── variables_basic.in └── variables_basic.out /.github/workflows/go.yml: -------------------------------------------------------------------------------- 1 | # This workflow will test a golang project 2 | # For more information see: https://docs.github.com/en/actions/automating-builds-and-tests/building-and-testing-go 3 | 4 | name: Go 5 | 6 | on: 7 | push: 8 | branches: [ "main" ] 9 | pull_request: 10 | branches: [ "main" ] 11 | 12 | jobs: 13 | 14 | build: 15 | runs-on: ubuntu-latest 16 | steps: 17 | - uses: actions/checkout@v4 18 | name: Checkout Code 19 | 20 | - name: Set up Go 21 | uses: actions/setup-go@v5 22 | with: 23 | go-version: '1.22' 24 | 25 | - name: Install dependencies 26 | run: go mod vendor 27 | 28 | - name: Run Go Tests 29 | run: go test -v -race ./... 30 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | vendor 2 | bin 3 | .vimjectrc 4 | .vimspector.json 5 | program.fsa 6 | test 7 | -------------------------------------------------------------------------------- /CONTRIBUTORS: -------------------------------------------------------------------------------- 1 | @WheeskyJack, 2 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | Copyright 2024 Armand Halbert 2 | 3 | Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: 4 | 5 | 1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. 6 | 7 | 2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. 8 | 9 | 3. Neither the name of the copyright holder nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission. 10 | 11 | THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS “AS IS” AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 12 | -------------------------------------------------------------------------------- /Makefile: -------------------------------------------------------------------------------- 1 | 2 | build: 3 | go build -o bin/ted ted.go 4 | install: 5 | go install ted.go 6 | clean: 7 | rm -rf bin 8 | rm -rf program.fsa 9 | rm -rf test 10 | test: build 11 | ./tests/test.zsh 12 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 |
2 | 3 | KeyTik Icon 4 | 5 | 6 | 7 |

ted: Turing EDitor

8 | A tool for editing files according to the rules of a provided Turing Machine 9 | 10 |
11 | 12 | * [Demo](#demo) 13 | * [Motivation](#motivation) 14 | * [Installing](#installing) 15 | * [Examples](#examples) 16 | * [Flags](#flags) 17 | * [Syntax](#syntax) 18 | * [Contact](#contact) 19 | 20 | ## Demo 21 | 22 | ted now has a live demo! [Try it out here](https://www.ahalbert.com/projects/ted/ted.html). 23 | 24 | ## Motivation 25 | 26 | Once, I was presented with an the following file (abridged) 27 | 28 | ``` 29 | INFO:2024-12-07 13:01:40:Trace:198d079c-af9a-45b2-8236-7fbb2a012f69:Starting Procedure foo 30 | ERROR:2024-12-07 13:01:41:Trace:198d079c-af9a-45b2-8236-7fbb2a012f69:Error 1 31 | INFO:2024-12-07 13:01:41:Trace:198d079c-af9a-45b2-8236-7fbb2a012f69:Ending Procedure foo 32 | INFO:2024-12-07 13:01:41:Trace:198d079c-af9a-45b2-8236-7fbb2a012f69:Starting Procedure bar 33 | INFO:2024-12-07 13:01:41:Trace:198d079c-af9a-45b2-8236-7fbb2a012f69:Error 2 34 | INFO:2024-12-07 13:01:41:Trace:198d079c-af9a-45b2-8236-7fbb2a012f69:Success 35 | INFO:2024-12-07 13:01:42:Trace:198d079c-af9a-45b2-8236-7fbb2a012f69:Ending Procedure bar 36 | INFO:2024-12-07 13:01:42:Trace:30019fff-7645-4d07-9fc4-0bbb39aa09db:Starting Procedure foo 37 | INFO:2024-12-07 13:01:42:Trace:30019fff-7645-4d07-9fc4-0bbb39aa09db:Success 38 | INFO:2024-12-07 13:01:42:Trace:30019fff-7645-4d07-9fc4-0bbb39aa09db:Ending Procedure foo 39 | INFO:2024-12-07 13:01:43:Trace:30019fff-7645-4d07-9fc4-0bbb39aa09db:Starting Procedure bar 40 | ERROR:2024-12-07 13:01:43:Trace:30019fff-7645-4d07-9fc4-0bbb39aa09db:Error 3 41 | ERROR:2024-12-07 13:01:43:Trace:30019fff-7645-4d07-9fc4-0bbb39aa09db:Error 4 42 | INFO:2024-12-07 13:01:44:Trace:30019fff-7645-4d07-9fc4-0bbb39aa09db:Ending Procedure bar 43 | 44 | ``` 45 | 46 | I wanted only the errors that did not have a success in the procedure. In this case, we should only get Errors 1,3,4 47 | 48 | ``` 49 | ERROR:2024-12-07 13:01:41:Trace:198d079c-af9a-45b2-8236-7fbb2a012f69:Error 1 50 | ERROR:2024-12-07 13:01:43:Trace:30019fff-7645-4d07-9fc4-0bbb39aa09db:Error 3 51 | ERROR:2024-12-07 13:01:43:Trace:30019fff-7645-4d07-9fc4-0bbb39aa09db:Error 4 52 | ``` 53 | 54 | 55 | I created an `awk` program to keep track of things and get the correct output. But I thought it was easier to express what I wanted as a state machine. Thus was born `ted`, a language for specifying state machines and using them to process files. 56 | 57 | An equivalent `ted` program: 58 | 59 | ``` 60 | startstate: /Starting.Procedure/ -> capture_begin 61 | capture_begin: { 62 | start capture 63 | -> lookforsuccessorending 64 | /Success/ -> startstate 65 | } 66 | lookforsuccessorending: /Success/ -> startstate 67 | lookforsuccessorending: /Ending.Procedure/ { 68 | stop capture 69 | print 70 | -> startstate 71 | } 72 | 73 | ``` 74 | 75 | # Installing 76 | 77 | Requires go 1.22 78 | 79 | ``` 80 | git clone git@github.com:ahalbert/ted.git 81 | cd ted 82 | go install 83 | ``` 84 | 85 | You can build the code using 86 | 87 | ``` 88 | make build 89 | make test 90 | ``` 91 | 92 | ## Examples 93 | 94 | ### Run sed only after seeing multiple patterns 95 | 96 | Given the input: 97 | 98 | ``` 99 | baz 100 | foo 101 | baz 102 | bar 103 | baz 104 | ``` 105 | 106 | And you only want to edit the final `baz` into `bang`, use this command: 107 | 108 | ``` 109 | $ echo "baz\nfoo\nbaz\nbar\nbaz" | ted '/foo/ -> /bar/ -> do s/baz/bang/' 110 | ``` 111 | 112 | Results in: 113 | 114 | ``` 115 | baz 116 | foo 117 | baz 118 | bar 119 | bang 120 | ``` 121 | 122 | ### Print Lines Between /regex/ 123 | 124 | Given the input: 125 | 126 | ``` 127 | DO NOT PRINT THIS LINE 128 | baz - DO NOT PRINT THIS EITHER 129 | foo 130 | bar 131 | baz - DO NOT PRINT THIS EITHER 132 | DO NOT PRINT THIS LINE 133 | ``` 134 | 135 | And you only want to print what's between the `baz`s 136 | 137 | ``` 138 | $ ted -n 'stop:/baz/ -> start start:/baz/ -> 1 start: print' < file.txt 139 | ``` 140 | 141 | Results in: 142 | 143 | ``` 144 | foo 145 | bar 146 | ``` 147 | 148 | ### Run sed if /regexs/ are seen, but reset if /badregex/ is seen 149 | 150 | Given the input: 151 | 152 | ``` 153 | beep 154 | boop 155 | buzz 156 | cheater 157 | beep 158 | boop 159 | cheater 160 | ``` 161 | 162 | And you want to modify `cheater` to `nose` only if you see a beep and buzz, but if there's a `buzz`, start looking for `/beep/` again 163 | 164 | ``` 165 | $ ted '/beep/ -> {/boop/ -> /buzz/ -> 1} {do s/cheater/nose/ /buzz/ -> 1}' < file.txt 166 | ``` 167 | 168 | Results In: 169 | 170 | ``` 171 | beep 172 | boop 173 | buzz 174 | cheater 175 | beep 176 | boop 177 | nose 178 | ``` 179 | 180 | ### Capturing 181 | 182 | *Capturing* enables you to read input into a variable rather than printing it on the screen. 183 | 184 | #### Capture single line 185 | 186 | Given the input: 187 | 188 | ``` 189 | beep 190 | boop 191 | foo 192 | bar 193 | baz 194 | buzz 195 | ``` 196 | 197 | You can capture one line as so: 198 | 199 | ``` 200 | 1: /beep/ -> 201 | 2: {capture mycapture -> } 202 | 3: do s/THIS.IS.CAPTURED/CAPTURED/ mycapture 203 | 3: /buzz/ -> 204 | 4: print mycapture 205 | 206 | ``` 207 | 208 | This program removes the `boop`, captured into the variable `$_`: 209 | 210 | ``` 211 | beep 212 | boop 213 | foo 214 | buzz 215 | CAPTURED 216 | bar 217 | CAPTURED 218 | baz 219 | ``` 220 | 221 | #### Capture multiple lines 222 | 223 | Given the input: 224 | 225 | ``` 226 | beep 227 | boop - CAPTURED 228 | foo - CAPTURED 229 | bar - CAPTURED 230 | baz 231 | buzz 232 | ``` 233 | 234 | And running this `ted` program with `--no-print` option: 235 | 236 | ``` 237 | /beep/ -> 238 | /boop/ {start capture ->} 239 | /baz/ {stop capture print -> 1} 240 | ``` 241 | 242 | 243 | Yields: 244 | 245 | ``` 246 | boop - CAPTURED 247 | foo - CAPTURED 248 | bar - CAPTURED 249 | ``` 250 | 251 | ### Regex Capture Groups 252 | 253 | You can store capture groups in a variable and refer to them later. 254 | 255 | Given the input: 256 | 257 | ``` 258 | beep 259 | boop 260 | i want these and those 261 | foo 262 | bar 263 | baz 264 | buzz 265 | ``` 266 | 267 | And this program with the `--no-print` option: 268 | 269 | ``` 270 | /i.*want.(these).and.(those)/ -> 271 | {println $1 println $2 ->} 272 | ``` 273 | 274 | Yields: 275 | 276 | ``` 277 | these 278 | those 279 | ``` 280 | 281 | ### Rewind and Fast-Forward 282 | 283 | You can rewind or fast-forward the input file to any point matching `/regex/` 284 | 285 | Given the file: 286 | 287 | ``` 288 | beep 289 | boop 290 | buzz 291 | foo 292 | bar 293 | baz 294 | ``` 295 | 296 | #### Print everything after regex 297 | 298 | ``` 299 | { capture fastforward /buzz/ -> } 300 | 301 | ``` 302 | 303 | #### Print a file twice 304 | 305 | ``` 306 | /baz/ { rewind /beep/ -> } 307 | ``` 308 | 309 | ## Flags 310 | 311 | ``` 312 | Usage: ted [--fsa-file FSAFILE] [--no-print] [--debug] [--var key=value] [PROGRAM [INPUTFILE [INPUTFILE ...]]] 313 | 314 | Positional arguments: 315 | PROGRAM Program to run. 316 | INPUTFILE File to use as input. 317 | 318 | Options: 319 | --fsa-file FSAFILE, -f FSAFILE 320 | Finite State Autonoma file to run. 321 | --no-print, -n Do not print lines by default. 322 | --debug Provides Lexer and Parser information. 323 | --var key=value Variable in the format name=value. 324 | --help, -h display this help and exit 325 | ``` 326 | 327 | ## Syntax 328 | 329 | ted consists of *states*, which contain *actions*. During each execution, `ted` will: 330 | 331 | 1. Read a line from the input. 332 | 2. Execute each action for that state in the order parsed 333 | 3. If an action requires it to move state, stops executing actions and moves to the next line. 334 | 4. Prints a line unless `--no-print` or capturing is on. 335 | 336 | 337 | ### Statement 338 | 339 | ``` 340 | [:] Action 341 | ``` 342 | 343 | Binds the Action to the state `statename`. If a state is not specified, it is an *Anonymous State*, and assigned a name from 1..N, incrementing each time a new state is created. Multiple actions in a statement can be combined using `{ }`. If you want to specify multiple different rules for the same state, use `,` 344 | 345 | 346 | ### Action 347 | 348 | Various actions can be specified in a state: 349 | 350 | 351 | #### Assign Variable 352 | 353 | `let variable = expression` 354 | 355 | Assigns `variable` to `expression` 356 | 357 | 358 | #### Expressions 359 | 360 | Supports addition, subtraction, multiplication and division. Attempts to coerce strings to integers when doing math. 361 | 362 | #### Do action on Regex 363 | 364 | `// Action` 365 | 366 | Perform `Action` if the current line matches regex. If capture groups are used, you may assign them to variables using `$0, $1, $2...` 367 | 368 | #### Do Sed Action 369 | 370 | `do s/sed/command/g [variable]` 371 | 372 | Execute `sed` command on `variable`. If no `variable` is specified, assumes the current line or capture. 373 | 374 | 375 | #### Dountil 376 | 377 | `dountil s/sed/command/g [variable] Action` 378 | 379 | Perform sed command, and run Action on successful substitution 380 | 381 | #### Goto Action 382 | 383 | `-> [statename]` 384 | 385 | Change current state to `statename`. If a state is not specified, assumes the next state listed in the program. If this is the last state, goes to state "0". 386 | 387 | #### Goto start state 388 | 389 | `-->` 390 | 391 | Transitions state to start state. 392 | 393 | #### Do multiple actions 394 | 395 | `{ Action... }` 396 | 397 | Runs all the actions between the `{` and `}`. 398 | 399 | #### Print 400 | 401 | `print [variable]` 402 | 403 | Prints `variable`. If a variable is not specified, uses `$_` which can be the current line or capture. 404 | 405 | `println [variable]` 406 | 407 | Prints `variable` with a newline. If a variable is not specified, uses `$_` which can be the current line or capture. 408 | 409 | 410 | #### Capture 411 | 412 | `[start|stop] capture [variable]` 413 | 414 | Starts/Stops capturing to `variable`. When capturing is started, input lines are redirected to . If variable is not specified, defaults to `$_`. If `start|stop` is not given, only captures the current line. 415 | 416 | 417 | #### Move head 418 | 419 | `rewind|fastforward /regex/` 420 | 421 | Moves the head backwards/forward to the first line matching `regex`. Stops if it hits the beginning, or halts if it hits the end of file. 422 | 423 | #### if/else 424 | 425 | `if BoolExpr Action [else Action]` 426 | 427 | Executes `action` only if the condtion in BoolExpr is true. An optional else clause is also possible. 428 | 429 | 430 | ### Special States 431 | 432 | Special pre-defined states exist as well. 433 | 434 | `BEGIN`: Actions in this state are run once before consuming any input. Transitioning will stop executing the action. 435 | 436 | `END`: Actions in this state are run after all input is consumed. Cannot transition in this state. 437 | 438 | `ALL`: Actions that are run after every state, even if state transitioned during that cycle. Does not apply to `BEGIN` and `END` 439 | 440 | ### Predefined Variables 441 | 442 | * `$_` The default variable used by arguments. At the beginning of an iteration, stores the current line in `$_` unless it is being used to capture. 443 | * `$@` Contains the original line read in during the iteration. 444 | * `$0` Contains the matched text of the last regex compared. 445 | * `$1..$N` Contains the first to N capture groups in the last regex compared 446 | 447 | ## Contact 448 | 449 | Feedback is always appreciated, you can contact me at armand (dot) halbert (at) gmail.com 450 | -------------------------------------------------------------------------------- /go.mod: -------------------------------------------------------------------------------- 1 | module github.com/ahalbert/ted 2 | 3 | go 1.22.2 4 | 5 | require ( 6 | github.com/alexflint/go-arg v1.5.1 7 | github.com/edsrzf/mmap-go v1.2.0 8 | github.com/rwtodd/Go.Sed v0.0.0-20240405174034-bb8ed5da0fd0 9 | ) 10 | 11 | require ( 12 | github.com/alexflint/go-scalar v1.2.0 // indirect 13 | golang.org/x/sys v0.0.0-20211216021012-1d35b9e2eb4e // indirect 14 | ) 15 | -------------------------------------------------------------------------------- /go.sum: -------------------------------------------------------------------------------- 1 | github.com/alexflint/go-arg v1.5.1 h1:nBuWUCpuRy0snAG+uIJ6N0UvYxpxA0/ghA/AaHxlT8Y= 2 | github.com/alexflint/go-arg v1.5.1/go.mod h1:A7vTJzvjoaSTypg4biM5uYNTkJ27SkNTArtYXnlqVO8= 3 | github.com/alexflint/go-scalar v1.2.0 h1:WR7JPKkeNpnYIOfHRa7ivM21aWAdHD0gEWHCx+WQBRw= 4 | github.com/alexflint/go-scalar v1.2.0/go.mod h1:LoFvNMqS1CPrMVltza4LvnGKhaSpc3oyLEBUZVhhS2o= 5 | github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c= 6 | github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= 7 | github.com/edsrzf/mmap-go v1.2.0 h1:hXLYlkbaPzt1SaQk+anYwKSRNhufIDCchSPkUD6dD84= 8 | github.com/edsrzf/mmap-go v1.2.0/go.mod h1:19H/e8pUPLicwkyNgOykDXkJ9F0MHE+Z52B8EIth78Q= 9 | github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM= 10 | github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4= 11 | github.com/rwtodd/Go.Sed v0.0.0-20240405174034-bb8ed5da0fd0 h1:Sm5QvnDuFhkajkdjAHX51h+gyuv+LmkjX//zjpZwIvA= 12 | github.com/rwtodd/Go.Sed v0.0.0-20240405174034-bb8ed5da0fd0/go.mod h1:c6qgHcSUeSISur4+Kcf3WYTvpL07S8eAsoP40hDiQ1I= 13 | github.com/stretchr/testify v1.2.2/go.mod h1:a8OnRcib4nhh0OaRAV+Yts87kKdq0PP7pXfy6kDkUVs= 14 | github.com/stretchr/testify v1.7.0 h1:nwc3DEeHmmLAfoZucVR881uASk0Mfjw8xYJ99tb5CcY= 15 | github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg= 16 | golang.org/x/sys v0.0.0-20211216021012-1d35b9e2eb4e h1:fLOSk5Q00efkSvAm+4xcoXD+RRmLmmulPn5I3Y9F2EM= 17 | golang.org/x/sys v0.0.0-20211216021012-1d35b9e2eb4e/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= 18 | gopkg.in/yaml.v3 v3.0.0 h1:hjy8E9ON/egN1tAYqKb61G10WtihqetD4sz2H+8nIeA= 19 | gopkg.in/yaml.v3 v3.0.0/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM= 20 | -------------------------------------------------------------------------------- /static/fsa.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ahalbert/ted/b25804cea6383764194bc03ad5735b783f21fa6c/static/fsa.jpg -------------------------------------------------------------------------------- /static/fsa.tex: -------------------------------------------------------------------------------- 1 | \documentclass{article} 2 | 3 | \usepackage{amsmath} 4 | \usepackage{tikz} 5 | % For diamond initial state 6 | \usetikzlibrary{shapes.geometric} 7 | \usetikzlibrary{automata} 8 | % For auto-positioning of labels 9 | \usetikzlibrary{positioning} 10 | \usetikzlibrary{arrows} 11 | \usetikzlibrary{calc} 12 | 13 | \begin{document} 14 | 15 | \begin{tikzpicture}[shorten >=1pt,node distance=2cm,on grid,auto] 16 | 17 | \node[state,initial] (a) {$a$}; 18 | \node[state] (b) [above right=of a] {$b$}; 19 | \node[state] (c) [below right=of a] {$c$}; 20 | \node[state] (d) [below right=of b] {$d$}; 21 | 22 | \path[->] (a) edge node {beep} (b) 23 | edge node [swap] {boop} (c) 24 | edge node [swap] {buzz} (d) 25 | (b) edge node {buzz} (d) 26 | (c) edge node [swap] {buzz} (d); 27 | \end{tikzpicture} 28 | 29 | \begin{tikzpicture}[every node/.style={block}, 30 | block/.style={minimum height=1.5em,outer sep=0pt,draw,rectangle,node distance=0pt}] 31 | \node (A) {$beep$}; 32 | \node (B) [left=of A] {$boop$}; 33 | \node (C) [left=of B] {$ \hat{} $}; 34 | \node (D) [right=of A] {$buzz$}; 35 | \node (E) [right=of D] {$\$ $}; 36 | \node (F) [above = 0.75cm of A] {HEAD}; 37 | \draw[-latex] (F) -- (A); 38 | \draw[-latex,blue] ($(F.east)!0.5!(A.east)$) -- ++(7mm,0); 39 | \draw (C.north west) -- ++(-1cm,0) (C.south west) -- ++ (-1cm,0) 40 | (E.north east) -- ++(1cm,0) (E.south east) -- ++ (1cm,0); 41 | \end{tikzpicture} 42 | 43 | \end{document} 44 | -------------------------------------------------------------------------------- /ted.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import ( 4 | "fmt" 5 | "io" 6 | "os" 7 | "regexp" 8 | 9 | "github.com/ahalbert/ted/ted/flags" 10 | "github.com/ahalbert/ted/ted/lexer" 11 | "github.com/ahalbert/ted/ted/parser" 12 | "github.com/ahalbert/ted/ted/runner" 13 | "github.com/ahalbert/ted/ted/token" 14 | "github.com/alexflint/go-arg" 15 | ) 16 | 17 | func main() { 18 | 19 | arg.MustParse(&flags.Flags) 20 | var program string 21 | if flags.Flags.ProgramFile != "" { 22 | buf, err := os.ReadFile(flags.Flags.ProgramFile) 23 | if err != nil { 24 | panic("FSA File " + flags.Flags.ProgramFile + " not found") 25 | } 26 | program = string(buf) 27 | if flags.Flags.Program != "" { 28 | flags.Flags.InputFiles = append([]string{flags.Flags.Program}, flags.Flags.InputFiles...) 29 | } 30 | } else { 31 | program = flags.Flags.Program 32 | } 33 | 34 | if program == "" { 35 | panic("no FSA supplied") 36 | } 37 | 38 | if flags.Flags.DebugMode { 39 | l := lexer.New(program) 40 | for tok := l.NextToken(); tok.Type != token.EOF; tok = l.NextToken() { 41 | fmt.Printf("%+v\n", tok) 42 | } 43 | } 44 | 45 | l := lexer.New(program) 46 | p := parser.New(l) 47 | 48 | parsedFSA, errors := p.ParseFSA() 49 | if len(errors) > 0 { 50 | for _, err := range errors { 51 | fmt.Println(err) 52 | } 53 | os.Exit(1) 54 | } 55 | 56 | if flags.Flags.DebugMode { 57 | io.WriteString(os.Stdout, parsedFSA.String()) 58 | } 59 | 60 | variables := make(map[string]string) 61 | if flags.Flags.Seperator == "" { 62 | variables["$RS"] = "\n" 63 | } else { 64 | variables["$RS"] = flags.Flags.Seperator 65 | } 66 | 67 | if flags.Flags.NoPrint { 68 | variables["$PRINTMODE"] = "noprint" 69 | } else { 70 | variables["$PRINTMODE"] = "print" 71 | } 72 | 73 | for _, varstring := range flags.Flags.Variables { 74 | re, err := regexp.Compile("(.*?)=(.*)") 75 | if err != nil { 76 | panic("regex compile error") 77 | } 78 | matches := re.FindStringSubmatch(varstring) 79 | if matches != nil { 80 | variables[matches[1]] = matches[2] 81 | } else { 82 | panic("unparsable variable --var " + varstring) 83 | } 84 | } 85 | 86 | r := runner.NewRunner(parsedFSA, variables) 87 | if len(flags.Flags.InputFiles) > 0 { 88 | for _, infile := range flags.Flags.InputFiles { 89 | reader, err := os.Open(infile) 90 | if err != nil { 91 | panic("Input file " + infile + " not found") 92 | } 93 | r.RunFSAFromFile(reader, os.Stdout) 94 | } 95 | } else { 96 | stdin, err := io.ReadAll(os.Stdin) 97 | if err != nil { 98 | panic(err) 99 | } 100 | str := string(stdin) 101 | if len(str) > 0 { 102 | str = str[:len(str)-1] 103 | } 104 | r.RunFSAFromString(str, os.Stdout) 105 | } 106 | } 107 | -------------------------------------------------------------------------------- /ted/ast/ast.go: -------------------------------------------------------------------------------- 1 | package ast 2 | 3 | import ( 4 | "bytes" 5 | ) 6 | 7 | // The base Node interface 8 | type Node interface { 9 | String() string 10 | } 11 | 12 | // All statement nodes implement this 13 | type Statement interface { 14 | Node 15 | statementNode() 16 | } 17 | 18 | type Action interface { 19 | Node 20 | } 21 | 22 | type FSA struct { 23 | Statements []Statement 24 | } 25 | 26 | func (fsa *FSA) String() string { 27 | var out bytes.Buffer 28 | 29 | for _, s := range fsa.Statements { 30 | out.WriteString(s.String() + "\n") 31 | } 32 | 33 | return out.String() 34 | } 35 | 36 | type StateStatement struct { 37 | StateName string 38 | Action Action 39 | } 40 | 41 | func (ss *StateStatement) statementNode() {} 42 | func (ss *StateStatement) String() string { 43 | var out bytes.Buffer 44 | out.WriteString(ss.StateName + ":" + ss.Action.String()) 45 | return out.String() 46 | } 47 | 48 | type FunctionStatement struct { 49 | Name string 50 | Function Expression 51 | } 52 | 53 | func (fs *FunctionStatement) statementNode() {} 54 | func (fs *FunctionStatement) String() string { 55 | var out bytes.Buffer 56 | 57 | out.WriteString("function " + fs.Name) 58 | out.WriteString(fs.Function.String()) 59 | 60 | return out.String() 61 | } 62 | 63 | type ActionBlock struct { 64 | Actions []Action 65 | } 66 | 67 | func (ab *ActionBlock) String() string { 68 | var out bytes.Buffer 69 | out.WriteString("{ ") 70 | for _, action := range ab.Actions { 71 | out.WriteString(action.String() + "; ") 72 | } 73 | out.WriteString(" }") 74 | return out.String() 75 | } 76 | 77 | type RegexAction struct { 78 | Rule string 79 | Action Action 80 | } 81 | 82 | func (ra *RegexAction) String() string { 83 | var out bytes.Buffer 84 | out.WriteString("/" + ra.Rule + "/ " + " :: " + (ra.Action).String()) 85 | return out.String() 86 | } 87 | 88 | type GotoAction struct { 89 | Target string 90 | } 91 | 92 | func (ga *GotoAction) String() string { 93 | var out bytes.Buffer 94 | out.WriteString("goto: " + ga.Target) 95 | return out.String() 96 | } 97 | 98 | type ResetAction struct { 99 | } 100 | 101 | func (ra *ResetAction) String() string { 102 | var out bytes.Buffer 103 | out.WriteString("reset") 104 | return out.String() 105 | } 106 | 107 | type DoSedAction struct { 108 | Variable string 109 | Command string 110 | } 111 | 112 | func (da *DoSedAction) String() string { 113 | var out bytes.Buffer 114 | out.WriteString("sed '" + da.Command + "' using var '" + da.Variable + "'") 115 | return out.String() 116 | } 117 | 118 | type DoUntilSedAction struct { 119 | Variable string 120 | Command string 121 | Action Action 122 | } 123 | 124 | func (da *DoUntilSedAction) String() string { 125 | var out bytes.Buffer 126 | out.WriteString("sed '" + da.Command + "' using var '" + da.Variable + "'") 127 | out.WriteString("if change then '" + da.Action.String()) 128 | return out.String() 129 | } 130 | 131 | type PrintAction struct { 132 | Expression Expression 133 | } 134 | 135 | func (pa *PrintAction) String() string { 136 | var out bytes.Buffer 137 | out.WriteString("print '" + pa.Expression.String() + "'") 138 | return out.String() 139 | } 140 | 141 | type PrintLnAction struct { 142 | Expression Expression 143 | } 144 | 145 | func (pa *PrintLnAction) String() string { 146 | var out bytes.Buffer 147 | out.WriteString("println '" + pa.Expression.String() + "'") 148 | return out.String() 149 | } 150 | 151 | type StartStopCaptureAction struct { 152 | Command string 153 | Variable string 154 | } 155 | 156 | func (sscp *StartStopCaptureAction) String() string { 157 | var out bytes.Buffer 158 | out.WriteString(sscp.Command + " capture into:" + sscp.Variable) 159 | return out.String() 160 | } 161 | 162 | type CaptureAction struct { 163 | Variable string 164 | } 165 | 166 | func (ca *CaptureAction) String() string { 167 | var out bytes.Buffer 168 | out.WriteString("temp capture into:" + ca.Variable) 169 | return out.String() 170 | } 171 | 172 | type ClearAction struct { 173 | Variable string 174 | } 175 | 176 | func (ca *ClearAction) String() string { 177 | var out bytes.Buffer 178 | out.WriteString("clear:" + ca.Variable) 179 | return out.String() 180 | } 181 | 182 | type AssignAction struct { 183 | Target string 184 | Expression Expression 185 | } 186 | 187 | func (aa *AssignAction) String() string { 188 | var out bytes.Buffer 189 | out.WriteString("set:'" + aa.Target + "'= ") 190 | out.WriteString("'" + aa.Expression.String() + "'") 191 | return out.String() 192 | } 193 | 194 | type MoveHeadAction struct { 195 | Command string 196 | Regex string 197 | } 198 | 199 | func (ha *MoveHeadAction) String() string { 200 | var out bytes.Buffer 201 | out.WriteString(ha.Command + " head") 202 | if ha.Regex != "" { 203 | out.WriteString(" to /" + ha.Regex + "/") 204 | } 205 | return out.String() 206 | } 207 | 208 | type IfAction struct { 209 | Condition Expression 210 | Consequence Action 211 | Alternative Action 212 | } 213 | 214 | func (ia *IfAction) String() string { 215 | var out bytes.Buffer 216 | 217 | out.WriteString("if (") 218 | out.WriteString(ia.Condition.String()) 219 | out.WriteString(") then:") 220 | out.WriteString(ia.Consequence.String()) 221 | 222 | if ia.Alternative != nil { 223 | out.WriteString("else:") 224 | out.WriteString(ia.Alternative.String()) 225 | } 226 | 227 | return out.String() 228 | } 229 | 230 | type ExpressionAction struct { 231 | Expression Expression 232 | } 233 | 234 | func (ea *ExpressionAction) String() string { 235 | return ea.Expression.String() 236 | } 237 | -------------------------------------------------------------------------------- /ted/ast/expression.go: -------------------------------------------------------------------------------- 1 | package ast 2 | 3 | import ( 4 | "bytes" 5 | "strconv" 6 | "strings" 7 | ) 8 | 9 | type Expression interface { 10 | Node 11 | expressionNode() 12 | } 13 | 14 | type Identifier struct { 15 | Value string 16 | } 17 | 18 | func (i *Identifier) expressionNode() {} 19 | func (i *Identifier) String() string { return i.Value } 20 | 21 | type StringLiteral struct { 22 | Value string 23 | } 24 | 25 | func (sl *StringLiteral) expressionNode() {} 26 | func (sl *StringLiteral) String() string { return sl.Value } 27 | 28 | type Boolean struct { 29 | Value bool 30 | } 31 | 32 | func (b *Boolean) expressionNode() {} 33 | func (b *Boolean) String() string { 34 | if b.Value { 35 | return "true" 36 | } 37 | return "false" 38 | } 39 | 40 | type IntegerLiteral struct { 41 | Value int 42 | } 43 | 44 | func (il *IntegerLiteral) expressionNode() {} 45 | func (il *IntegerLiteral) String() string { return strconv.Itoa(il.Value) } 46 | 47 | type PrefixExpression struct { 48 | Operator string 49 | Right Expression 50 | } 51 | 52 | func (pe *PrefixExpression) expressionNode() {} 53 | func (pe *PrefixExpression) String() string { 54 | var out bytes.Buffer 55 | 56 | out.WriteString("(") 57 | out.WriteString(pe.Operator) 58 | out.WriteString(pe.Right.String()) 59 | out.WriteString(")") 60 | 61 | return out.String() 62 | } 63 | 64 | type InfixExpression struct { 65 | Left Expression 66 | Operator string 67 | Right Expression 68 | } 69 | 70 | func (oe *InfixExpression) expressionNode() {} 71 | func (oe *InfixExpression) String() string { 72 | var out bytes.Buffer 73 | 74 | out.WriteString("(") 75 | out.WriteString(oe.Left.String()) 76 | out.WriteString(" " + oe.Operator + " ") 77 | out.WriteString(oe.Right.String()) 78 | out.WriteString(")") 79 | 80 | return out.String() 81 | } 82 | 83 | type FunctionLiteral struct { 84 | Parameters []*Identifier 85 | Body Action 86 | } 87 | 88 | func (fl *FunctionLiteral) expressionNode() {} 89 | func (fl *FunctionLiteral) String() string { 90 | var out bytes.Buffer 91 | 92 | params := []string{} 93 | for _, p := range fl.Parameters { 94 | params = append(params, p.String()) 95 | } 96 | 97 | out.WriteString("(") 98 | out.WriteString(strings.Join(params, ", ")) 99 | out.WriteString(") ") 100 | out.WriteString(fl.Body.String()) 101 | 102 | return out.String() 103 | } 104 | 105 | type CallExpression struct { 106 | Function Expression // Identifier or FunctionLiteral 107 | Arguments []Expression 108 | } 109 | 110 | func (ce *CallExpression) expressionNode() {} 111 | func (ce *CallExpression) String() string { 112 | var out bytes.Buffer 113 | 114 | args := []string{} 115 | for _, a := range ce.Arguments { 116 | args = append(args, a.String()) 117 | } 118 | 119 | out.WriteString(ce.Function.String()) 120 | out.WriteString("(") 121 | out.WriteString(strings.Join(args, ", ")) 122 | out.WriteString(")") 123 | 124 | return out.String() 125 | } 126 | -------------------------------------------------------------------------------- /ted/flags/flags.go: -------------------------------------------------------------------------------- 1 | package flags 2 | 3 | var Flags struct { 4 | ProgramFile string `arg:"-f,--fsa-file" placeholder:"FSAFILE" help:"Finite State Autonoma file to run."` 5 | NoPrint bool `arg:"-n,--no-print" help:"Do not print lines by default."` 6 | Seperator string `arg:"-s,--seperator" help:"Record Seperator. Defaults to \\n"` 7 | DebugMode bool `arg:"--debug" help:"Provides Lexer and Parser information."` 8 | Variables []string `arg:"--var,separate" placeholder:"key=value" help:"Variable in the format name=value."` 9 | Program string `arg:"positional" help:"Program to run."` 10 | InputFiles []string `arg:"positional" placeholder:"INPUTFILE" help:"File to use as input."` 11 | } 12 | -------------------------------------------------------------------------------- /ted/lexer/lexer.go: -------------------------------------------------------------------------------- 1 | package lexer 2 | 3 | import ( 4 | "slices" 5 | 6 | "github.com/ahalbert/ted/ted/token" 7 | ) 8 | 9 | type Lexer struct { 10 | input string 11 | position int // current position in input (points to current char) 12 | readPosition int // current reading position in input (after current char) 13 | ch byte // current char under examination 14 | lineNum int 15 | linePosition int 16 | } 17 | 18 | func New(input string) *Lexer { 19 | l := &Lexer{input: input, lineNum: 1, linePosition: 1} 20 | l.readChar() 21 | return l 22 | } 23 | 24 | func (l *Lexer) readChar() { 25 | if l.readPosition >= len(l.input) { 26 | l.ch = 0 27 | } else { 28 | l.ch = l.input[l.readPosition] 29 | 30 | if l.ch == '\n' { 31 | l.lineNum++ 32 | l.linePosition = 1 33 | } 34 | } 35 | l.linePosition++ 36 | l.position = l.readPosition 37 | l.readPosition += 1 38 | } 39 | 40 | func (l *Lexer) peek(lookahead int) string { 41 | if l.readPosition+lookahead >= len(l.input) { 42 | return "" 43 | } else { 44 | return l.input[l.readPosition : l.readPosition+lookahead] 45 | } 46 | } 47 | 48 | func (l *Lexer) NextToken() token.Token { 49 | var tok token.Token 50 | 51 | l.skipWhitespace() 52 | 53 | switch l.ch { 54 | case '/': 55 | l.readChar() 56 | if l.ch == ' ' { 57 | tok = l.newToken(token.SLASH, "/") 58 | } else { 59 | tok = l.newToken(token.REGEX, l.readUntilChar('/')) 60 | } 61 | case '"': 62 | l.readChar() 63 | tok = l.newToken(token.STRING, l.readUntilChar('"')) 64 | case '\'': 65 | l.readChar() 66 | tok = l.newToken(token.STRING, l.readUntilChar('\'')) 67 | case '`': 68 | l.readChar() 69 | tok = l.newToken(token.STRING, l.readUntilChar('`')) 70 | case '-': 71 | l.readChar() 72 | if l.ch == '-' && l.peek(1) == ">" { 73 | l.readChar() 74 | tok = l.newToken(token.RESET, "-->") 75 | } else if l.ch == '>' { 76 | tok = l.newToken(token.GOTO, "->") 77 | } else { 78 | tok = l.newToken(token.MINUS, "-") 79 | } 80 | case '=': 81 | if l.peek(1) == "=" { 82 | l.readChar() 83 | tok = l.newToken(token.EQ, "==") 84 | } else { 85 | tok = l.newToken(token.ASSIGN, "=") 86 | } 87 | case '{': 88 | tok = l.newToken(token.LBRACE, "{") 89 | case '}': 90 | tok = l.newToken(token.RBRACE, "}") 91 | case '(': 92 | tok = l.newToken(token.LPAREN, "(") 93 | case ')': 94 | tok = l.newToken(token.RPAREN, ")") 95 | case ',': 96 | tok = l.newToken(token.COMMA, ",") 97 | case ':': 98 | tok = l.newToken(token.COLON, ":") 99 | case ';': 100 | tok = l.newToken(token.SEMICOLON, ";") 101 | case '+': 102 | tok = l.newToken(token.PLUS, "+") 103 | case '*': 104 | tok = l.newToken(token.ASTERISK, "*") 105 | case 0: 106 | tok = l.newToken(token.EOF, "") 107 | default: 108 | if isLetter(l.ch) { 109 | tok.LineNum = l.lineNum 110 | tok.Position = l.linePosition 111 | tok.Literal = l.readIdentifier() 112 | tok.Type = token.LookupIdent(tok.Literal) 113 | switch tok.Type { 114 | case token.DO: 115 | tok.Literal = l.readDo() 116 | l.readChar() 117 | case token.DOUNTIL: 118 | tok.Literal = l.readDo() 119 | l.readChar() 120 | case token.IDENT: 121 | tok = l.handleIdentfierSpecialCases(tok) 122 | } 123 | return tok 124 | } else { 125 | tok = l.newToken(token.ILLEGAL, string(l.ch)) 126 | } 127 | } 128 | 129 | l.readChar() 130 | 131 | return tok 132 | } 133 | 134 | func (l *Lexer) handleIdentfierSpecialCases(t token.Token) token.Token { 135 | if l.ch == ':' { 136 | l.readChar() 137 | return l.newToken(token.LABEL, t.Literal) 138 | } 139 | return t 140 | } 141 | 142 | func (l *Lexer) readDo() string { 143 | l.skipWhitespace() 144 | switch l.ch { 145 | case '"': 146 | l.readChar() 147 | return l.readUntilChar('"') 148 | case '\'': 149 | l.readChar() 150 | return l.readUntilChar('\'') 151 | case '`': 152 | l.readChar() 153 | return l.readUntilChar('`') 154 | default: 155 | return l.readUntilChar(' ', '\t', '\n', '\r') 156 | } 157 | } 158 | 159 | func (l *Lexer) readIdentifier() string { 160 | position := l.position 161 | for isLetter(l.ch) && l.ch != 0 { 162 | l.readChar() 163 | } 164 | return l.input[position:l.position] 165 | } 166 | 167 | func (l *Lexer) skipWhitespace() { 168 | for l.ch == '#' || l.ch == ' ' || l.ch == '\t' || l.ch == '\n' || l.ch == '\r' { 169 | if l.ch == '#' { 170 | l.readUntilChar('\n') 171 | } 172 | l.readChar() 173 | } 174 | } 175 | 176 | func (l *Lexer) readUntilChar(chars ...byte) string { 177 | position := l.position 178 | for !slices.Contains(chars, l.ch) && l.ch != 0 { 179 | l.readChar() 180 | } 181 | return l.input[position:l.position] 182 | } 183 | 184 | func isLetter(ch byte) bool { 185 | return 'a' <= ch && ch <= 'z' || 'A' <= ch && ch <= 'Z' || '0' <= ch && ch <= '9' || ch == '_' || ch == '$' || ch == '@' 186 | } 187 | 188 | func (l *Lexer) newToken(tokenType token.TokenType, s string) token.Token { 189 | return token.Token{Type: tokenType, Literal: s, LineNum: l.lineNum, Position: l.linePosition} 190 | } 191 | -------------------------------------------------------------------------------- /ted/lexer/lexer_test.go: -------------------------------------------------------------------------------- 1 | package lexer 2 | 3 | import ( 4 | "testing" 5 | 6 | "github.com/ahalbert/ted/ted/token" 7 | ) 8 | 9 | func TestReadChar(t *testing.T) { 10 | tests := []struct { 11 | input string 12 | expectedChars []byte 13 | expectedPositions []int 14 | expectedReadPositions []int 15 | }{ 16 | { 17 | input: "test", 18 | expectedChars: []byte{'t', 'e', 's', 't', 0}, 19 | expectedPositions: []int{0, 1, 2, 3, 4}, 20 | expectedReadPositions: []int{1, 2, 3, 4, 5}, 21 | }, 22 | { 23 | input: "hello", 24 | expectedChars: []byte{'h', 'e', 'l', 'l', 'o', 0}, 25 | expectedPositions: []int{0, 1, 2, 3, 4, 5}, 26 | expectedReadPositions: []int{1, 2, 3, 4, 5, 6}, 27 | }, 28 | { 29 | input: "a\nb", 30 | expectedChars: []byte{'a', '\n', 'b', 0}, 31 | expectedPositions: []int{0, 1, 2, 3}, 32 | expectedReadPositions: []int{1, 2, 3, 4}, 33 | }, 34 | { 35 | input: "", 36 | expectedChars: []byte{0}, 37 | expectedPositions: []int{0}, 38 | expectedReadPositions: []int{1}, 39 | }, 40 | } 41 | 42 | for i, tt := range tests { 43 | l := New(tt.input) 44 | for j := 0; j < len(tt.expectedChars); j++ { 45 | if l.ch != tt.expectedChars[j] { 46 | t.Errorf("test[%d] - readChar() ch = %q, want %q", i, l.ch, tt.expectedChars[j]) 47 | } 48 | if l.position != tt.expectedPositions[j] { 49 | t.Errorf("test[%d] - readChar() position = %d, want %d", i, l.position, tt.expectedPositions[j]) 50 | } 51 | if l.readPosition != tt.expectedReadPositions[j] { 52 | t.Errorf("test[%d] - readChar() readPosition = %d, want %d", i, l.readPosition, tt.expectedReadPositions[j]) 53 | } 54 | l.readChar() 55 | } 56 | } 57 | } 58 | 59 | func TestPeek(t *testing.T) { 60 | input := "test" 61 | l := New(input) 62 | 63 | tests := []struct { 64 | lookahead int 65 | expectedPeek string 66 | }{ 67 | {0, ""}, 68 | {1, "e"}, 69 | {2, "es"}, 70 | {3, ""}, 71 | {4, ""}, 72 | {5, ""}, 73 | } 74 | 75 | for i, tt := range tests { 76 | peeked := l.peek(tt.lookahead) 77 | if peeked != tt.expectedPeek { 78 | t.Errorf("test[%d] - peek(%d) = %q, want %q", i, tt.lookahead, peeked, tt.expectedPeek) 79 | } 80 | } 81 | } 82 | 83 | func TestReadDo(t *testing.T) { 84 | tests := []struct { 85 | input string 86 | expectedOutput string 87 | }{ 88 | {`"hello w"`, "hello w"}, 89 | {`' h world'`, " h world"}, 90 | {"`test`", "test"}, 91 | {"simple text", "simple"}, 92 | {" whitespace", "whitespace"}, 93 | {"whitespace ", "whitespace"}, 94 | {"#whitespace \nab", "ab"}, 95 | } 96 | 97 | for i, tt := range tests { 98 | l := New(tt.input) 99 | output := l.readDo() 100 | if output != tt.expectedOutput { 101 | t.Errorf("test[%d] - readDo() = %q, want %q", i, output, tt.expectedOutput) 102 | } 103 | } 104 | } 105 | 106 | func TestReadIdentifier(t *testing.T) { 107 | tests := []struct { 108 | input string 109 | expectedOutput string 110 | }{ 111 | {"identifier", "identifier"}, 112 | {"_underscore", "_underscore"}, 113 | {"$dollar", "$dollar"}, 114 | {"with123numbers", "with123numbers"}, 115 | {"059mixed_Case$", "059mixed_Case$"}, 116 | {"", ""}, 117 | {"!notIdentifier", ""}, 118 | } 119 | 120 | for i, tt := range tests { 121 | l := New(tt.input) 122 | output := l.readIdentifier() 123 | if output != tt.expectedOutput { 124 | t.Errorf("test[%d] - readIdentifier() = %q, want %q", i, output, tt.expectedOutput) 125 | } 126 | } 127 | } 128 | 129 | func TestNextToken(t *testing.T) { 130 | tests := []struct { 131 | input string 132 | expectedTokens []struct { 133 | expectedType token.TokenType 134 | expectedLiteral string 135 | } 136 | }{ 137 | { 138 | input: `=+(){},;*:`, // char tests 139 | expectedTokens: []struct { 140 | expectedType token.TokenType 141 | expectedLiteral string 142 | }{ 143 | {token.ASSIGN, "="}, 144 | {token.PLUS, "+"}, 145 | {token.LPAREN, "("}, 146 | {token.RPAREN, ")"}, 147 | {token.LBRACE, "{"}, 148 | {token.RBRACE, "}"}, 149 | {token.COMMA, ","}, 150 | {token.SEMICOLON, ";"}, 151 | {token.ASTERISK, "*"}, 152 | {token.COLON, ":"}, 153 | {token.EOF, ""}, 154 | }, 155 | }, 156 | { 157 | input: `/ab{2}/`, // regexp test 158 | expectedTokens: []struct { 159 | expectedType token.TokenType 160 | expectedLiteral string 161 | }{ 162 | {token.REGEX, "ab{2}"}, 163 | {token.EOF, ""}, 164 | }, 165 | }, 166 | { 167 | input: `/ +`, // slash test 168 | expectedTokens: []struct { 169 | expectedType token.TokenType 170 | expectedLiteral string 171 | }{ 172 | {token.SLASH, "/"}, 173 | {token.PLUS, "+"}, 174 | {token.EOF, ""}, 175 | }, 176 | }, 177 | { 178 | input: `"abcd"`, // string test 179 | expectedTokens: []struct { 180 | expectedType token.TokenType 181 | expectedLiteral string 182 | }{ 183 | {token.STRING, "abcd"}, 184 | {token.EOF, ""}, 185 | }, 186 | }, 187 | { 188 | input: `'abcd'`, // string test 189 | expectedTokens: []struct { 190 | expectedType token.TokenType 191 | expectedLiteral string 192 | }{ 193 | {token.STRING, "abcd"}, 194 | {token.EOF, ""}, 195 | }, 196 | }, 197 | { 198 | input: "`abcd`", // string test 199 | expectedTokens: []struct { 200 | expectedType token.TokenType 201 | expectedLiteral string 202 | }{ 203 | {token.STRING, "abcd"}, 204 | {token.EOF, ""}, 205 | }, 206 | }, 207 | { 208 | input: `!`, // illegal char test 209 | expectedTokens: []struct { 210 | expectedType token.TokenType 211 | expectedLiteral string 212 | }{ 213 | {token.ILLEGAL, "!"}, 214 | {token.EOF, ""}, 215 | }, 216 | }, 217 | { 218 | input: `/foo/ -> /bar/ -> do s/baz/bang/`, // sample input 219 | expectedTokens: []struct { 220 | expectedType token.TokenType 221 | expectedLiteral string 222 | }{ 223 | {token.REGEX, "foo"}, 224 | {token.GOTO, "->"}, 225 | {token.REGEX, "bar"}, 226 | {token.GOTO, "->"}, 227 | {token.DO, "s/baz/bang/"}, 228 | {token.EOF, ""}, 229 | }, 230 | }, 231 | { 232 | input: `{ capture fastforward /buzz/ -> }`, // sample input 233 | expectedTokens: []struct { 234 | expectedType token.TokenType 235 | expectedLiteral string 236 | }{ 237 | {token.LBRACE, "{"}, 238 | {token.CAPTURE, "capture"}, 239 | {token.FASTFWD, "fastforward"}, 240 | {token.REGEX, "buzz"}, 241 | {token.GOTO, "->"}, 242 | {token.RBRACE, "}"}, 243 | {token.EOF, ""}, 244 | }, 245 | }, 246 | { 247 | input: `dountil s/buzz/boop/ ->`, // sample input 248 | expectedTokens: []struct { 249 | expectedType token.TokenType 250 | expectedLiteral string 251 | }{ 252 | {token.DOUNTIL, "s/buzz/boop/"}, 253 | {token.GOTO, "->"}, 254 | {token.EOF, ""}, 255 | }, 256 | }, 257 | { 258 | input: `# Welcome to Ted! 259 | /foo/ -> /bar/ -> do s/baz/bang/`, // sample input 260 | expectedTokens: []struct { 261 | expectedType token.TokenType 262 | expectedLiteral string 263 | }{ 264 | {token.REGEX, "foo"}, 265 | {token.GOTO, "->"}, 266 | {token.REGEX, "bar"}, 267 | {token.GOTO, "->"}, 268 | {token.DO, "s/baz/bang/"}, 269 | {token.EOF, ""}, 270 | }, 271 | }, 272 | { 273 | input: `/buzz/ {println myvar}`, // sample input 274 | expectedTokens: []struct { 275 | expectedType token.TokenType 276 | expectedLiteral string 277 | }{ 278 | {token.REGEX, "buzz"}, 279 | {token.LBRACE, "{"}, 280 | {token.PRINTLN, "println"}, 281 | {token.IDENT, "myvar"}, 282 | {token.RBRACE, "}"}, 283 | {token.EOF, ""}, 284 | }, 285 | }, 286 | { 287 | input: `ALL: /Success/ --> `, // sample input 288 | expectedTokens: []struct { 289 | expectedType token.TokenType 290 | expectedLiteral string 291 | }{ 292 | {token.LABEL, "ALL"}, 293 | {token.REGEX, "Success"}, 294 | {token.RESET, "-->"}, 295 | {token.EOF, ""}, 296 | }, 297 | }, 298 | { 299 | input: `5: { println count /div/ let count = count - 1 if count == 0 -> }`, // sample input 300 | expectedTokens: []struct { 301 | expectedType token.TokenType 302 | expectedLiteral string 303 | }{ 304 | {token.LABEL, "5"}, 305 | {token.LBRACE, "{"}, 306 | {token.PRINTLN, "println"}, 307 | {token.IDENT, "count"}, 308 | {token.REGEX, "div"}, 309 | {token.LET, "let"}, 310 | {token.IDENT, "count"}, 311 | {token.ASSIGN, "="}, 312 | {token.IDENT, "count"}, 313 | {token.MINUS, "-"}, 314 | {token.IDENT, "1"}, 315 | {token.IF, "if"}, 316 | {token.IDENT, "count"}, 317 | {token.EQ, "=="}, 318 | {token.IDENT, "0"}, 319 | {token.GOTO, "->"}, 320 | {token.RBRACE, "}"}, 321 | {token.EOF, ""}, 322 | }, 323 | }, 324 | } 325 | 326 | for _, tt := range tests { 327 | l := New(tt.input) 328 | 329 | for i, expectedToken := range tt.expectedTokens { 330 | tok := l.NextToken() 331 | 332 | if tok.Type != expectedToken.expectedType { 333 | t.Fatalf("input: %q, tests[%d] - tokentype wrong. expected=%q, got=%q", tt.input, i, expectedToken.expectedType, tok.Type) 334 | } 335 | 336 | if tok.Literal != expectedToken.expectedLiteral { 337 | t.Fatalf("input: %q, tests[%d] - literal wrong. expected=%q, got=%q", tt.input, i, expectedToken.expectedLiteral, tok.Literal) 338 | } 339 | } 340 | } 341 | } 342 | -------------------------------------------------------------------------------- /ted/parser/parser.go: -------------------------------------------------------------------------------- 1 | package parser 2 | 3 | import ( 4 | "fmt" 5 | "strconv" 6 | 7 | "github.com/ahalbert/ted/ted/ast" 8 | "github.com/ahalbert/ted/ted/lexer" 9 | "github.com/ahalbert/ted/ted/token" 10 | ) 11 | 12 | const ( 13 | _ int = iota 14 | LOWEST 15 | EQUALS // == 16 | LESSGREATER // > or < 17 | SUM // + 18 | PRODUCT // * 19 | PREFIX // -X or !X 20 | CALL // myFunction(X) 21 | ) 22 | 23 | var precedences = map[token.TokenType]int{ 24 | token.EQ: EQUALS, 25 | token.NOT_EQ: EQUALS, 26 | token.LT: LESSGREATER, 27 | token.GT: LESSGREATER, 28 | token.PLUS: SUM, 29 | token.MINUS: SUM, 30 | token.SLASH: PRODUCT, 31 | token.ASTERISK: PRODUCT, 32 | token.LPAREN: CALL, 33 | } 34 | 35 | type ( 36 | prefixParseFn func() ast.Expression 37 | infixParseFn func(ast.Expression) ast.Expression 38 | ) 39 | 40 | type Parser struct { 41 | l *lexer.Lexer 42 | errors []string 43 | 44 | curToken token.Token 45 | peekToken token.Token 46 | prefixParseFns map[token.TokenType]prefixParseFn 47 | infixParseFns map[token.TokenType]infixParseFn 48 | 49 | AnonymousStates int 50 | } 51 | 52 | func New(l *lexer.Lexer) *Parser { 53 | p := &Parser{ 54 | l: l, 55 | errors: []string{}, 56 | AnonymousStates: 1, 57 | } 58 | 59 | p.prefixParseFns = make(map[token.TokenType]prefixParseFn) 60 | p.infixParseFns = make(map[token.TokenType]infixParseFn) 61 | p.registerPrefix(token.IDENT, p.parseIdentifierExpr) 62 | p.registerPrefix(token.STRING, p.parseStringLiteralExpr) 63 | 64 | p.registerPrefix(token.BANG, p.parsePrefixExpression) 65 | p.registerPrefix(token.MINUS, p.parsePrefixExpression) 66 | p.registerPrefix(token.LPAREN, p.parseGroupedExpression) 67 | p.registerPrefix(token.FUNCTION, p.parseFunctionLiteral) 68 | 69 | p.registerInfix(token.PLUS, p.parseInfixExpression) 70 | p.registerInfix(token.MINUS, p.parseInfixExpression) 71 | p.registerInfix(token.SLASH, p.parseInfixExpression) 72 | p.registerInfix(token.ASTERISK, p.parseInfixExpression) 73 | p.registerInfix(token.EQ, p.parseInfixExpression) 74 | p.registerInfix(token.NOT_EQ, p.parseInfixExpression) 75 | p.registerInfix(token.LT, p.parseInfixExpression) 76 | p.registerInfix(token.GT, p.parseInfixExpression) 77 | 78 | p.registerInfix(token.LPAREN, p.parseCallExpression) 79 | // Read two tokens, so curToken and peekToken are both set 80 | p.nextToken() 81 | p.nextToken() 82 | 83 | return p 84 | } 85 | 86 | func (p *Parser) registerPrefix(tokenType token.TokenType, fn prefixParseFn) { 87 | p.prefixParseFns[tokenType] = fn 88 | } 89 | 90 | func (p *Parser) registerInfix(tokenType token.TokenType, fn infixParseFn) { 91 | p.infixParseFns[tokenType] = fn 92 | } 93 | 94 | func (p *Parser) nextToken() { 95 | p.curToken = p.peekToken 96 | p.peekToken = p.l.NextToken() 97 | } 98 | 99 | func (p *Parser) curTokenIs(t token.TokenType) bool { 100 | return p.curToken.Type == t 101 | } 102 | 103 | func (p *Parser) peekTokenIs(t token.TokenType) bool { 104 | return p.peekToken.Type == t 105 | } 106 | 107 | func (p *Parser) curPrecedence() int { 108 | if precedence, ok := precedences[p.curToken.Type]; ok { 109 | return precedence 110 | } 111 | return LOWEST 112 | } 113 | 114 | func (p *Parser) peekPrecedence() int { 115 | if precedence, ok := precedences[p.peekToken.Type]; ok { 116 | return precedence 117 | } 118 | return LOWEST 119 | } 120 | 121 | func (p *Parser) expectPeek(t token.TokenType) bool { 122 | if p.peekTokenIs(t) { 123 | p.nextToken() 124 | return true 125 | } else { 126 | p.peekError(t) 127 | return false 128 | } 129 | } 130 | 131 | func (p *Parser) addError(msg string) { 132 | m := fmt.Sprintf("parser error at line %d col %d: ", p.curToken.LineNum, p.curToken.Position) + msg 133 | p.errors = append(p.errors, m) 134 | p.nextToken() 135 | } 136 | 137 | func (p *Parser) peekError(t token.TokenType) { 138 | msg := fmt.Sprintf("expected next token to be %s, got %s instead", 139 | t, p.peekToken.Type) 140 | p.addError(msg) 141 | } 142 | 143 | func (p *Parser) noPrefixParseFnError(t token.TokenType) { 144 | msg := fmt.Sprintf("no prefix parse function for %s found", t) 145 | p.addError(msg) 146 | } 147 | 148 | func (p *Parser) ParseFSA() (ast.FSA, []string) { 149 | program := ast.FSA{} 150 | program.Statements = []ast.Statement{} 151 | p.errors = []string{} 152 | 153 | for !p.curTokenIs(token.EOF) { 154 | stmt := p.parseStatement() 155 | program.Statements = append(program.Statements, stmt) 156 | switch stmt.(type) { 157 | case *ast.StateStatement: 158 | statename := stmt.(*ast.StateStatement).StateName 159 | for p.curTokenIs(token.COMMA) { 160 | stmt := &ast.StateStatement{StateName: statename} 161 | p.nextToken() 162 | stmt.Action = p.parseAction() 163 | program.Statements = append(program.Statements, stmt) 164 | } 165 | } 166 | } 167 | 168 | return program, p.errors 169 | } 170 | 171 | func (p *Parser) parseStatement() ast.Statement { 172 | statement := &ast.StateStatement{} 173 | if p.curTokenIs(token.LABEL) { 174 | statement.StateName = p.curToken.Literal 175 | p.nextToken() 176 | } else if p.curTokenIs(token.FUNCTION) { 177 | return p.parseFunctionStatement() 178 | } else { 179 | statement.StateName = strconv.Itoa(p.AnonymousStates) 180 | p.AnonymousStates++ 181 | } 182 | 183 | statement.Action = p.parseAction() 184 | return statement 185 | } 186 | 187 | func (p *Parser) parseFunctionStatement() *ast.FunctionStatement { 188 | function := &ast.FunctionStatement{} 189 | p.nextToken() 190 | if !p.curTokenIs(token.IDENT) { 191 | p.addError("expected function identifier") 192 | return nil 193 | } 194 | function.Name = p.curToken.Literal 195 | p.nextToken() 196 | function.Function = p.parseFunctionLiteral() 197 | return function 198 | } 199 | 200 | func (p *Parser) parseAction() ast.Action { 201 | var action ast.Action 202 | switch p.curToken.Type { 203 | case token.LBRACE: 204 | action = p.parseActionBlock() 205 | case token.REGEX: 206 | action = p.parseRegexAction() 207 | case token.GOTO: 208 | action = p.parseGotoAction() 209 | case token.RESET: 210 | action = p.parseResetAction() 211 | case token.DO: 212 | action = p.parseDoAction() 213 | case token.DOUNTIL: 214 | action = p.parseDoUntilAction() 215 | case token.PRINT: 216 | action = p.parsePrintAction() 217 | case token.PRINTLN: 218 | action = p.parsePrintLnAction() 219 | case token.START: 220 | action = p.parseStartStopCaptureAction() 221 | case token.STOP: 222 | action = p.parseStartStopCaptureAction() 223 | case token.CAPTURE: 224 | action = p.parseCaptureAction() 225 | case token.CLEAR: 226 | action = p.parseClearAction() 227 | case token.LET: 228 | action = p.parseAssignAction() 229 | case token.REWIND: 230 | action = p.parseMoveHeadAction() 231 | case token.FASTFWD: 232 | action = p.parseMoveHeadAction() 233 | case token.IF: 234 | action = p.parseIfAction() 235 | case token.ILLEGAL: 236 | p.addError(fmt.Sprintf("expected action, got illegal token %s", p.curToken.Literal)) 237 | return nil 238 | default: 239 | action = p.parseExpressionAction() 240 | // p.addError(fmt.Sprintf("expected action, got %s %s", p.curToken.Type, p.curToken.Literal)) 241 | // return nil 242 | } 243 | return action 244 | } 245 | 246 | func (p *Parser) parseActionBlock() *ast.ActionBlock { 247 | action := &ast.ActionBlock{} 248 | p.nextToken() 249 | for !p.curTokenIs(token.RBRACE) { 250 | action.Actions = append(action.Actions, p.parseAction()) 251 | } 252 | p.nextToken() 253 | return action 254 | } 255 | 256 | func (p *Parser) parseRegexAction() *ast.RegexAction { 257 | action := &ast.RegexAction{Rule: p.curToken.Literal} 258 | p.nextToken() 259 | action.Action = p.parseAction() 260 | return action 261 | } 262 | 263 | func (p *Parser) parseGotoAction() *ast.GotoAction { 264 | action := &ast.GotoAction{} 265 | if p.peekTokenIs(token.IDENT) { 266 | p.nextToken() 267 | action.Target = p.curToken.Literal 268 | } 269 | 270 | p.nextToken() 271 | return action 272 | } 273 | 274 | func (p *Parser) parseResetAction() *ast.ResetAction { 275 | action := &ast.ResetAction{} 276 | 277 | p.nextToken() 278 | return action 279 | } 280 | 281 | func (p *Parser) parseDoAction() *ast.DoSedAction { 282 | action := &ast.DoSedAction{Command: p.curToken.Literal} 283 | action.Variable = p.helpCheckForOptionalVarArg() 284 | return action 285 | } 286 | 287 | func (p *Parser) parseDoUntilAction() *ast.DoUntilSedAction { 288 | action := &ast.DoUntilSedAction{Command: p.curToken.Literal} 289 | action.Variable = p.helpCheckForOptionalVarArg() 290 | action.Action = p.parseAction() 291 | return action 292 | } 293 | 294 | func (p *Parser) parsePrintAction() *ast.PrintAction { 295 | action := &ast.PrintAction{} 296 | action.Expression = p.helpCheckForOptionalExpr() 297 | return action 298 | } 299 | 300 | func (p *Parser) parsePrintLnAction() *ast.PrintLnAction { 301 | action := &ast.PrintLnAction{} 302 | action.Expression = p.helpCheckForOptionalExpr() 303 | return action 304 | } 305 | 306 | func (p *Parser) parseClearAction() *ast.ClearAction { 307 | action := &ast.ClearAction{} 308 | action.Variable = p.helpCheckForOptionalVarArg() 309 | return action 310 | } 311 | 312 | func (p *Parser) parseStartStopCaptureAction() *ast.StartStopCaptureAction { 313 | action := &ast.StartStopCaptureAction{Command: p.curToken.Literal} 314 | p.nextToken() 315 | if p.curTokenIs(token.CAPTURE) { 316 | action.Variable = p.helpCheckForOptionalVarArg() 317 | } else { 318 | p.addError(fmt.Sprintf("expected keyword capture, got %s %s", p.curToken.Type, p.curToken.Literal)) 319 | return nil 320 | } 321 | return action 322 | } 323 | 324 | func (p *Parser) helpCheckForOptionalVarArg() string { 325 | p.nextToken() 326 | if p.curTokenIs(token.IDENT) { 327 | variable := p.curToken.Literal 328 | p.nextToken() 329 | return variable 330 | } else { 331 | return "$_" 332 | } 333 | } 334 | 335 | func (p *Parser) helpCheckForOptionalExpr() ast.Expression { 336 | p.nextToken() 337 | expr := p.parseExpression(LOWEST) 338 | if expr != nil { 339 | return expr 340 | } else { 341 | return &ast.Identifier{Value: "$_"} 342 | } 343 | } 344 | 345 | func (p *Parser) parseCaptureAction() *ast.CaptureAction { 346 | action := &ast.CaptureAction{} 347 | action.Variable = p.helpCheckForOptionalVarArg() 348 | return action 349 | } 350 | 351 | func (p *Parser) parseAssignAction() *ast.AssignAction { 352 | action := &ast.AssignAction{} 353 | p.nextToken() 354 | if p.curTokenIs(token.IDENT) { 355 | //TODO: Check is valid variable 356 | action.Target = p.curToken.Literal 357 | } else { 358 | p.addError(fmt.Sprintf("expected variable, got %s %s", p.curToken.Type, p.curToken.Literal)) 359 | return nil 360 | } 361 | 362 | p.nextToken() 363 | if !p.curTokenIs(token.ASSIGN) { 364 | p.addError(fmt.Sprintf("expected =, got %s %s", p.curToken.Type, p.curToken.Literal)) 365 | return nil 366 | } 367 | 368 | p.nextToken() 369 | action.Expression = p.parseExpression(LOWEST) 370 | 371 | return action 372 | } 373 | 374 | func (p *Parser) parseIfAction() *ast.IfAction { 375 | action := &ast.IfAction{} 376 | p.nextToken() 377 | action.Condition = p.parseExpression(LOWEST) 378 | 379 | action.Consequence = p.parseAction() 380 | 381 | if p.curTokenIs(token.ELSE) { 382 | p.nextToken() 383 | action.Alternative = p.parseAction() 384 | } 385 | return action 386 | } 387 | 388 | func (p *Parser) parseMoveHeadAction() *ast.MoveHeadAction { 389 | t := p.curToken.Type 390 | action := &ast.MoveHeadAction{Command: p.curToken.Literal} 391 | p.nextToken() 392 | if t == token.REWIND || t == token.FASTFWD { 393 | if p.curTokenIs(token.REGEX) { 394 | action.Regex = p.curToken.Literal 395 | p.nextToken() 396 | } else { 397 | p.addError(fmt.Sprintf("%s expected regex, got %s %s", action.Command, p.curToken.Type, p.curToken.Literal)) 398 | return nil 399 | } 400 | } 401 | return action 402 | } 403 | 404 | func (p *Parser) parseExpression(precedence int) ast.Expression { 405 | prefix := p.prefixParseFns[p.curToken.Type] 406 | if prefix == nil { 407 | //p.noPrefixParseFnError(p.curToken.Type) 408 | return nil 409 | } 410 | leftExp := prefix() 411 | for precedence < p.curPrecedence() { 412 | infix := p.infixParseFns[p.curToken.Type] 413 | if infix == nil { 414 | return leftExp 415 | } 416 | leftExp = infix(leftExp) 417 | } 418 | 419 | return leftExp 420 | } 421 | 422 | func (p *Parser) parseIdentifierExpr() ast.Expression { 423 | defer p.nextToken() 424 | val, err := strconv.Atoi(p.curToken.Literal) 425 | if err == nil { 426 | return &ast.IntegerLiteral{Value: val} 427 | } 428 | if p.curToken.Literal == "false" { 429 | return &ast.Boolean{Value: false} 430 | } 431 | if p.curToken.Literal == "true" { 432 | return &ast.Boolean{Value: true} 433 | } 434 | return &ast.Identifier{Value: p.curToken.Literal} 435 | } 436 | 437 | func (p *Parser) parseStringLiteralExpr() ast.Expression { 438 | lit := &ast.StringLiteral{Value: p.curToken.Literal} 439 | p.nextToken() 440 | return lit 441 | } 442 | 443 | func (p *Parser) parsePrefixExpression() ast.Expression { 444 | expression := &ast.PrefixExpression{Operator: p.curToken.Literal} 445 | p.nextToken() 446 | expression.Right = p.parseExpression(PREFIX) 447 | return expression 448 | } 449 | 450 | func (p *Parser) parseGroupedExpression() ast.Expression { 451 | p.nextToken() 452 | exp := p.parseExpression(LOWEST) 453 | if !p.curTokenIs(token.RPAREN) { 454 | p.addError(fmt.Sprintf("expected ), got %s %s", p.curToken.Type, p.curToken.Literal)) 455 | return nil 456 | } 457 | p.nextToken() 458 | return exp 459 | } 460 | 461 | func (p *Parser) parseFunctionLiteral() ast.Expression { 462 | lit := &ast.FunctionLiteral{} 463 | 464 | if !p.curTokenIs(token.LPAREN) { 465 | p.addError(fmt.Sprintf("expected (, got %s %s", p.curToken.Type, p.curToken.Literal)) 466 | return nil 467 | } 468 | 469 | lit.Parameters = p.parseFunctionParameters() 470 | 471 | if !p.expectPeek(token.LBRACE) { 472 | return nil 473 | } 474 | 475 | lit.Body = p.parseAction() 476 | 477 | return lit 478 | } 479 | 480 | func (p *Parser) parseFunctionParameters() []*ast.Identifier { 481 | identifiers := []*ast.Identifier{} 482 | 483 | if p.peekTokenIs(token.RPAREN) { 484 | p.nextToken() 485 | return identifiers 486 | } 487 | 488 | p.nextToken() 489 | 490 | ident := &ast.Identifier{Value: p.curToken.Literal} 491 | identifiers = append(identifiers, ident) 492 | 493 | for p.peekTokenIs(token.COMMA) { 494 | p.nextToken() 495 | p.nextToken() 496 | ident := &ast.Identifier{Value: p.curToken.Literal} 497 | identifiers = append(identifiers, ident) 498 | } 499 | 500 | if !p.expectPeek(token.RPAREN) { 501 | return nil 502 | } 503 | 504 | return identifiers 505 | } 506 | 507 | func (p *Parser) parseInfixExpression(left ast.Expression) ast.Expression { 508 | expression := &ast.InfixExpression{ 509 | Operator: p.curToken.Literal, 510 | Left: left, 511 | } 512 | 513 | precedence := p.curPrecedence() 514 | p.nextToken() 515 | expression.Right = p.parseExpression(precedence) 516 | 517 | return expression 518 | } 519 | 520 | func (p *Parser) parseCallExpression(function ast.Expression) ast.Expression { 521 | exp := &ast.CallExpression{Function: function} 522 | exp.Arguments = p.parseCallArguments() 523 | return exp 524 | } 525 | 526 | func (p *Parser) parseCallArguments() []ast.Expression { 527 | args := []ast.Expression{} 528 | 529 | if p.peekTokenIs(token.RPAREN) { 530 | p.nextToken() 531 | p.nextToken() 532 | return args 533 | } 534 | 535 | p.nextToken() 536 | args = append(args, p.parseExpression(LOWEST)) 537 | 538 | for p.peekTokenIs(token.COMMA) { 539 | p.nextToken() 540 | p.nextToken() 541 | args = append(args, p.parseExpression(LOWEST)) 542 | } 543 | 544 | if !p.expectPeek(token.RPAREN) { 545 | return nil 546 | } 547 | 548 | return args 549 | } 550 | 551 | func (p *Parser) parseExpressionAction() *ast.ExpressionAction { 552 | action := &ast.ExpressionAction{} 553 | action.Expression = p.parseExpression(LOWEST) 554 | return action 555 | } 556 | -------------------------------------------------------------------------------- /ted/runner/runner.go: -------------------------------------------------------------------------------- 1 | package runner 2 | 3 | import ( 4 | "bytes" 5 | "fmt" 6 | "io" 7 | "os" 8 | "regexp" 9 | "slices" 10 | "strconv" 11 | "strings" 12 | "text/template" 13 | 14 | "github.com/ahalbert/ted/ted/ast" 15 | "github.com/edsrzf/mmap-go" 16 | "github.com/rwtodd/Go.Sed/sed" 17 | ) 18 | 19 | type Runner struct { 20 | States map[string]*State 21 | Variables map[string]string 22 | Functions map[string]*ast.FunctionLiteral 23 | StartState string 24 | CurrState string 25 | DidTransition bool 26 | DidResetUnderscoreVar bool 27 | CaptureMode string 28 | CaptureVar string 29 | Tape Tape 30 | OutputTape io.Writer 31 | ShouldHalt bool 32 | DidFatalError bool 33 | } 34 | 35 | type State struct { 36 | StateName string 37 | NextState string 38 | Actions []ast.Action 39 | } 40 | 41 | func NewRunner(fsa ast.FSA, vars map[string]string) *Runner { 42 | r := &Runner{ 43 | States: make(map[string]*State), 44 | Variables: vars, 45 | Functions: make(map[string]*ast.FunctionLiteral), 46 | } 47 | r.States["0"] = newState("0") 48 | r.Variables["$_"] = "" 49 | 50 | _, ok := r.Variables["$RS"] 51 | if !ok { 52 | r.Variables["$RS"] = "\n" 53 | } 54 | _, ok = r.Variables["$PRINTMODE"] 55 | if !ok { 56 | r.Variables["$PRINTMODE"] = "print" 57 | } 58 | 59 | for idx, statement := range fsa.Statements { 60 | switch statement.(type) { 61 | case *ast.StateStatement: 62 | r.processStateStatement(statement.(*ast.StateStatement), getNextStateInList(fsa.Statements, idx, statement.(*ast.StateStatement).StateName)) 63 | case *ast.FunctionStatement: 64 | r.processFunctionStatement(statement.(*ast.FunctionStatement)) 65 | } 66 | } 67 | return r 68 | } 69 | 70 | func getNextStateInList(statements []ast.Statement, idx int, currState string) string { 71 | if idx+1 >= len(statements) { 72 | return "0" 73 | } 74 | 75 | found := false 76 | for _, statement := range statements[idx:] { 77 | switch statement.(type) { 78 | case *ast.StateStatement: 79 | stmt := statement.(*ast.StateStatement) 80 | if stmt.StateName == currState { 81 | found = true 82 | } else if found && stmt.StateName != "BEGIN" && stmt.StateName != "END" && stmt.StateName != "ALL" { 83 | return stmt.StateName 84 | } 85 | } 86 | } 87 | return "0" 88 | } 89 | 90 | func (r *Runner) processStateStatement(statement *ast.StateStatement, nextState string) { 91 | if r.StartState == "" && statement.StateName != "BEGIN" && statement.StateName != "END" && statement.StateName != "ALL" { 92 | r.StartState = statement.StateName 93 | } 94 | _, ok := r.States[statement.StateName] 95 | if !ok { 96 | r.States[statement.StateName] = newState(statement.StateName) 97 | } 98 | state, _ := r.States[statement.StateName] 99 | 100 | if state.NextState == "" { 101 | state.NextState = nextState 102 | } 103 | state.addRule(statement.Action) 104 | 105 | } 106 | 107 | func (r *Runner) processFunctionStatement(statement *ast.FunctionStatement) { 108 | switch statement.Function.(type) { 109 | case *ast.FunctionLiteral: 110 | fn := statement.Function.(*ast.FunctionLiteral) 111 | r.Functions[statement.Name] = fn 112 | default: 113 | panic("non-function expr") 114 | } 115 | 116 | } 117 | 118 | func newState(stateName string) *State { 119 | return &State{StateName: stateName, 120 | Actions: []ast.Action{}, 121 | NextState: "", 122 | } 123 | } 124 | 125 | func (s *State) addRule(action ast.Action) { 126 | s.Actions = append(s.Actions, action) 127 | } 128 | 129 | func (r *Runner) RunFSAFromString(input string, out io.Writer) { 130 | r.Tape = NewStringTape(input) 131 | r.OutputTape = out 132 | r.RunFSA() 133 | } 134 | 135 | func (r *Runner) RunFSAFromFile(in *os.File, out io.Writer) { 136 | mmap, err := mmap.Map(in, mmap.RDONLY, 0) 137 | if err != nil { 138 | panic("mmap error") 139 | } 140 | defer mmap.Unmap() 141 | r.Tape = NewReversibleScanner(mmap) 142 | r.OutputTape = out 143 | r.RunFSA() 144 | } 145 | 146 | func (r *Runner) RunFSA() { 147 | r.DidFatalError = false 148 | 149 | if r.StartState == "" { 150 | r.StartState = "0" 151 | } 152 | r.CurrState = r.StartState 153 | //Run BEGIN State - may have transitions so we should set CurrState before running any. 154 | state, ok := r.States["BEGIN"] 155 | if ok { 156 | for _, action := range state.Actions { 157 | if r.DidTransition { 158 | break 159 | } 160 | r.doAction(action) 161 | } 162 | } 163 | 164 | r.Tape.Split(r.getVariable("$RS")) 165 | 166 | if r.getVariable("$PRINTMODE") == "noprint" { 167 | r.CaptureMode = "capture" 168 | r.CaptureVar = "$NULL" 169 | } else { 170 | r.CaptureMode = "nocapture" 171 | } 172 | 173 | //Run FSA 174 | for !r.ShouldHalt { 175 | if !r.Tape.Next() { 176 | r.ShouldHalt = true 177 | break 178 | } 179 | line := r.Tape.Text() 180 | r.clearAndSetVariable("$@", line) 181 | 182 | if !(r.CaptureVar == "$_" && r.CaptureMode == "capture") { 183 | r.clearAndSetVariable("$_", r.getVariable("$@")) 184 | r.DidResetUnderscoreVar = true 185 | } else { 186 | r.DidResetUnderscoreVar = false 187 | } 188 | 189 | r.DidTransition = false 190 | state, ok := r.States[r.CurrState] 191 | if !ok { 192 | panic("missing state:" + r.CurrState) 193 | } 194 | for _, action := range state.Actions { 195 | if r.DidTransition || r.ShouldHalt { 196 | break 197 | } 198 | r.doAction(action) 199 | } 200 | 201 | r.DidTransition = false 202 | state, ok = r.States["ALL"] 203 | if ok { 204 | for _, action := range state.Actions { 205 | if r.DidTransition || r.ShouldHalt { 206 | break 207 | } 208 | r.doAction(action) 209 | } 210 | } 211 | 212 | if r.CaptureMode == "capture" { 213 | r.appendToVariable(r.CaptureVar, r.getVariable("$@")+r.getVariable("$RS")) 214 | } else if r.CaptureMode == "temp" { 215 | r.CaptureMode = "nocapture" 216 | } else if r.getVariable("$PRINTMODE") == "print" { 217 | _, err := io.WriteString(r.OutputTape, r.getVariable("$_")+r.getVariable("$RS")) 218 | if err != nil { 219 | r.fatalError(err.Error(), nil) 220 | } 221 | r.clearAndSetVariable("$_", "") 222 | } else { 223 | r.clearAndSetVariable("$_", "") 224 | } 225 | } 226 | 227 | //Run END state 228 | r.CurrState = "END" 229 | state, ok = r.States[r.CurrState] 230 | if ok && !r.DidFatalError { 231 | for _, action := range state.Actions { 232 | if r.DidFatalError || r.DidTransition { 233 | break 234 | } 235 | r.doAction(action) 236 | } 237 | } 238 | } 239 | 240 | func (r *Runner) getVariable(key string) string { 241 | val, ok := r.Variables[key] 242 | if !ok { 243 | r.fatalError("Attempted to reference non-existent variable:"+key, nil) 244 | return "" 245 | } 246 | return val 247 | } 248 | 249 | func (r *Runner) appendToVariable(key string, apendee string) string { 250 | if key == "$NULL" { 251 | return "" 252 | } 253 | val, ok := r.Variables[key] 254 | if !ok { 255 | val = "" 256 | } 257 | val = val + apendee 258 | r.Variables[key] = val 259 | return val 260 | } 261 | 262 | func (r *Runner) clearAndSetVariable(key string, toset string) { 263 | r.Variables[key] = toset 264 | } 265 | 266 | func (r *Runner) doTransition(newState string) { 267 | r.CurrState = newState 268 | r.DidTransition = true 269 | } 270 | 271 | func (r *Runner) applyVariablesToString(input string) string { 272 | var output bytes.Buffer 273 | t := template.Must(template.New("").Parse(input)) 274 | t.Execute(&output, r.Variables) 275 | return output.String() 276 | } 277 | 278 | func (r *Runner) doAction(action ast.Action) { 279 | switch action.(type) { 280 | case *ast.ActionBlock: 281 | r.doActionBlock(action.(*ast.ActionBlock)) 282 | case *ast.RegexAction: 283 | r.doRegexAction(action.(*ast.RegexAction)) 284 | case *ast.DoSedAction: 285 | r.doSedAction(action.(*ast.DoSedAction)) 286 | case *ast.DoUntilSedAction: 287 | r.doUntilSedAction(action.(*ast.DoUntilSedAction)) 288 | case *ast.GotoAction: 289 | r.doGotoAction(action.(*ast.GotoAction)) 290 | case *ast.ResetAction: 291 | r.doResetAction(action.(*ast.ResetAction)) 292 | case *ast.PrintAction: 293 | r.doPrintAction(action.(*ast.PrintAction)) 294 | case *ast.PrintLnAction: 295 | r.doPrintLnAction(action.(*ast.PrintLnAction)) 296 | case *ast.StartStopCaptureAction: 297 | r.doStartStopCapture(action.(*ast.StartStopCaptureAction)) 298 | case *ast.CaptureAction: 299 | r.doCaptureAction(action.(*ast.CaptureAction)) 300 | case *ast.ClearAction: 301 | r.doClearAction(action.(*ast.ClearAction)) 302 | case *ast.AssignAction: 303 | r.doAssignAction(action.(*ast.AssignAction)) 304 | case *ast.MoveHeadAction: 305 | r.doMoveHeadAction(action.(*ast.MoveHeadAction)) 306 | case *ast.IfAction: 307 | r.doIfAction(action.(*ast.IfAction)) 308 | case *ast.ExpressionAction: 309 | r.doExpressionAction(action.(*ast.ExpressionAction)) 310 | case nil: 311 | r.doNoOp() 312 | default: 313 | r.fatalError("Unknown Action!", action) 314 | } 315 | } 316 | 317 | func (r *Runner) doActionBlock(block *ast.ActionBlock) { 318 | for _, action := range block.Actions { 319 | if r.ShouldHalt && r.CurrState != "END" { 320 | break 321 | } 322 | r.doAction(action) 323 | } 324 | } 325 | 326 | func (r *Runner) doRegexAction(action *ast.RegexAction) { 327 | rule := r.applyVariablesToString(action.Rule) 328 | re, err := regexp.Compile(rule) 329 | if err != nil { 330 | r.fatalError("regexp error, supplied: "+action.Rule+"\n formatted as: "+rule, action) 331 | return 332 | } 333 | 334 | matches := re.FindStringSubmatch(r.getVariable("$@")) 335 | if matches != nil { 336 | for idx, match := range matches { 337 | stridx := "$" + strconv.Itoa(idx) 338 | r.clearAndSetVariable(stridx, match) 339 | } 340 | r.doAction(action.Action) 341 | } 342 | } 343 | 344 | func (r *Runner) doSedAction(action *ast.DoSedAction) { 345 | command := r.applyVariablesToString(action.Command) 346 | engine, err := sed.New(strings.NewReader(command)) 347 | if err != nil { 348 | r.fatalError("error building sed engine with command: '"+action.Command+"'\n formatted as: '"+command+"'", action) 349 | return 350 | } 351 | result, err := engine.RunString(r.getVariable(action.Variable)) 352 | if err != nil { 353 | r.fatalError(fmt.Errorf("error running sed: %w", err).Error(), action) 354 | return 355 | } 356 | if len(result) > 0 { 357 | result = result[:len(result)-1] 358 | } 359 | if action.Variable == "$_" && r.CaptureMode != "capture" { 360 | r.clearAndSetVariable(action.Variable, result) 361 | } else { 362 | r.clearAndSetVariable(action.Variable, result+r.getVariable("$RS")) 363 | } 364 | } 365 | 366 | func (r *Runner) doUntilSedAction(action *ast.DoUntilSedAction) { 367 | command := r.applyVariablesToString(action.Command) 368 | engine, err := sed.New(strings.NewReader(command)) 369 | if err != nil { 370 | r.fatalError("error building sed engine with command: '"+action.Command+"'\n formatted as: '"+command+"'", action) 371 | return 372 | } 373 | orig := r.getVariable(action.Variable) 374 | result, err := engine.RunString(orig) 375 | if err != nil { 376 | r.fatalError(fmt.Errorf("error running sed: %w", err).Error(), action) 377 | return 378 | } 379 | result = result[:len(result)-1] 380 | if action.Variable == "$_" && r.CaptureMode != "capture" { 381 | r.clearAndSetVariable(action.Variable, result) 382 | } else { 383 | r.clearAndSetVariable(action.Variable, result+r.getVariable("$RS")) 384 | } 385 | if orig != result { 386 | r.doAction(action.Action) 387 | } 388 | } 389 | 390 | func (r *Runner) doGotoAction(action *ast.GotoAction) { 391 | if action.Target == "" { 392 | state, ok := r.States[r.CurrState] 393 | if !ok { 394 | r.fatalError(fmt.Sprintf("State %s not found", r.CurrState), action) 395 | } 396 | r.CurrState = state.NextState 397 | } else { 398 | r.CurrState = action.Target 399 | } 400 | r.DidTransition = true 401 | } 402 | 403 | func (r *Runner) doResetAction(action *ast.ResetAction) { 404 | r.CurrState = r.StartState 405 | r.DidTransition = true 406 | } 407 | 408 | func (r *Runner) doPrintAction(action *ast.PrintAction) { 409 | val := r.evaluateExpression(action.Expression) 410 | var err error 411 | switch val.(type) { 412 | case *ast.StringLiteral: 413 | _, err = io.WriteString(r.OutputTape, val.(*ast.StringLiteral).Value) 414 | case *ast.IntegerLiteral: 415 | _, err = io.WriteString(r.OutputTape, strconv.Itoa(val.(*ast.IntegerLiteral).Value)) 416 | case *ast.Boolean: 417 | if val.(*ast.Boolean).Value { 418 | _, err = io.WriteString(r.OutputTape, "true") 419 | } else { 420 | _, err = io.WriteString(r.OutputTape, "false") 421 | } 422 | default: 423 | r.fatalError(fmt.Sprintf("cannot print type %v", val), action) 424 | return 425 | } 426 | if err != nil { 427 | r.fatalError(err.Error(), action) 428 | } 429 | } 430 | 431 | func (r *Runner) doPrintLnAction(action *ast.PrintLnAction) { 432 | val := r.evaluateExpression(action.Expression) 433 | var err error 434 | switch val.(type) { 435 | case *ast.StringLiteral: 436 | _, err = io.WriteString(r.OutputTape, val.(*ast.StringLiteral).Value+"\n") 437 | case *ast.IntegerLiteral: 438 | _, err = io.WriteString(r.OutputTape, strconv.Itoa(val.(*ast.IntegerLiteral).Value)+"\n") 439 | case *ast.Boolean: 440 | if val.(*ast.Boolean).Value { 441 | _, err = io.WriteString(r.OutputTape, "true"+"\n") 442 | } else { 443 | _, err = io.WriteString(r.OutputTape, "false"+"\n") 444 | } 445 | default: 446 | r.fatalError(fmt.Sprintf("cannot print type %v", val), action) 447 | } 448 | if err != nil { 449 | r.fatalError(err.Error(), action) 450 | } 451 | } 452 | 453 | func (r *Runner) doCaptureAction(action *ast.CaptureAction) { 454 | if action.Variable == "$_" && r.DidResetUnderscoreVar { 455 | r.clearAndSetVariable("$_", "") 456 | } 457 | r.appendToVariable(action.Variable, r.getVariable("$@")) 458 | r.CaptureMode = "temp" 459 | } 460 | 461 | func (r *Runner) doStartStopCapture(action *ast.StartStopCaptureAction) { 462 | if action.Command == "start" { 463 | if action.Variable == "$_" { 464 | r.clearAndSetVariable("$_", "") 465 | } 466 | r.CaptureMode = "capture" 467 | r.CaptureVar = action.Variable 468 | } else if action.Command == "stop" { 469 | r.CaptureMode = "nocapture" 470 | } else { 471 | r.fatalError("unknown command: "+action.Command+" in start/stop action", action) 472 | return 473 | } 474 | } 475 | 476 | func (r *Runner) doClearAction(action *ast.ClearAction) { 477 | if action.Variable == "$_" { 478 | r.CaptureMode = "nocapture" 479 | } 480 | r.clearAndSetVariable(action.Variable, "") 481 | } 482 | 483 | func (r *Runner) doAssignAction(action *ast.AssignAction) { 484 | val := r.evaluateExpression(action.Expression).String() //TODO: check if this is safe 485 | r.Variables[action.Target] = val 486 | } 487 | 488 | func (r *Runner) evaluateExpression(expression ast.Expression) ast.Expression { 489 | switch expression.(type) { 490 | case *ast.Boolean: 491 | return expression 492 | case *ast.IntegerLiteral: 493 | return expression 494 | case *ast.StringLiteral: 495 | return expression 496 | case *ast.Identifier: 497 | return &ast.StringLiteral{Value: r.getVariable(expression.(*ast.Identifier).Value)} 498 | case *ast.PrefixExpression: 499 | return r.evaluatePrefixExpression(expression.(*ast.PrefixExpression)) 500 | case *ast.InfixExpression: 501 | return r.evaluateInfixExpression(expression.(*ast.InfixExpression)) 502 | case *ast.CallExpression: 503 | return r.evaluateCallExpression(expression.(*ast.CallExpression)) 504 | } 505 | return nil 506 | } 507 | 508 | func (r *Runner) evaluatePrefixExpression(expression *ast.PrefixExpression) ast.Expression { 509 | right := r.evaluateExpression(expression.Right) 510 | switch expression.Operator { 511 | case "!": 512 | switch right.(type) { 513 | case *ast.Boolean: 514 | return &ast.Boolean{Value: !right.(*ast.Boolean).Value} 515 | default: 516 | r.fatalError("! operation expects boolean.", &ast.ExpressionAction{Expression: right}) 517 | } 518 | case "-": 519 | switch right.(type) { 520 | case *ast.IntegerLiteral: 521 | return &ast.IntegerLiteral{Value: -1 * right.(*ast.IntegerLiteral).Value} 522 | default: 523 | r.fatalError("- operation expects integer.", &ast.ExpressionAction{Expression: right}) 524 | } 525 | } 526 | return nil 527 | } 528 | 529 | func (r *Runner) evaluateInfixExpression(expression *ast.InfixExpression) ast.Expression { 530 | left := r.evaluateExpression(expression.Left) 531 | right := r.evaluateExpression(expression.Right) 532 | if slices.Contains([]string{"+", "-", "*", "/"}, expression.Operator) { 533 | return r.evaluateArithmetic(left, right, expression.Operator) 534 | 535 | } else if slices.Contains([]string{">", "<", "!=", "=="}, expression.Operator) { 536 | result, err := r.tryCompareInt(left, right, expression.Operator) 537 | if err == nil { 538 | return result 539 | } 540 | result, err = r.tryCompareBool(left, right, expression.Operator) 541 | if err == nil { 542 | return result 543 | } 544 | result, err = r.tryCompareString(left, right, expression.Operator) 545 | if err == nil { 546 | return result 547 | } 548 | r.fatalError("unable to make comparison", &ast.ExpressionAction{Expression: expression}) 549 | } 550 | return nil 551 | } 552 | 553 | func (r *Runner) evaluateArithmetic(left ast.Expression, right ast.Expression, op string) ast.Expression { 554 | l_int, _ := r.convertToInt(left) 555 | r_int, r_err := r.convertToInt(right) 556 | switch op { 557 | case "+": 558 | return &ast.IntegerLiteral{Value: l_int + r_int} 559 | case "-": 560 | return &ast.IntegerLiteral{Value: l_int - r_int} 561 | case "*": 562 | return &ast.IntegerLiteral{Value: l_int * r_int} 563 | case "/": 564 | if r_err != nil { 565 | return &ast.IntegerLiteral{Value: 0} 566 | } 567 | return &ast.IntegerLiteral{Value: l_int / r_int} 568 | } 569 | return nil 570 | } 571 | 572 | func (r *Runner) convertToInt(expression ast.Expression) (int, error) { 573 | switch expression.(type) { 574 | case *ast.StringLiteral: 575 | val, err := strconv.Atoi(expression.(*ast.StringLiteral).Value) 576 | if err != nil { 577 | return 0, fmt.Errorf("type error expected int or string-like int") 578 | } 579 | return val, nil 580 | case *ast.IntegerLiteral: 581 | return expression.(*ast.IntegerLiteral).Value, nil 582 | default: 583 | return 0, fmt.Errorf("type error expected int or string-like int") 584 | } 585 | } 586 | 587 | func (r *Runner) tryCompareInt(left ast.Expression, right ast.Expression, op string) (ast.Expression, error) { 588 | l_int, l_err := r.convertToInt(left) 589 | r_int, r_err := r.convertToInt(right) 590 | if l_err != nil || r_err != nil { 591 | return nil, fmt.Errorf("unable to convert to int") 592 | } 593 | switch op { 594 | case ">": 595 | return &ast.Boolean{Value: l_int > r_int}, nil 596 | case "<": 597 | return &ast.Boolean{Value: l_int < r_int}, nil 598 | case "==": 599 | return &ast.Boolean{Value: l_int == r_int}, nil 600 | case "!=": 601 | return &ast.Boolean{Value: l_int != r_int}, nil 602 | } 603 | return nil, fmt.Errorf("unknown operator") 604 | } 605 | 606 | func (r *Runner) convertToBool(expression ast.Expression) (bool, error) { 607 | switch expression.(type) { 608 | case *ast.StringLiteral: 609 | val := expression.(*ast.StringLiteral).Value 610 | if val == "true" { 611 | return true, nil 612 | } 613 | if val == "false" { 614 | return false, nil 615 | } 616 | return false, fmt.Errorf("unable to convert to bool.") 617 | case *ast.Boolean: 618 | return expression.(*ast.Boolean).Value, nil 619 | default: 620 | return false, fmt.Errorf("type error expected bool or string-like bool") 621 | } 622 | } 623 | 624 | func (r *Runner) tryCompareBool(left ast.Expression, right ast.Expression, op string) (ast.Expression, error) { 625 | lbool, l_err := r.convertToBool(left) 626 | rbool, r_err := r.convertToBool(right) 627 | if l_err != nil || r_err != nil { 628 | return nil, fmt.Errorf("unable to convert to int") 629 | } 630 | switch op { 631 | case ">": 632 | return &ast.Boolean{Value: false}, fmt.Errorf("> not compatible with bool compare") 633 | case "<": 634 | return &ast.Boolean{Value: false}, fmt.Errorf("< not compatible with bool compare") 635 | case "==": 636 | return &ast.Boolean{Value: lbool == rbool}, nil 637 | case "!=": 638 | return &ast.Boolean{Value: lbool != rbool}, nil 639 | } 640 | return nil, fmt.Errorf("unknown operator") 641 | } 642 | 643 | func (r *Runner) tryCompareString(left ast.Expression, right ast.Expression, op string) (ast.Expression, error) { 644 | leftStr := left.(*ast.StringLiteral).Value 645 | rightStr := right.(*ast.StringLiteral).Value 646 | switch op { 647 | case ">": 648 | return &ast.Boolean{Value: leftStr > rightStr}, nil 649 | case "<": 650 | return &ast.Boolean{Value: leftStr < rightStr}, nil 651 | case "==": 652 | return &ast.Boolean{Value: leftStr == rightStr}, nil 653 | case "!=": 654 | return &ast.Boolean{Value: leftStr != rightStr}, nil 655 | } 656 | return nil, fmt.Errorf("unknown operator") 657 | } 658 | 659 | func (r *Runner) evaluateCallExpression(expression *ast.CallExpression) ast.Expression { 660 | switch expression.Function.(type) { 661 | case *ast.Identifier: 662 | r.lookupAndEvaluateFunction(expression) 663 | case *ast.FunctionLiteral: 664 | r.evaluateFunctionLiteral(expression) 665 | } 666 | return expression 667 | } 668 | 669 | func (r *Runner) lookupAndEvaluateFunction(expression *ast.CallExpression) { 670 | fnName := expression.Function.(*ast.Identifier).Value 671 | fn, ok := r.Functions[fnName] 672 | if !ok { 673 | r.fatalError("function "+fnName+" not found!", &ast.ExpressionAction{Expression: expression}) 674 | return 675 | } 676 | r.doAction(fn.Body) 677 | } 678 | 679 | func (r *Runner) evaluateFunctionLiteral(expression *ast.CallExpression) { 680 | function := expression.Function.(*ast.FunctionLiteral) 681 | r.doAction(function.Body) 682 | } 683 | 684 | func (r *Runner) doMoveHeadAction(action *ast.MoveHeadAction) { 685 | if action.Command == "fastforward" { 686 | r.doFastForward(action.Regex) 687 | } else if action.Command == "rewind" { 688 | r.doRewind(action.Regex) 689 | } 690 | } 691 | 692 | func (r *Runner) doFastForward(target string) { 693 | rule := r.applyVariablesToString(target) 694 | re, err := regexp.Compile(rule) 695 | if err != nil { 696 | r.fatalError(err.Error(), nil) 697 | } 698 | line := "" 699 | for ok := true; ok; ok = (!re.MatchString(line)) { 700 | if !r.Tape.Next() { 701 | r.ShouldHalt = true 702 | return 703 | } 704 | line = r.Tape.Text() 705 | } 706 | r.clearAndSetVariable("$@", line) 707 | r.Tape.Prev() 708 | } 709 | 710 | func (r *Runner) doRewind(target string) { 711 | rule := r.applyVariablesToString(target) 712 | re, err := regexp.Compile(rule) 713 | if err != nil { 714 | r.fatalError(err.Error(), nil) 715 | } 716 | line := "" 717 | for ok := true; ok; ok = (!re.MatchString(line)) { 718 | if !r.Tape.Prev() { 719 | return 720 | } 721 | line = r.Tape.Text() 722 | } 723 | r.clearAndSetVariable("$@", line) 724 | r.Tape.Prev() 725 | } 726 | 727 | func (r *Runner) doIfAction(action *ast.IfAction) { 728 | exprResult := false 729 | result := r.evaluateExpression(action.Condition) 730 | switch result.(type) { 731 | case *ast.Boolean: 732 | exprResult = result.(*ast.Boolean).Value 733 | default: 734 | r.fatalError("type error expected bool in if statement", action) 735 | } 736 | if exprResult { 737 | r.doAction(action.Consequence) 738 | } else if action.Alternative != nil { 739 | r.doAction(action.Alternative) 740 | } 741 | } 742 | 743 | func (r *Runner) doExpressionAction(action *ast.ExpressionAction) { 744 | r.evaluateExpression(action.Expression) 745 | } 746 | 747 | func (r *Runner) doNoOp() {} 748 | 749 | func (r *Runner) fatalError(msg string, action ast.Action) { 750 | r.ShouldHalt = true 751 | r.DidFatalError = true 752 | if r.CurrState == "" { 753 | r.CurrState = "INIT" 754 | } 755 | _, err := io.WriteString(r.OutputTape, "Runtime Error in state: "+r.CurrState+"\n") 756 | if err != nil { 757 | panic(err) 758 | } 759 | if action != nil { 760 | _, err = io.WriteString(r.OutputTape, "Action: "+action.String()+"\n") 761 | if err != nil { 762 | panic(err) 763 | } 764 | } 765 | _, err = io.WriteString(r.OutputTape, msg+"\n") 766 | if err != nil { 767 | panic(err) 768 | } 769 | } 770 | -------------------------------------------------------------------------------- /ted/runner/tape.go: -------------------------------------------------------------------------------- 1 | package runner 2 | 3 | import ( 4 | "errors" 5 | "io" 6 | "strings" 7 | "unicode/utf8" 8 | 9 | "github.com/edsrzf/mmap-go" 10 | ) 11 | 12 | type Tape interface { 13 | Split(seperator string) 14 | Scan() bool 15 | Text() string 16 | Seek(int, int) (int, error) 17 | Prev() bool 18 | Next() bool 19 | } 20 | 21 | var ErrEof = errors.New("EOF") 22 | var ErrBof = errors.New("BOF") 23 | 24 | type StringTape struct { 25 | input string 26 | groups []string 27 | offset int 28 | seperator string 29 | } 30 | 31 | func NewStringTape(in string) *StringTape { 32 | return &StringTape{input: in, 33 | offset: -1, 34 | seperator: "\n", 35 | groups: strings.Split(in, "\n"), 36 | } 37 | } 38 | 39 | func (ss *StringTape) Text() string { 40 | return ss.groups[ss.offset] 41 | } 42 | 43 | func (ss *StringTape) Split(seperator string) { 44 | ss.seperator = seperator 45 | ss.groups = strings.Split(ss.input, ss.seperator) 46 | } 47 | 48 | func (ss *StringTape) Prev() bool { 49 | if ss.offset <= 0 { 50 | return false 51 | } 52 | ss.offset-- 53 | return true 54 | } 55 | 56 | func (ss *StringTape) Next() bool { 57 | ss.offset++ 58 | if ss.offset >= len(ss.groups) { 59 | return false 60 | } 61 | return true 62 | } 63 | 64 | func (ss *StringTape) Scan() bool { 65 | ss.offset++ 66 | if ss.offset >= len(ss.groups) { 67 | return false 68 | } 69 | return true 70 | } 71 | 72 | func (ss *StringTape) Seek(offset int, whence int) (int, error) { 73 | var whenceOffset int 74 | switch whence { 75 | case io.SeekStart: 76 | whenceOffset = 0 77 | case io.SeekCurrent: 78 | whenceOffset = ss.offset 79 | case io.SeekEnd: 80 | whenceOffset = len(ss.groups) 81 | } 82 | newOffset := offset + whenceOffset 83 | if newOffset >= len(ss.groups) { 84 | return 0, ErrEof 85 | } 86 | if newOffset < 0 { 87 | return 0, ErrBof 88 | } 89 | ss.offset = newOffset 90 | return newOffset, nil 91 | } 92 | 93 | type stringPosition struct { 94 | begin int 95 | end int 96 | } 97 | 98 | type ReversibleScanner struct { 99 | mmap mmap.MMap 100 | pos int 101 | curr string 102 | offsets map[int]stringPosition 103 | seperator string 104 | offset int 105 | maxOffset int 106 | readAll bool 107 | } 108 | 109 | func NewReversibleScanner(m mmap.MMap) *ReversibleScanner { 110 | ofs := make(map[int]stringPosition) 111 | return &ReversibleScanner{mmap: m, 112 | pos: 0, 113 | offsets: ofs, 114 | seperator: "\n", 115 | readAll: false, 116 | offset: -1, 117 | maxOffset: -1, 118 | } 119 | } 120 | 121 | func (rs *ReversibleScanner) Split(sep string) { 122 | rs.pos = 0 123 | rs.seperator = sep 124 | rs.offsets = make(map[int]stringPosition) 125 | rs.readAll = false 126 | rs.offset = -1 127 | rs.maxOffset = -1 128 | } 129 | 130 | func (rs *ReversibleScanner) Text() string { 131 | return rs.curr 132 | } 133 | 134 | func (rs *ReversibleScanner) Prev() bool { 135 | rs.offset-- 136 | if rs.offset < 0 { 137 | rs.curr = "" 138 | return false 139 | } 140 | _, err := rs.Seek(rs.offset, io.SeekStart) 141 | if err != nil { 142 | return false 143 | } 144 | return true 145 | } 146 | 147 | func (rs *ReversibleScanner) Next() bool { 148 | rs.offset++ 149 | _, err := rs.Seek(rs.offset, io.SeekStart) 150 | if err != nil { 151 | return false 152 | } 153 | return true 154 | } 155 | 156 | func (rs *ReversibleScanner) Seek(offset int, whence int) (int, error) { 157 | var whenceOffset int 158 | switch whence { 159 | case io.SeekStart: 160 | whenceOffset = 0 161 | case io.SeekCurrent: 162 | whenceOffset = rs.offset 163 | case io.SeekEnd: 164 | for rs.Scan() { 165 | } 166 | whenceOffset = rs.maxOffset 167 | } 168 | newOffset := offset + whenceOffset 169 | if newOffset < 0 { 170 | return 0, ErrBof 171 | } 172 | if newOffset > rs.maxOffset { 173 | for rs.Scan() && newOffset > rs.maxOffset { 174 | } 175 | if newOffset > rs.maxOffset { 176 | return 0, ErrEof 177 | } 178 | } 179 | position, ok := rs.offsets[newOffset] 180 | if ok { 181 | rs.offset = newOffset 182 | rs.curr = string(rs.mmap[position.begin:position.end]) 183 | return int(position.begin), nil 184 | } 185 | return 0, errors.New("Could not find offset") 186 | } 187 | 188 | func (rs *ReversibleScanner) Scan() bool { 189 | if rs.readAll { 190 | return false 191 | } 192 | rs.curr = "" 193 | var begin int 194 | currentPos, ok := rs.offsets[rs.maxOffset] 195 | if ok { 196 | begin = int(currentPos.end) + len(rs.seperator) 197 | rs.pos = begin 198 | } else { 199 | begin = 0 200 | } 201 | end := begin 202 | for rs.curr[max(0, len(rs.curr)-len(rs.seperator)):] != rs.seperator { 203 | nextRune, size, err := rs.readRune() 204 | if err != nil { 205 | rs.readAll = true 206 | if errors.Is(err, ErrEof) { 207 | return false 208 | } 209 | if begin == end { 210 | return false 211 | } 212 | } 213 | rs.curr += string(nextRune) 214 | rs.pos += size 215 | end += size 216 | } 217 | rs.maxOffset++ 218 | rs.curr = rs.curr[:len(rs.curr)-len(rs.seperator)] 219 | rs.offsets[rs.maxOffset] = stringPosition{begin: begin, end: end - len(rs.seperator)} 220 | return true 221 | } 222 | 223 | func (rs *ReversibleScanner) readRune() (rune, int, error) { 224 | if rs.pos >= len(rs.mmap) { 225 | return rune(0), 0, ErrEof 226 | } 227 | 228 | if rs.mmap[rs.pos] < utf8.RuneSelf { 229 | return rune(rs.mmap[rs.pos]), 1, nil 230 | } 231 | 232 | r, width := utf8.DecodeRune(rs.mmap[rs.pos:]) 233 | if width > 1 { 234 | return r, width, nil 235 | } 236 | return utf8.RuneError, 1, nil 237 | 238 | } 239 | -------------------------------------------------------------------------------- /ted/token/tokens.go: -------------------------------------------------------------------------------- 1 | package token 2 | 3 | type TokenType string 4 | 5 | type Token struct { 6 | Type TokenType 7 | Literal string 8 | LineNum int 9 | Position int 10 | } 11 | 12 | const ( 13 | ILLEGAL = "ILLEGAL" 14 | EOF = "EOF" 15 | 16 | //Identfiers 17 | IDENT = "IDENT" 18 | REGEX = "REGEX" 19 | STRING = "STRING" 20 | 21 | //symbols 22 | 23 | COLON = ":" 24 | ASSIGN = "=" 25 | SEMICOLON = ";" 26 | COMMA = "," 27 | GOTO = "->" 28 | RESET = "-->" 29 | LBRACE = "{" 30 | RBRACE = "}" 31 | 32 | PLUS = "+" 33 | MINUS = "-" 34 | BANG = "!" 35 | ASTERISK = "*" 36 | SLASH = "/" 37 | 38 | LT = "<" 39 | GT = ">" 40 | 41 | EQ = "==" 42 | NOT_EQ = "!=" 43 | 44 | LPAREN = "(" 45 | RPAREN = ")" 46 | 47 | //Keywords 48 | DO = "DO" 49 | DOUNTIL = "DOUNTIL" 50 | START = "START" 51 | STOP = "STOP" 52 | CAPTURE = "CAPTURE" 53 | LABEL = "LABEL" 54 | LET = "LET" 55 | PRINT = "PRINT" 56 | PRINTLN = "PRINTLN" 57 | CLEAR = "CLEAR" 58 | REWIND = "REWIND" 59 | FASTFWD = "FASTFORWARD" 60 | PAUSE = "PAUSE" 61 | PLAY = "PLAY" 62 | IF = "IF" 63 | ELSE = "ELSE" 64 | RETURN = "RETURN" 65 | FUNCTION = "FUNCTION" 66 | ) 67 | 68 | var keywords = map[string]TokenType{ 69 | "do": DO, 70 | "dountil": DOUNTIL, 71 | "capture": CAPTURE, 72 | "print": PRINT, 73 | "println": PRINTLN, 74 | "start": START, 75 | "stop": STOP, 76 | "clear": CLEAR, 77 | "let": LET, 78 | "rewind": REWIND, 79 | "fastforward": FASTFWD, 80 | "pause": PAUSE, 81 | "play": PLAY, 82 | "if": IF, 83 | "else": ELSE, 84 | "function": FUNCTION, 85 | "return": RETURN, 86 | } 87 | 88 | func LookupIdent(ident string) TokenType { 89 | if tok, ok := keywords[ident]; ok { 90 | return tok 91 | } 92 | return IDENT 93 | } 94 | -------------------------------------------------------------------------------- /tests/defaults/capture.fsa: -------------------------------------------------------------------------------- 1 | /beep/ -> 2 | {capture mycapture -> } 3 | do s/THIS.IS.CAPTURED/CAPTURED/ mycapture, /buzz/ -> 4 | print mycapture 5 | -------------------------------------------------------------------------------- /tests/defaults/capture.in: -------------------------------------------------------------------------------- 1 | beep 2 | THIS IS CAPTURED 3 | boop 4 | foo 5 | buzz 6 | bar 7 | baz 8 | -------------------------------------------------------------------------------- /tests/defaults/capture.out: -------------------------------------------------------------------------------- 1 | beep 2 | boop 3 | foo 4 | buzz 5 | CAPTURED 6 | bar 7 | CAPTURED 8 | baz 9 | -------------------------------------------------------------------------------- /tests/defaults/capture_variables.fsa: -------------------------------------------------------------------------------- 1 | /beep/ -> 2 | /boop/ {capture let myvar = $_ ->} 3 | /buzz/ {println myvar} 4 | -------------------------------------------------------------------------------- /tests/defaults/capture_variables.in: -------------------------------------------------------------------------------- 1 | beep 2 | boop 3 | foo 4 | bar 5 | baz 6 | buzz 7 | -------------------------------------------------------------------------------- /tests/defaults/capture_variables.out: -------------------------------------------------------------------------------- 1 | beep 2 | foo 3 | bar 4 | baz 5 | boop 6 | buzz 7 | -------------------------------------------------------------------------------- /tests/defaults/dountil.fsa: -------------------------------------------------------------------------------- 1 | dountil s/buzz/boop/ -> 2 | -------------------------------------------------------------------------------- /tests/defaults/dountil.in: -------------------------------------------------------------------------------- 1 | beep 2 | boop 3 | buzz 4 | buzz 5 | buzz 6 | buzz 7 | -------------------------------------------------------------------------------- /tests/defaults/dountil.out: -------------------------------------------------------------------------------- 1 | beep 2 | boop 3 | boop 4 | buzz 5 | buzz 6 | buzz 7 | -------------------------------------------------------------------------------- /tests/defaults/empty_line.fsa: -------------------------------------------------------------------------------- 1 | do s/foo/bar/ 2 | -------------------------------------------------------------------------------- /tests/defaults/empty_line.in: -------------------------------------------------------------------------------- 1 | foo 2 | bar 3 | 4 | foo 5 | bar 6 | -------------------------------------------------------------------------------- /tests/defaults/empty_line.out: -------------------------------------------------------------------------------- 1 | bar 2 | bar 3 | 4 | bar 5 | bar 6 | -------------------------------------------------------------------------------- /tests/defaults/example_1.fsa: -------------------------------------------------------------------------------- 1 | # Welcome to Ted! 2 | /foo/ -> /bar/ -> do s/baz/bang/ 3 | -------------------------------------------------------------------------------- /tests/defaults/example_1.in: -------------------------------------------------------------------------------- 1 | baz 2 | foo 3 | baz 4 | bar 5 | baz 6 | -------------------------------------------------------------------------------- /tests/defaults/example_1.out: -------------------------------------------------------------------------------- 1 | baz 2 | foo 3 | baz 4 | bar 5 | bang 6 | -------------------------------------------------------------------------------- /tests/defaults/example_3.fsa: -------------------------------------------------------------------------------- 1 | /beep/ -> {/boop/ -> /buzz/ -> 1} {do s/cheater/nose/ /buzz/ -> 1} 2 | -------------------------------------------------------------------------------- /tests/defaults/example_3.in: -------------------------------------------------------------------------------- 1 | beep 2 | boop 3 | buzz 4 | cheater 5 | beep 6 | boop 7 | cheater 8 | -------------------------------------------------------------------------------- /tests/defaults/example_3.out: -------------------------------------------------------------------------------- 1 | beep 2 | boop 3 | buzz 4 | cheater 5 | beep 6 | boop 7 | nose 8 | -------------------------------------------------------------------------------- /tests/defaults/fastforward.fsa: -------------------------------------------------------------------------------- 1 | { capture fastforward /buzz/ -> } 2 | -------------------------------------------------------------------------------- /tests/defaults/fastforward.in: -------------------------------------------------------------------------------- 1 | beep 2 | boop 3 | buzz 4 | foo 5 | bar 6 | baz 7 | -------------------------------------------------------------------------------- /tests/defaults/fastforward.out: -------------------------------------------------------------------------------- 1 | buzz 2 | foo 3 | bar 4 | baz 5 | -------------------------------------------------------------------------------- /tests/defaults/flags: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/ahalbert/ted/b25804cea6383764194bc03ad5735b783f21fa6c/tests/defaults/flags -------------------------------------------------------------------------------- /tests/defaults/rewind.fsa: -------------------------------------------------------------------------------- 1 | /buzz/ { rewind /beep/ -> } 2 | -------------------------------------------------------------------------------- /tests/defaults/rewind.in: -------------------------------------------------------------------------------- 1 | beep 2 | boop 3 | buzz 4 | -------------------------------------------------------------------------------- /tests/defaults/rewind.out: -------------------------------------------------------------------------------- 1 | beep 2 | boop 3 | buzz 4 | beep 5 | boop 6 | buzz 7 | -------------------------------------------------------------------------------- /tests/noprint/begin_and_end.fsa: -------------------------------------------------------------------------------- 1 | BEGIN: { println "hello" } 2 | END: { println "buzz" } 3 | -------------------------------------------------------------------------------- /tests/noprint/begin_and_end.in: -------------------------------------------------------------------------------- 1 |
2 |
3 |
4 |
5 | target 6 |
7 |
8 |
9 |
10 | -------------------------------------------------------------------------------- /tests/noprint/begin_and_end.out: -------------------------------------------------------------------------------- 1 | hello 2 | buzz 3 | -------------------------------------------------------------------------------- /tests/noprint/capture.fsa: -------------------------------------------------------------------------------- 1 | /beep/ -> 2 | /boop/ {start capture ->} 3 | /baz/ {stop capture print -> 1} 4 | -------------------------------------------------------------------------------- /tests/noprint/capture.in: -------------------------------------------------------------------------------- 1 | beep 2 | boop - CAPTURED 3 | foo - CAPTURED 4 | bar - CAPTURED 5 | baz 6 | buzz 7 | -------------------------------------------------------------------------------- /tests/noprint/capture.out: -------------------------------------------------------------------------------- 1 | boop - CAPTURED 2 | foo - CAPTURED 3 | bar - CAPTURED 4 | -------------------------------------------------------------------------------- /tests/noprint/capture_groups.fsa: -------------------------------------------------------------------------------- 1 | /i.*want.(these).and.(those)/ -> 2 | {println $1 println $2 ->} 3 | -------------------------------------------------------------------------------- /tests/noprint/capture_groups.in: -------------------------------------------------------------------------------- 1 | beep 2 | boop 3 | i want these and those 4 | foo 5 | bar 6 | baz 7 | buzz 8 | -------------------------------------------------------------------------------- /tests/noprint/capture_groups.out: -------------------------------------------------------------------------------- 1 | these 2 | those 3 | -------------------------------------------------------------------------------- /tests/noprint/example_2.fsa: -------------------------------------------------------------------------------- 1 | /baz/ -> /baz/ -> 1 2: println 2 | -------------------------------------------------------------------------------- /tests/noprint/example_2.in: -------------------------------------------------------------------------------- 1 | DO NOT PRINT THIS LINE 2 | baz - DO NOT PRINT THIS EITHER 3 | foo 4 | bar 5 | baz - DO NOT PRINT THIS EITHER 6 | DO NOT PRINT THIS LINE 7 | -------------------------------------------------------------------------------- /tests/noprint/example_2.out: -------------------------------------------------------------------------------- 1 | foo 2 | bar 3 | -------------------------------------------------------------------------------- /tests/noprint/expressions.fsa: -------------------------------------------------------------------------------- 1 | 1: /target/ -> 2 | 2: {let count = 3 rewind /div/ -> } 3 | 3: {if count == 0 { -> } else { let count = count - 1 rewind /div/ } } 4 | 4: { let count = 5 start capture myvar -> } 5 | 5: { println count /div/ let count = count - 1 if count == 0 -> } 6 | 6: {print myvar ->} 7 | -------------------------------------------------------------------------------- /tests/noprint/expressions.in: -------------------------------------------------------------------------------- 1 |
2 |
3 |
4 |
5 | target 6 |
7 |
8 |
9 |
10 | -------------------------------------------------------------------------------- /tests/noprint/expressions.out: -------------------------------------------------------------------------------- 1 | 5 2 | 4 3 | 3 4 | 3 5 | 2 6 | 1 7 |
8 |
9 |
10 | target 11 |
12 |
13 |
14 | -------------------------------------------------------------------------------- /tests/noprint/flags: -------------------------------------------------------------------------------- 1 | --no-print 2 | -------------------------------------------------------------------------------- /tests/noprint/motivation.fsa: -------------------------------------------------------------------------------- 1 | startstate: /Starting.Procedure/ -> capturebegin 2 | capturebegin: { start capture -> lookforsuccessorending /Success/ -> startstate} 3 | lookforsuccessorending: /Success/ {stop capture -> startstate } 4 | lookforsuccessorending: /Ending.Procedure/ { stop capture print -> startstate } 5 | -------------------------------------------------------------------------------- /tests/noprint/motivation.in: -------------------------------------------------------------------------------- 1 | INFO:2024-12-07 13:01:40:Trace:198d079c-af9a-45b2-8236-7fbb2a012f69:Starting... 2 | INFO:2024-12-07 13:01:40:Trace:198d079c-af9a-45b2-8236-7fbb2a012f69:Starting Procedure foo 3 | ERROR:2024-12-07 13:01:41:Trace:198d079c-af9a-45b2-8236-7fbb2a012f69:Error 1 4 | INFO:2024-12-07 13:01:41:Trace:198d079c-af9a-45b2-8236-7fbb2a012f69:Ending Procedure foo 5 | INFO:2024-12-07 13:01:41:Trace:198d079c-af9a-45b2-8236-7fbb2a012f69:Starting Procedure bar 6 | INFO:2024-12-07 13:01:41:Trace:198d079c-af9a-45b2-8236-7fbb2a012f69:Error 2 7 | INFO:2024-12-07 13:01:41:Trace:198d079c-af9a-45b2-8236-7fbb2a012f69:Success 8 | INFO:2024-12-07 13:01:42:Trace:198d079c-af9a-45b2-8236-7fbb2a012f69:Ending Procedure bar 9 | INFO:2024-12-07 13:01:42:Trace:30019fff-7645-4d07-9fc4-0bbb39aa09db:Starting... 10 | INFO:2024-12-07 13:01:42:Trace:30019fff-7645-4d07-9fc4-0bbb39aa09db:Starting Procedure foo 11 | INFO:2024-12-07 13:01:42:Trace:30019fff-7645-4d07-9fc4-0bbb39aa09db:Success 12 | INFO:2024-12-07 13:01:42:Trace:30019fff-7645-4d07-9fc4-0bbb39aa09db:Ending Procedure foo 13 | INFO:2024-12-07 13:01:43:Trace:30019fff-7645-4d07-9fc4-0bbb39aa09db:Starting Procedure bar 14 | ERROR:2024-12-07 13:01:43:Trace:30019fff-7645-4d07-9fc4-0bbb39aa09db:Error 3 15 | ERROR:2024-12-07 13:01:43:Trace:30019fff-7645-4d07-9fc4-0bbb39aa09db:Error 4 16 | INFO:2024-12-07 13:01:44:Trace:30019fff-7645-4d07-9fc4-0bbb39aa09db:Ending Procedure bar 17 | -------------------------------------------------------------------------------- /tests/noprint/motivation.out: -------------------------------------------------------------------------------- 1 | ERROR:2024-12-07 13:01:41:Trace:198d079c-af9a-45b2-8236-7fbb2a012f69:Error 1 2 | ERROR:2024-12-07 13:01:43:Trace:30019fff-7645-4d07-9fc4-0bbb39aa09db:Error 3 3 | ERROR:2024-12-07 13:01:43:Trace:30019fff-7645-4d07-9fc4-0bbb39aa09db:Error 4 4 | -------------------------------------------------------------------------------- /tests/noprint/motivation_all.fsa: -------------------------------------------------------------------------------- 1 | startstate: /Starting.Procedure/ -> capturebegin 2 | capturebegin: { start capture -> lookforsuccessorending } 3 | lookforsuccessorending: /Ending.Procedure/ { stop capture print -> startstate } 4 | ALL: /Success/ -> startstate 5 | -------------------------------------------------------------------------------- /tests/noprint/motivation_all.in: -------------------------------------------------------------------------------- 1 | INFO:2024-12-07 13:01:40:Trace:198d079c-af9a-45b2-8236-7fbb2a012f69:Starting... 2 | INFO:2024-12-07 13:01:40:Trace:198d079c-af9a-45b2-8236-7fbb2a012f69:Starting Procedure foo 3 | ERROR:2024-12-07 13:01:41:Trace:198d079c-af9a-45b2-8236-7fbb2a012f69:Error 1 4 | INFO:2024-12-07 13:01:41:Trace:198d079c-af9a-45b2-8236-7fbb2a012f69:Ending Procedure foo 5 | INFO:2024-12-07 13:01:41:Trace:198d079c-af9a-45b2-8236-7fbb2a012f69:Starting Procedure bar 6 | INFO:2024-12-07 13:01:41:Trace:198d079c-af9a-45b2-8236-7fbb2a012f69:Error 2 7 | INFO:2024-12-07 13:01:41:Trace:198d079c-af9a-45b2-8236-7fbb2a012f69:Success 8 | INFO:2024-12-07 13:01:42:Trace:198d079c-af9a-45b2-8236-7fbb2a012f69:Ending Procedure bar 9 | INFO:2024-12-07 13:01:42:Trace:30019fff-7645-4d07-9fc4-0bbb39aa09db:Starting... 10 | INFO:2024-12-07 13:01:42:Trace:30019fff-7645-4d07-9fc4-0bbb39aa09db:Starting Procedure foo 11 | INFO:2024-12-07 13:01:42:Trace:30019fff-7645-4d07-9fc4-0bbb39aa09db:Success 12 | INFO:2024-12-07 13:01:42:Trace:30019fff-7645-4d07-9fc4-0bbb39aa09db:Ending Procedure foo 13 | INFO:2024-12-07 13:01:43:Trace:30019fff-7645-4d07-9fc4-0bbb39aa09db:Starting Procedure bar 14 | ERROR:2024-12-07 13:01:43:Trace:30019fff-7645-4d07-9fc4-0bbb39aa09db:Error 3 15 | ERROR:2024-12-07 13:01:43:Trace:30019fff-7645-4d07-9fc4-0bbb39aa09db:Error 4 16 | INFO:2024-12-07 13:01:44:Trace:30019fff-7645-4d07-9fc4-0bbb39aa09db:Ending Procedure bar 17 | -------------------------------------------------------------------------------- /tests/noprint/motivation_all.out: -------------------------------------------------------------------------------- 1 | ERROR:2024-12-07 13:01:41:Trace:198d079c-af9a-45b2-8236-7fbb2a012f69:Error 1 2 | ERROR:2024-12-07 13:01:43:Trace:30019fff-7645-4d07-9fc4-0bbb39aa09db:Error 3 3 | ERROR:2024-12-07 13:01:43:Trace:30019fff-7645-4d07-9fc4-0bbb39aa09db:Error 4 4 | -------------------------------------------------------------------------------- /tests/noprint/motivation_reset.fsa: -------------------------------------------------------------------------------- 1 | startstate: /Starting.Procedure/ -> capturebegin 2 | capturebegin: { start capture -> lookforsuccessorending } 3 | lookforsuccessorending: /Ending.Procedure/ { stop capture print -> startstate } 4 | ALL: /Success/ --> 5 | -------------------------------------------------------------------------------- /tests/noprint/motivation_reset.in: -------------------------------------------------------------------------------- 1 | INFO:2024-12-07 13:01:40:Trace:198d079c-af9a-45b2-8236-7fbb2a012f69:Starting... 2 | INFO:2024-12-07 13:01:40:Trace:198d079c-af9a-45b2-8236-7fbb2a012f69:Starting Procedure foo 3 | ERROR:2024-12-07 13:01:41:Trace:198d079c-af9a-45b2-8236-7fbb2a012f69:Error 1 4 | INFO:2024-12-07 13:01:41:Trace:198d079c-af9a-45b2-8236-7fbb2a012f69:Ending Procedure foo 5 | INFO:2024-12-07 13:01:41:Trace:198d079c-af9a-45b2-8236-7fbb2a012f69:Starting Procedure bar 6 | INFO:2024-12-07 13:01:41:Trace:198d079c-af9a-45b2-8236-7fbb2a012f69:Error 2 7 | INFO:2024-12-07 13:01:41:Trace:198d079c-af9a-45b2-8236-7fbb2a012f69:Success 8 | INFO:2024-12-07 13:01:42:Trace:198d079c-af9a-45b2-8236-7fbb2a012f69:Ending Procedure bar 9 | INFO:2024-12-07 13:01:42:Trace:30019fff-7645-4d07-9fc4-0bbb39aa09db:Starting... 10 | INFO:2024-12-07 13:01:42:Trace:30019fff-7645-4d07-9fc4-0bbb39aa09db:Starting Procedure foo 11 | INFO:2024-12-07 13:01:42:Trace:30019fff-7645-4d07-9fc4-0bbb39aa09db:Success 12 | INFO:2024-12-07 13:01:42:Trace:30019fff-7645-4d07-9fc4-0bbb39aa09db:Ending Procedure foo 13 | INFO:2024-12-07 13:01:43:Trace:30019fff-7645-4d07-9fc4-0bbb39aa09db:Starting Procedure bar 14 | ERROR:2024-12-07 13:01:43:Trace:30019fff-7645-4d07-9fc4-0bbb39aa09db:Error 3 15 | ERROR:2024-12-07 13:01:43:Trace:30019fff-7645-4d07-9fc4-0bbb39aa09db:Error 4 16 | INFO:2024-12-07 13:01:44:Trace:30019fff-7645-4d07-9fc4-0bbb39aa09db:Ending Procedure bar 17 | -------------------------------------------------------------------------------- /tests/noprint/motivation_reset.out: -------------------------------------------------------------------------------- 1 | ERROR:2024-12-07 13:01:41:Trace:198d079c-af9a-45b2-8236-7fbb2a012f69:Error 1 2 | ERROR:2024-12-07 13:01:43:Trace:30019fff-7645-4d07-9fc4-0bbb39aa09db:Error 3 3 | ERROR:2024-12-07 13:01:43:Trace:30019fff-7645-4d07-9fc4-0bbb39aa09db:Error 4 4 | -------------------------------------------------------------------------------- /tests/test.zsh: -------------------------------------------------------------------------------- 1 | #!/bin/zsh 2 | 3 | set -o errexit 4 | set -o nounset 5 | set -o pipefail 6 | 7 | for testfile in tests/**/*.fsa; do 8 | testname=$(basename $testfile | sed 's/.fsa$//') 9 | echo "running test $testfile..." 10 | infile=$(echo $testfile | sed 's/.fsa$/.in/') 11 | outfile=$(echo $testfile | sed 's/.fsa$/.out/') 12 | flags=$(cat "$testfile:A:h/flags") 13 | ./bin/ted -f "$testfile" $(echo $flags) "$infile" > ./bin/output 14 | if ! diff ./bin/output "$outfile" > /dev/null; then 15 | echo "ERROR: test $testname failed!" 16 | fi 17 | done 18 | -------------------------------------------------------------------------------- /tests/variables/flags: -------------------------------------------------------------------------------- 1 | --var beepvar=beep --var boopvar=boop --var buzzvar=buzz 2 | -------------------------------------------------------------------------------- /tests/variables/variables_basic.fsa: -------------------------------------------------------------------------------- 1 | a: /{{ .beepvar }}/ -> b 2 | b: do "s/{{ .boopvar }}/{{ .buzzvar }}/" 3 | -------------------------------------------------------------------------------- /tests/variables/variables_basic.in: -------------------------------------------------------------------------------- 1 | beep 2 | boop 3 | buzz 4 | -------------------------------------------------------------------------------- /tests/variables/variables_basic.out: -------------------------------------------------------------------------------- 1 | beep 2 | buzz 3 | buzz 4 | --------------------------------------------------------------------------------