├── .gitignore ├── CHANGELOG.md ├── Cargo.toml ├── LICENSE ├── README.md ├── examples ├── abc ├── abc.mst ├── abc_def ├── def ├── example_basic.rs ├── example_character_range.rs ├── example_invalid.rs ├── example_output_file.rs ├── example_pretty_error.rs ├── example_read_mst_file.rs ├── example_save_fa.rs ├── example_save_load_scanner.rs ├── example_skip_category.rs ├── example_skip_whitespace.rs ├── example_visualize_dfa.rs ├── example_visualize_minimal_dfa.rs └── example_visualize_nfa.rs ├── images ├── constructed_dfa.png ├── constructed_minimal_dfa.png ├── constructed_nfa.png ├── fa_visualization.png ├── invalid.png └── valid.png ├── src ├── bin │ └── lexviz.rs ├── dfa.rs ├── fa.rs ├── lib.rs ├── nfa.rs ├── regex.rs ├── scanner.rs └── visualizer.rs ├── test_data ├── atleast_atmost.mst ├── atleast_atmost.snek ├── buffer_test_text.txt ├── comments.mst ├── comments.snek ├── empty.mst ├── empty.snek ├── everything.mst ├── everything.snek ├── invalid.snek ├── range.mst ├── range.snek ├── range_invalid.mst ├── range_invalid.snek ├── sample.mst └── valid.snek └── tests └── integration_tests.rs /.gitignore: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/nagendrajamadagni/Lexer/HEAD/.gitignore -------------------------------------------------------------------------------- /CHANGELOG.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/nagendrajamadagni/Lexer/HEAD/CHANGELOG.md -------------------------------------------------------------------------------- /Cargo.toml: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/nagendrajamadagni/Lexer/HEAD/Cargo.toml -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/nagendrajamadagni/Lexer/HEAD/LICENSE -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/nagendrajamadagni/Lexer/HEAD/README.md -------------------------------------------------------------------------------- /examples/abc: -------------------------------------------------------------------------------- 1 | abc -------------------------------------------------------------------------------- /examples/abc.mst: -------------------------------------------------------------------------------- 1 | abc::ABC 2 | -------------------------------------------------------------------------------- /examples/abc_def: -------------------------------------------------------------------------------- 1 | abc def 2 | -------------------------------------------------------------------------------- /examples/def: -------------------------------------------------------------------------------- 1 | def 2 | -------------------------------------------------------------------------------- /examples/example_basic.rs: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/nagendrajamadagni/Lexer/HEAD/examples/example_basic.rs -------------------------------------------------------------------------------- /examples/example_character_range.rs: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/nagendrajamadagni/Lexer/HEAD/examples/example_character_range.rs -------------------------------------------------------------------------------- /examples/example_invalid.rs: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/nagendrajamadagni/Lexer/HEAD/examples/example_invalid.rs -------------------------------------------------------------------------------- /examples/example_output_file.rs: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/nagendrajamadagni/Lexer/HEAD/examples/example_output_file.rs -------------------------------------------------------------------------------- /examples/example_pretty_error.rs: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/nagendrajamadagni/Lexer/HEAD/examples/example_pretty_error.rs -------------------------------------------------------------------------------- /examples/example_read_mst_file.rs: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/nagendrajamadagni/Lexer/HEAD/examples/example_read_mst_file.rs -------------------------------------------------------------------------------- /examples/example_save_fa.rs: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/nagendrajamadagni/Lexer/HEAD/examples/example_save_fa.rs -------------------------------------------------------------------------------- /examples/example_save_load_scanner.rs: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/nagendrajamadagni/Lexer/HEAD/examples/example_save_load_scanner.rs -------------------------------------------------------------------------------- /examples/example_skip_category.rs: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/nagendrajamadagni/Lexer/HEAD/examples/example_skip_category.rs -------------------------------------------------------------------------------- /examples/example_skip_whitespace.rs: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/nagendrajamadagni/Lexer/HEAD/examples/example_skip_whitespace.rs -------------------------------------------------------------------------------- /examples/example_visualize_dfa.rs: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/nagendrajamadagni/Lexer/HEAD/examples/example_visualize_dfa.rs -------------------------------------------------------------------------------- /examples/example_visualize_minimal_dfa.rs: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/nagendrajamadagni/Lexer/HEAD/examples/example_visualize_minimal_dfa.rs -------------------------------------------------------------------------------- /examples/example_visualize_nfa.rs: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/nagendrajamadagni/Lexer/HEAD/examples/example_visualize_nfa.rs -------------------------------------------------------------------------------- /images/constructed_dfa.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/nagendrajamadagni/Lexer/HEAD/images/constructed_dfa.png -------------------------------------------------------------------------------- /images/constructed_minimal_dfa.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/nagendrajamadagni/Lexer/HEAD/images/constructed_minimal_dfa.png -------------------------------------------------------------------------------- /images/constructed_nfa.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/nagendrajamadagni/Lexer/HEAD/images/constructed_nfa.png -------------------------------------------------------------------------------- /images/fa_visualization.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/nagendrajamadagni/Lexer/HEAD/images/fa_visualization.png -------------------------------------------------------------------------------- /images/invalid.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/nagendrajamadagni/Lexer/HEAD/images/invalid.png -------------------------------------------------------------------------------- /images/valid.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/nagendrajamadagni/Lexer/HEAD/images/valid.png -------------------------------------------------------------------------------- /src/bin/lexviz.rs: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/nagendrajamadagni/Lexer/HEAD/src/bin/lexviz.rs -------------------------------------------------------------------------------- /src/dfa.rs: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/nagendrajamadagni/Lexer/HEAD/src/dfa.rs -------------------------------------------------------------------------------- /src/fa.rs: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/nagendrajamadagni/Lexer/HEAD/src/fa.rs -------------------------------------------------------------------------------- /src/lib.rs: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/nagendrajamadagni/Lexer/HEAD/src/lib.rs -------------------------------------------------------------------------------- /src/nfa.rs: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/nagendrajamadagni/Lexer/HEAD/src/nfa.rs -------------------------------------------------------------------------------- /src/regex.rs: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/nagendrajamadagni/Lexer/HEAD/src/regex.rs -------------------------------------------------------------------------------- /src/scanner.rs: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/nagendrajamadagni/Lexer/HEAD/src/scanner.rs -------------------------------------------------------------------------------- /src/visualizer.rs: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/nagendrajamadagni/Lexer/HEAD/src/visualizer.rs -------------------------------------------------------------------------------- /test_data/atleast_atmost.mst: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/nagendrajamadagni/Lexer/HEAD/test_data/atleast_atmost.mst -------------------------------------------------------------------------------- /test_data/atleast_atmost.snek: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/nagendrajamadagni/Lexer/HEAD/test_data/atleast_atmost.snek -------------------------------------------------------------------------------- /test_data/buffer_test_text.txt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/nagendrajamadagni/Lexer/HEAD/test_data/buffer_test_text.txt -------------------------------------------------------------------------------- /test_data/comments.mst: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/nagendrajamadagni/Lexer/HEAD/test_data/comments.mst -------------------------------------------------------------------------------- /test_data/comments.snek: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/nagendrajamadagni/Lexer/HEAD/test_data/comments.snek -------------------------------------------------------------------------------- /test_data/empty.mst: -------------------------------------------------------------------------------- 1 | \(\)::EMPTY 2 | -------------------------------------------------------------------------------- /test_data/empty.snek: -------------------------------------------------------------------------------- 1 | () 2 | -------------------------------------------------------------------------------- /test_data/everything.mst: -------------------------------------------------------------------------------- 1 | .::EVERYTHING 2 | \n::NEWLINE 3 | -------------------------------------------------------------------------------- /test_data/everything.snek: -------------------------------------------------------------------------------- 1 | abc 2 | -------------------------------------------------------------------------------- /test_data/invalid.snek: -------------------------------------------------------------------------------- 1 | , 2 | -------------------------------------------------------------------------------- /test_data/range.mst: -------------------------------------------------------------------------------- 1 | a{3-5}::THREE_FIVE_A 2 | \n::NEWLINE 3 | -------------------------------------------------------------------------------- /test_data/range.snek: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/nagendrajamadagni/Lexer/HEAD/test_data/range.snek -------------------------------------------------------------------------------- /test_data/range_invalid.mst: -------------------------------------------------------------------------------- 1 | a{3-5}\n::THREE_FIVE_A 2 | -------------------------------------------------------------------------------- /test_data/range_invalid.snek: -------------------------------------------------------------------------------- 1 | KL.(F. -------------------------------------------------------------------------------- /test_data/sample.mst: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/nagendrajamadagni/Lexer/HEAD/test_data/sample.mst -------------------------------------------------------------------------------- /test_data/valid.snek: -------------------------------------------------------------------------------- 1 | main(input) { 2 | add1 (sub1 (negate(5))); 3 | } 4 | -------------------------------------------------------------------------------- /tests/integration_tests.rs: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/nagendrajamadagni/Lexer/HEAD/tests/integration_tests.rs --------------------------------------------------------------------------------