├── .gitignore ├── LICENSE ├── README.md ├── lib ├── antlr4dart.dart ├── antlr4dart_deprecation_fix.dart └── src │ ├── atn │ ├── atn.dart │ ├── atn_config.dart │ ├── atn_config_set.dart │ ├── atn_deserializer.dart │ ├── atn_simulator.dart │ ├── atn_states.dart │ ├── atn_type.dart │ ├── contexts.dart │ ├── lexer_actions.dart │ ├── ll1_analyser.dart │ ├── prediction_mode.dart │ ├── semantic_context.dart │ └── transitions.dart │ ├── contexts.dart │ ├── deprecation_fix │ ├── deprecated_parser_mixin.dart │ ├── deprecated_recognizer_mixin.dart │ ├── deprecation_fix.dart │ └── error_listener.dart │ ├── dfa.dart │ ├── error_strategy.dart │ ├── events.dart │ ├── exceptions.dart │ ├── input_source.dart │ ├── interpreters.dart │ ├── lexer.dart │ ├── parser.dart │ ├── recognizer.dart │ ├── token.dart │ ├── token_provider.dart │ ├── token_source.dart │ ├── tree │ ├── parse_tree.dart │ ├── parse_tree_visitor.dart │ ├── parse_tree_walker.dart │ └── trees.dart │ └── util │ ├── bit_set.dart │ ├── double_key_map.dart │ ├── interval.dart │ ├── interval_set.dart │ ├── murmur_hash.dart │ └── pair.dart └── pubspec.yaml /.gitignore: -------------------------------------------------------------------------------- 1 | test/ 2 | main.dart 3 | pubspec.lock 4 | packages/ 5 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | This license applies to all parts of antlr4dart that are not externally 2 | maintained libraries. 3 | 4 | Copyright 2014, the antlr4dart project authors. All rights reserved. Redistribution 5 | and use in source and binary forms, with or without modification, are permitted provided that 6 | the following conditions are met: 7 | 8 | * Redistributions of source code must retain the above copyright notice, this list of conditions 9 | and the following disclaimer; 10 | * Redistributions in binary form must reproduce the above copyright notice, this list of conditions 11 | and the following disclaimer in the documentation and/or other materials provided with the distribution; 12 | * Neither the name of antlr4dart team. nor the names of its contributors may be used to 13 | endorse or promote products derived from this software without specific prior written permission. 14 | 15 | THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR 16 | IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY 17 | AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR 18 | CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL 19 | DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, 20 | DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER 21 | IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT 22 | OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 23 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | **ANTLR 4 runtime for Dart** 2 | 3 | #### Description 4 | 5 | Fully-featured ANTLR 4 runtime library for Dart. 6 | 7 | ANTLR (ANother Tool for Language Recognition) is a tool that is used to 8 | generate code for performing a variety of language recognition tasks: 9 | lexing, parsing, abstract syntax tree construction and manipulation, tree 10 | structure recognition, and input translation. The tool operates similarly 11 | to other parser generators, taking in a grammar specification written in 12 | the special ANTLR metalanguage and producing source code that implements 13 | the recognition functionality. 14 | 15 | While the tool itself is implemented in Java, it has an extensible design 16 | that allows for code generation in other programming languages. To implement 17 | an ANTLR language target, a developer may supply a set of templates written 18 | in the StringTemplate ([http://www.stringtemplate.org](http://www.stringtemplate.org)) language. 19 | 20 | This dart lib is a complete implementation of the majority of features 21 | ANTLR provides for other language targets, such as Java and CSharp. It 22 | contains a dart runtime library that collects classes used throughout the 23 | code that the modified ANTLR4 ([https://github.com/tiagomazzutti/antlr4dart](https://github.com/tiagomazzutti/antlr4dart)) generates. 24 | 25 | #### Usage 26 | 27 | 1. Write an ANTLR4 grammar specification for a language: 28 | 29 | ``` 30 | grammar SomeLanguage; 31 | 32 | options { 33 | language = Dart; // <- this option must be set to Dart 34 | } 35 | 36 | top: expr ( ',' expr )* 37 | ; 38 | 39 | // and so on... 40 | ``` 41 | 42 | 2. Run the [ANTLR4](https://github.com/tiagomazzutti/antlr4dart) tool with the `java -jar path/to/antlr--with-dart-support.jar` command to generate output: 43 | 44 | ``` 45 | $> java -jar path/to/antlr-{VERSION}-with-dart-support.jar [OPTIONS] lang.g 46 | # creates: 47 | # langParser.dart 48 | # langLexer.dart 49 | # lang.tokens 50 | ``` 51 | 52 | alternatively, you can do: 53 | 54 | ``` 55 | $> export CLASSPATH=path/to/path/to/antlr-{VERSION}-with-dart-support.jar:$CLASSPATH 56 | 57 | $> java org.antlr.v4.Tool [OPTIONS] $grammar 58 | ``` 59 | 60 | NOTES: 61 | 62 | * Replace *VERSION* with the version number currently used in [ANTLR4](https://github.com/tiagomazzutti/antlr4dart), or your own build of ANLTR4; 63 | 64 | * Probably you will need to edit the `@header{}` section in your grammar: 65 | 66 | Use 67 | ``` 68 | @header { 69 | library your_library_name; 70 | import 'package:antlr4dart/antlr4dart.dart'; 71 | } 72 | ``` 73 | 74 | if the code should be generated in a dedicated Dart library. 75 | 76 | Use 77 | ``` 78 | @header { 79 | part of your_library_name; 80 | // no import statement here, add it to the parent library file 81 | } 82 | ``` 83 | 84 | if the code should be generated as part of another library. 85 | 86 | *More samples can be found in the [antlr4dart](https://github.com/tiagomazzutti/antlr4dart) test folder.* 87 | 88 | 3. Make sure your `pubspec.yaml` includes a dependency to `antlr4dart`: 89 | 90 | `antlr4dart` is hosted on pub.dartlang.org, the most simple dependency statement is therefore: 91 | 92 | ``` 93 | dependencies: 94 | antlr4dart: any 95 | ``` 96 | 97 | Alternatively, you can add a dependency to antlr4dart's GitHub repository: 98 | ``` 99 | dependencies: 100 | antlr4dart: 101 | git: git@github.com:tiagomazzutti/antlr4dart-runtime.git 102 | ``` 103 | 104 | 4. Try out the results directly: 105 | 106 | ``` 107 | import "package:antlr4dart/antlr4dart.dart"; 108 | import "SomeLanguageLexer.dart"; 109 | import "SomeLanguageParser.dart"; 110 | 111 | main() { 112 | var input = 'some text to be parsed...'; 113 | var source = new StringSource(input); 114 | var lexer = new SomeLanguageLexer(source); 115 | var tokens = new CommonTokenSource(lexer); 116 | var parser = new SomeLanguageParser(tokens); 117 | 118 | var result = parser.(); 119 | // ... 120 | } 121 | ``` 122 | -------------------------------------------------------------------------------- /lib/antlr4dart.dart: -------------------------------------------------------------------------------- 1 | library antlr4dart; 2 | 3 | import 'dart:collection'; 4 | import 'dart:math'; 5 | import 'dart:async'; 6 | import 'package:bignum/bignum.dart'; 7 | import 'package:collection/equality.dart'; 8 | 9 | part 'src/contexts.dart'; 10 | part 'src/dfa.dart'; 11 | part 'src/error_strategy.dart'; 12 | part 'src/events.dart'; 13 | part 'src/exceptions.dart'; 14 | part 'src/input_source.dart'; 15 | part 'src/lexer.dart'; 16 | part 'src/interpreters.dart'; 17 | part 'src/parser.dart'; 18 | part 'src/recognizer.dart'; 19 | part 'src/token.dart'; 20 | part 'src/token_provider.dart'; 21 | part 'src/token_source.dart'; 22 | part 'src/atn/atn.dart'; 23 | part 'src/atn/atn_config.dart'; 24 | part 'src/atn/atn_config_set.dart'; 25 | part 'src/atn/atn_deserializer.dart'; 26 | part 'src/atn/atn_simulator.dart'; 27 | part 'src/atn/atn_states.dart'; 28 | part 'src/atn/atn_type.dart'; 29 | part 'src/atn/contexts.dart'; 30 | part 'src/atn/lexer_actions.dart'; 31 | part 'src/atn/ll1_analyser.dart'; 32 | part 'src/atn/prediction_mode.dart'; 33 | part 'src/atn/semantic_context.dart'; 34 | part 'src/atn/transitions.dart'; 35 | part 'src/tree/parse_tree.dart'; 36 | part 'src/tree/parse_tree_visitor.dart'; 37 | part 'src/tree/parse_tree_walker.dart'; 38 | part 'src/tree/trees.dart'; 39 | part 'src/util/bit_set.dart'; 40 | part 'src/util/double_key_map.dart'; 41 | part 'src/util/interval.dart'; 42 | part 'src/util/interval_set.dart'; 43 | part 'src/util/murmur_hash.dart'; 44 | part 'src/util/pair.dart'; 45 | -------------------------------------------------------------------------------- /lib/antlr4dart_deprecation_fix.dart: -------------------------------------------------------------------------------- 1 | /** 2 | * Some changes to the recognizer interface occurred as of release 0.7.0 3 | * involving error listening. The old system of [ErrorListener]s was replaced 4 | * by a more dart-ish [Stream] interface. 5 | * 6 | * Old code that used the [ErrorListener] interface can be easily re-written 7 | * without the use of this package. However, for situations where the code 8 | * base is large and a quick fix is required, this library provides a two- 9 | * line solution. 10 | * 11 | * For each [Recognizer] using the old [ErrorListener] interface, mix in 12 | * [DeprecatedRecognizerMixin], and before use (presumably in the constructor), 13 | * call [deprecatedRecognizerMixinInit](). 14 | * 15 | * Old broken code 16 | * class MyLanguageSpecificParser extends Parser{ 17 | * MyLanguageSpecificParser(); 18 | * 19 | * //stuff involving funcitons removed from the Recognizer API 20 | * //throw NoSuchMethodErrors 21 | * } 22 | * 23 | * New code 24 | * class MyLanguageSpecificParser extends Parser with DeprecatedRecognizerMixin{ 25 | * MyLanguageSpecificParser(){ 26 | * deprecatedRecognizerMixinInit(); 27 | * //functions removed from the Recognizer API now work as expected. 28 | * } 29 | * 30 | * //stuff involving functions removed from the Recognizer API execute! 31 | * } 32 | */ 33 | 34 | @deprecated 35 | library antlr4dart.deprecation_fix; 36 | 37 | export 'src/deprecation_fix/deprecation_fix.dart'; -------------------------------------------------------------------------------- /lib/src/atn/atn.dart: -------------------------------------------------------------------------------- 1 | part of antlr4dart; 2 | 3 | class Atn { 4 | static const int INVALID_ALT_NUMBER = 0; 5 | 6 | final List states = new List(); 7 | 8 | /// Each subrule/rule is a decision point and we must track them so we 9 | /// can go back later and build DFA predictors for them. This includes 10 | /// all the rules, subrules, optional blocks, ()+, ()* etc... 11 | final List decisionToState = new List(); 12 | 13 | /// Maps from rule index to starting state number. 14 | List ruleToStartState; 15 | 16 | /// Maps from rule index to stop state number. 17 | List ruleToStopState; 18 | 19 | final modeNameToStartState = new LinkedHashMap(); 20 | 21 | /// The type of the ATN. 22 | final AtnType grammarType; 23 | 24 | /// The maximum value for any symbol recognized by a transition in the ATN. 25 | final int maxTokenType; 26 | 27 | /// For lexer ATNs, this maps the rule index to the resulting token type. 28 | /// 29 | /// This is `null` for parser ATNs. 30 | List ruleToTokenType; 31 | 32 | /// For lexer ATNs, this is an array of [LexerAction] objects which may 33 | /// be referenced by action transitions in the ATN. 34 | List lexerActions; 35 | 36 | final List modeToStartState = new List(); 37 | 38 | /// Used for runtime deserialization of ATNs from strings 39 | Atn(this.grammarType, this.maxTokenType); 40 | 41 | int get numberOfDecisions => decisionToState.length; 42 | 43 | /// Compute the set of valid tokens that can occur starting in [state]. 44 | /// If [context] is `null`, the set of tokens will not include what can 45 | /// follow the rule surrounding [state]. In other words, the set will be 46 | /// restricted to tokens reachable staying within [state]'s rule. 47 | IntervalSet nextTokens(AtnState state, RuleContext context) { 48 | return new Ll1Analyzer(this).look(state, context); 49 | } 50 | 51 | /// Compute the set of valid tokens that can occur starting in [state] and 52 | /// staying in same rule. [Token.EPSILON] is in set if we reach end of 53 | /// rule. 54 | IntervalSet nextTokensInSameRule(AtnState state) { 55 | if (state.nextTokenWithinRule != null) return state.nextTokenWithinRule; 56 | return (state 57 | ..nextTokenWithinRule = nextTokens(state, null) 58 | ..nextTokenWithinRule.isReadonly = true).nextTokenWithinRule; 59 | } 60 | 61 | void addState(AtnState state) { 62 | if (state != null) { 63 | state 64 | ..atn = this 65 | ..stateNumber = states.length; 66 | } 67 | states.add(state); 68 | } 69 | 70 | void removeState(AtnState state) { 71 | states[state.stateNumber] = null; 72 | } 73 | 74 | int defineDecisionState(DecisionState state) { 75 | decisionToState.add(state); 76 | state.decision = decisionToState.length - 1; 77 | return state.decision; 78 | } 79 | 80 | DecisionState getDecisionState(int decision) { 81 | return decisionToState.isNotEmpty ? decisionToState[decision] : null; 82 | } 83 | 84 | /// Computes the set of input symbols which could follow ATN state number 85 | /// [stateNumber] in the specified full [context]. 86 | /// 87 | /// This method considers the complete parser context, but does not evaluate 88 | /// semantic predicates (i.e. all predicates encountered during the 89 | /// calculation are assumed true). 90 | /// 91 | /// If a path in the ATN exists from the starting state to the [RuleStopState] 92 | /// of the outermost context without matching any symbols, [Token.EOF] is 93 | /// added to the returned set. 94 | /// 95 | /// If [context] is `null`, it is treated as [ParserRuleContext.EMPTY]. 96 | /// 97 | /// [stateNumber] is the ATN state number. 98 | /// [context] is the full parse context. 99 | /// 100 | /// Return is the set of potentially valid input symbols which could follow 101 | /// the specified state in the specified context. 102 | /// An [ArgumentError] occurs when the ATN does not contain a state with 103 | /// number [stateNumber]. 104 | IntervalSet getExpectedTokens(int stateNumber, RuleContext context) { 105 | if (stateNumber < 0 || stateNumber >= states.length) { 106 | throw new ArgumentError("Invalid state number."); 107 | } 108 | AtnState s = states[stateNumber]; 109 | IntervalSet following = nextTokensInSameRule(s); 110 | if (!following.contains(Token.EPSILON)) return following; 111 | IntervalSet expected = new IntervalSet() 112 | ..addAll(following) 113 | ..remove(Token.EPSILON); 114 | while (context != null && context.invokingState >= 0 115 | && following.contains(Token.EPSILON)) { 116 | AtnState invokingState = states[context.invokingState]; 117 | RuleTransition rt = invokingState.getTransition(0); 118 | following = nextTokensInSameRule(rt.followState); 119 | expected 120 | ..addAll(following) 121 | ..remove(Token.EPSILON); 122 | context = context.parent; 123 | } 124 | if (following.contains(Token.EPSILON)) expected.addSingle(Token.EOF); 125 | return expected; 126 | } 127 | } 128 | -------------------------------------------------------------------------------- /lib/src/atn/atn_config.dart: -------------------------------------------------------------------------------- 1 | part of antlr4dart; 2 | 3 | /// A tuple: (ATN state, predicted alt, syntactic, semantic context). 4 | /// The syntactic context is a graph-structured stack node whose 5 | /// path(s) to the root is the rule invocation(s) 6 | /// chain used to arrive at the state. The semantic context is 7 | /// the tree of semantic predicates encountered before reaching 8 | /// an ATN state. 9 | class AtnConfig { 10 | 11 | /// The ATN state associated with this configuration 12 | final AtnState state; 13 | 14 | /// What alt (or lexer rule) is predicted by this configuration 15 | final int alt; 16 | 17 | /// The stack of invoking states leading to the rule/states associated 18 | /// with this config. We track only those contexts pushed during 19 | /// execution of the ATN simulator. 20 | PredictionContext context; 21 | 22 | /// We cannot execute predicates dependent upon local context unless 23 | /// we know for sure we are in the correct context. Because there is 24 | /// no way to do this efficiently, we simply cannot evaluate 25 | /// dependent predicates unless we are in the rule that initially 26 | /// invokes the ATN simulator. 27 | int reachesIntoOuterContext = 0; 28 | 29 | final SemanticContext semanticContext; 30 | 31 | AtnConfig(AtnState state, 32 | int alt, 33 | PredictionContext context, 34 | [SemanticContext semanticContext]) 35 | : this.semanticContext = (semanticContext != null) 36 | ? semanticContext : SemanticContext.NONE, 37 | this.state = state, 38 | this.alt = alt, 39 | this.context = context; 40 | 41 | AtnConfig.from(AtnConfig c, 42 | {AtnState state, 43 | PredictionContext context, 44 | SemanticContext semanticContext}) 45 | : this.state = (state != null) ? state : c.state, 46 | this.context = (context != null) ? context : c.context, 47 | this.semanticContext = (semanticContext != null) 48 | ? semanticContext : c.semanticContext, 49 | alt = c.alt, 50 | reachesIntoOuterContext = c.reachesIntoOuterContext; 51 | 52 | /// An ATN configuration is equal to [other] if both have the same state, 53 | /// they predict the same alternative, and syntactic/semantic contexts are 54 | /// the same. 55 | bool operator==(Object other) { 56 | return other is AtnConfig 57 | && state.stateNumber == other.state.stateNumber 58 | && alt == other.alt 59 | && (context == other.context 60 | || context != null && context == other.context) 61 | && semanticContext == other.semanticContext; 62 | } 63 | 64 | int get hashCode { 65 | int hashCode = MurmurHash.initialize(7); 66 | hashCode = MurmurHash.update(hashCode, state.stateNumber); 67 | hashCode = MurmurHash.update(hashCode, alt); 68 | hashCode = MurmurHash.update(hashCode, context.hashCode); 69 | hashCode = MurmurHash.update(hashCode, semanticContext.hashCode); 70 | hashCode = MurmurHash.finish(hashCode, 4); 71 | return hashCode; 72 | } 73 | 74 | String toString([Recognizer recog, bool showAlt = true]) { 75 | StringBuffer sb = new StringBuffer('(')..write(state); 76 | if (showAlt) sb.write(",$alt"); 77 | if (context != null) sb.write(",[$context]"); 78 | if (semanticContext != null && semanticContext != SemanticContext.NONE) { 79 | sb.write(",$semanticContext"); 80 | } 81 | if (reachesIntoOuterContext > 0) { 82 | sb.write(",up=$reachesIntoOuterContext"); 83 | } 84 | sb.write(')'); 85 | return sb.toString(); 86 | } 87 | } 88 | 89 | class LexerAtnConfig extends AtnConfig { 90 | 91 | /// Capture lexer actions we traverse. 92 | final LexerActionExecutor lexerActionExecutor; 93 | 94 | final bool hasPassedThroughNonGreedyDecision; 95 | 96 | LexerAtnConfig(AtnState state, 97 | int alt, 98 | PredictionContext context, 99 | [this.lexerActionExecutor]) 100 | : hasPassedThroughNonGreedyDecision = false, 101 | super(state, alt, context, SemanticContext.NONE); 102 | 103 | 104 | LexerAtnConfig.from(LexerAtnConfig c, 105 | AtnState state, 106 | {LexerActionExecutor actionExecutor, 107 | PredictionContext context}) 108 | : lexerActionExecutor = (actionExecutor != null) 109 | ? actionExecutor : c.lexerActionExecutor, 110 | hasPassedThroughNonGreedyDecision = c.hasPassedThroughNonGreedyDecision 111 | || state is DecisionState && state.nonGreedy, 112 | super.from(c, state:state, context:(context != null) ? context: c.context, 113 | semanticContext:c.semanticContext); 114 | 115 | int get hashCode { 116 | int hashCode = MurmurHash.initialize(7); 117 | hashCode = MurmurHash.update(hashCode, state.stateNumber); 118 | hashCode = MurmurHash.update(hashCode, alt); 119 | hashCode = MurmurHash.update(hashCode, context.hashCode); 120 | hashCode = MurmurHash.update(hashCode, semanticContext.hashCode); 121 | hashCode = MurmurHash.update(hashCode, 122 | hasPassedThroughNonGreedyDecision ? 1 : 0); 123 | hashCode = MurmurHash.update(hashCode, lexerActionExecutor.hashCode); 124 | hashCode = MurmurHash.finish(hashCode, 6); 125 | return hashCode; 126 | } 127 | 128 | bool operator==(AtnConfig other) { 129 | return other is LexerAtnConfig 130 | && hasPassedThroughNonGreedyDecision 131 | == other.hasPassedThroughNonGreedyDecision 132 | && lexerActionExecutor == other.lexerActionExecutor 133 | && super == other; 134 | } 135 | } 136 | 137 | -------------------------------------------------------------------------------- /lib/src/atn/atn_config_set.dart: -------------------------------------------------------------------------------- 1 | part of antlr4dart; 2 | 3 | /// Specialized [Set]`<`[AtnConfig]`>` that can track info about the set, 4 | /// with support for combining similar configurations using a 5 | /// graph-structured stack. 6 | class AtnConfigSet { 7 | 8 | // Indicates that the set of configurations is read-only. Do not allow any 9 | // code to manipulate the set; DFA states will point at the sets and they 10 | // must not change. This does not protect the other fields; in particular, 11 | // conflictingAlts is set after we've made this readonly. 12 | bool _readonly = false; 13 | 14 | int _cachedHashCode = -1; 15 | BitSet _conflictingAlts; 16 | 17 | /// All configs but hashed by (s, i, _, pi) not including context. Wiped out 18 | /// when we go readonly as this set becomes a DFA state. 19 | HashSet configLookup; 20 | 21 | /// Track the elements as they are added to the set. 22 | final List configs = new List(); 23 | 24 | /// Used in parser and lexer. In lexer, it indicates we hit a pred while 25 | /// computing a closure operation. 26 | bool hasSemanticContext = false; 27 | 28 | bool dipsIntoOuterContext = false; 29 | 30 | int uniqueAlt = 0; 31 | 32 | /// Indicates that this configuration set is part of a full context 33 | /// LL prediction. It will be used to determine how to merge `$`. With SLL 34 | /// it's a wildcard whereas it is not for LL context merge. 35 | final bool fullCtx; 36 | 37 | AtnConfigSet([this.fullCtx = true]) { 38 | configLookup = new HashSet(); 39 | } 40 | 41 | AtnConfigSet.from(AtnConfigSet other) 42 | : fullCtx = other.fullCtx { 43 | configLookup = new HashSet(); 44 | addAll(other.configs); 45 | uniqueAlt = other.uniqueAlt; 46 | _conflictingAlts = other._conflictingAlts; 47 | hasSemanticContext = other.hasSemanticContext; 48 | dipsIntoOuterContext = other.dipsIntoOuterContext; 49 | } 50 | 51 | /// Return a List holding list of configs. 52 | List get elements => configs; 53 | 54 | Set get states { 55 | return new Set.from(configs.map((s) => s.state)); 56 | } 57 | 58 | List get predicates { 59 | List preds = new List(); 60 | for (AtnConfig c in configs) { 61 | if (c.semanticContext != SemanticContext.NONE) { 62 | preds.add(c.semanticContext); 63 | } 64 | } 65 | return preds; 66 | } 67 | 68 | int get hashCode { 69 | if (_readonly) { 70 | if (_cachedHashCode == -1) { 71 | configs.forEach((c) => _cachedHashCode += _hashCode(c)); 72 | _cachedHashCode &= 0xFFFFFFFF; 73 | } 74 | return _cachedHashCode; 75 | } 76 | int hash = -1; 77 | configs.forEach((c) => hash += _hashCode(c)); 78 | return hash & 0xFFFFFFFF; 79 | } 80 | 81 | int get length => configs.length; 82 | 83 | bool get isEmpty => configs.isEmpty; 84 | 85 | Iterator get iterator => configs.iterator; 86 | 87 | bool get isReadonly => _readonly; 88 | 89 | void set isReadonly(bool readonly) { 90 | _readonly = readonly; 91 | configLookup = null; // can't mod, no need for lookup cache 92 | } 93 | 94 | /// Gets the complete set of represented alternatives for the 95 | /// configuration set. 96 | BitSet get alts { 97 | BitSet alts = new BitSet(); 98 | for (AtnConfig config in configs) { 99 | alts.set(config.alt, true); 100 | } 101 | return alts; 102 | } 103 | 104 | AtnConfig operator[](int i) => configs[i]; 105 | 106 | void optimizeConfigs(AtnSimulator interpreter) { 107 | if (_readonly) throw new StateError("This set is readonly"); 108 | if (configLookup.isEmpty) return; 109 | for (AtnConfig config in configs) { 110 | config.context = interpreter.getCachedContext(config.context); 111 | } 112 | } 113 | 114 | bool addAll(Iterable configs) { 115 | configs.forEach((c) => add(c)); 116 | return false; 117 | } 118 | 119 | /// Adding a new config means merging contexts with existing configs for 120 | /// `(s, i, pi, _)`, where `s` is the [AtnConfig.state], `i` is the 121 | /// [AtnConfig.alt], and `pi` is the [AtnConfig.semanticContext]. 122 | /// 123 | /// This method updates [dipsIntoOuterContext] and [hasSemanticContext] 124 | /// when necessary. 125 | bool add(AtnConfig config, 126 | [DoubleKeyMap mergeCache]) { 128 | if (_readonly) throw new StateError("This set is readonly"); 129 | if (config.semanticContext 130 | != SemanticContext.NONE) hasSemanticContext = true; 131 | if (config.reachesIntoOuterContext > 0) dipsIntoOuterContext = true; 132 | AtnConfig existing = configLookup.lookup(config); 133 | if (existing == null) { // we added this new one 134 | configLookup.add(config); 135 | _cachedHashCode = -1; 136 | configs.add(config); // track order here 137 | return true; 138 | } 139 | // a previous (s,i,pi,_), merge with it and save result 140 | bool rootIsWildcard = !fullCtx; 141 | var merged = PredictionContext.merge( 142 | existing.context, config.context, rootIsWildcard, mergeCache); 143 | // no need to check for existing.context, config.context in cache 144 | // since only way to create new graphs is "call rule" and here. We 145 | // cache at both places. 146 | existing 147 | ..reachesIntoOuterContext = max( 148 | existing.reachesIntoOuterContext, config.reachesIntoOuterContext) 149 | ..context = merged; // replace context; no need to alt mapping 150 | return true; 151 | } 152 | 153 | bool operator==(Object other) { 154 | return other is AtnConfigSet 155 | && configs != null 156 | && _equalConfigs(other.configs) 157 | && fullCtx == other.fullCtx 158 | && uniqueAlt == other.uniqueAlt 159 | && _conflictingAlts == other._conflictingAlts 160 | && hasSemanticContext == other.hasSemanticContext 161 | && dipsIntoOuterContext == other.dipsIntoOuterContext; 162 | } 163 | 164 | bool contains(Object o) { 165 | if (configLookup == null) { 166 | throw new UnsupportedError( 167 | "This method is not implemented for readonly sets."); 168 | } 169 | return configLookup.contains(o); 170 | } 171 | 172 | void clear() { 173 | if (_readonly) throw new StateError("This set is readonly"); 174 | configs.clear(); 175 | _cachedHashCode = -1; 176 | configLookup.clear(); 177 | } 178 | 179 | String toString() { 180 | StringBuffer sb = new StringBuffer()..write(elements); 181 | if (hasSemanticContext) 182 | sb..write(",hasSemanticContext=")..write(hasSemanticContext); 183 | if (uniqueAlt != Atn.INVALID_ALT_NUMBER) 184 | sb..write(",uniqueAlt=")..write(uniqueAlt); 185 | if (_conflictingAlts != null) 186 | sb..write(",conflictingAlts=")..write(_conflictingAlts); 187 | if (dipsIntoOuterContext) sb..write(",dipsIntoOuterContext"); 188 | return sb.toString(); 189 | } 190 | 191 | bool _equalConfigs(List cfgs) { 192 | if (cfgs.length != configs.length) return false; 193 | for (int i = 0; i < configs.length; i++) { 194 | if (configs[i] != cfgs[i]) return false; 195 | } 196 | return true; 197 | } 198 | 199 | int _hashCode(AtnConfig other) { 200 | int hashCode = 7; 201 | hashCode = 31 * hashCode + other.state.stateNumber; 202 | hashCode = 31 * hashCode + other.alt; 203 | hashCode = 31 * hashCode + other.semanticContext.hashCode; 204 | return hashCode; 205 | } 206 | } 207 | 208 | 209 | 210 | 211 | -------------------------------------------------------------------------------- /lib/src/atn/atn_states.dart: -------------------------------------------------------------------------------- 1 | part of antlr4dart; 2 | 3 | abstract class AtnState { 4 | 5 | static const int INVALID_TYPE = 0; 6 | static const int BASIC = 1; 7 | static const int RULE_START = 2; 8 | static const int BLOCK_START = 3; 9 | static const int PLUS_BLOCK_START = 4; 10 | static const int STAR_BLOCK_START = 5; 11 | static const int TOKEN_START = 6; 12 | static const int RULE_STOP = 7; 13 | static const int BLOCK_END = 8; 14 | static const int STAR_LOOP_BACK = 9; 15 | static const int STAR_LOOP_ENTRY = 10; 16 | static const int PLUS_LOOP_BACK = 11; 17 | static const int LOOP_END = 12; 18 | 19 | static const List serializationNames = 20 | const [ "INVALID", "BASIC", "RULE_START", "BLOCK_START", 21 | "PLUS_BLOCK_START", "STAR_BLOCK_START", "TOKEN_START", "RULE_STOP", 22 | "BLOCK_END", "STAR_LOOP_BACK", "STAR_LOOP_ENTRY", "PLUS_LOOP_BACK", 23 | "LOOP_END" 24 | ]; 25 | 26 | static const int INVALID_STATE_NUMBER = -1; 27 | 28 | /// Which ATN are we in? 29 | Atn atn = null; 30 | 31 | int stateNumber = INVALID_STATE_NUMBER; 32 | 33 | int ruleIndex; 34 | 35 | bool epsilonOnlyTransitions = false; 36 | 37 | /// Track the transitions emanating from this ATN state. 38 | final List _transitions = new List(); 39 | 40 | /// Used to cache lookahead during parsing, not used during construction. 41 | IntervalSet nextTokenWithinRule; 42 | 43 | int get hashCode => stateNumber; 44 | 45 | int get stateType; 46 | 47 | bool get isNonGreedyExitState => false; 48 | 49 | List get transitions => new List.from(_transitions); 50 | 51 | int get numberOfTransitions => _transitions.length; 52 | 53 | bool get onlyHasEpsilonTransitions => epsilonOnlyTransitions; 54 | 55 | bool operator==(Object other) { 56 | return other is AtnState && stateNumber == other.stateNumber; 57 | } 58 | 59 | String toString() => "$stateNumber"; 60 | 61 | void addTransition(Transition e) { 62 | addTransitionAt(e); 63 | } 64 | 65 | void addTransitionAt(Transition e, [int index]) { 66 | if (_transitions.isEmpty) { 67 | epsilonOnlyTransitions = e.isEpsilon; 68 | } else if (epsilonOnlyTransitions != e.isEpsilon) { 69 | epsilonOnlyTransitions = false; 70 | } 71 | if (index != null) { 72 | _transitions.insert(index, e); 73 | } else { 74 | _transitions.add(e); 75 | } 76 | } 77 | 78 | Transition getTransition(int i) => _transitions[i]; 79 | 80 | void setTransition(int i, Transition e) { 81 | transitions[i] = e; 82 | } 83 | 84 | Transition removeTransition(int index) => _transitions.removeAt(index); 85 | } 86 | 87 | class BasicState extends AtnState { 88 | int get stateType => AtnState.BASIC; 89 | } 90 | 91 | class BasicBlockStartState extends BlockStartState { 92 | int get stateType => AtnState.BLOCK_START; 93 | } 94 | 95 | /// Terminal node of a simple `(a|b|c)` block. 96 | class BlockEndState extends AtnState { 97 | BlockStartState startState; 98 | int get stateType => AtnState.BLOCK_END; 99 | } 100 | 101 | /// The start of a regular `(...)` block. 102 | abstract class BlockStartState extends DecisionState { 103 | BlockEndState endState; 104 | } 105 | 106 | abstract class DecisionState extends AtnState { 107 | int decision = -1; 108 | bool nonGreedy = false; 109 | } 110 | 111 | /// Mark the end of a * or + loop. 112 | class LoopEndState extends AtnState { 113 | AtnState loopBackState; 114 | int get stateType => AtnState.LOOP_END; 115 | } 116 | 117 | /// Start of `(A|B|...)+` loop. Technically a decision state, but we don't 118 | /// use for code generation; somebody might need it, so I'm defining it for 119 | /// completeness. In reality, the [PlusLoopbackState] node is the real 120 | /// decision-making note for `A+`. 121 | class PlusBlockStartState extends BlockStartState { 122 | PlusLoopbackState loopBackState; 123 | int get stateType => AtnState.PLUS_BLOCK_START; 124 | } 125 | 126 | /// Decision state for `A+` and `(A|B)+`. It has two transitions: 127 | /// one to the loop back to start of the block and one to exit. 128 | class PlusLoopbackState extends DecisionState { 129 | int get stateType => AtnState.PLUS_LOOP_BACK; 130 | } 131 | 132 | class RuleStartState extends AtnState { 133 | RuleStopState stopState; 134 | bool isPrecedenceRule = false; 135 | int get stateType => AtnState.RULE_START; 136 | } 137 | 138 | /// The last node in the ATN for a rule, unless that rule is the start symbol. 139 | /// In that case, there is one transition to `EOF`. Later, we might encode 140 | /// references to all calls to this rule to compute `FOLLOW` sets for 141 | /// error handling. 142 | class RuleStopState extends AtnState { 143 | int get stateType => AtnState.RULE_STOP; 144 | } 145 | 146 | /// The block that begins a closure loop. 147 | class StarBlockStartState extends BlockStartState { 148 | int get stateType => AtnState.STAR_BLOCK_START; 149 | } 150 | 151 | class StarLoopbackState extends AtnState { 152 | int get stateType => AtnState.STAR_LOOP_BACK; 153 | StarLoopEntryState get loopEntryState => getTransition(0).target; 154 | } 155 | 156 | class StarLoopEntryState extends DecisionState { 157 | StarLoopbackState loopBackState; 158 | 159 | /// Indicates whether this state can benefit from a precedence Dfa 160 | /// during SLL decision making. 161 | /// 162 | /// This is a computed property that is calculated during ATN 163 | /// deserialization and stored for use in [ParserAtnSimulator] and 164 | /// [ParserInterpreter]. 165 | bool precedenceRuleDecision = false; 166 | 167 | int get stateType => AtnState.STAR_LOOP_ENTRY; 168 | } 169 | 170 | /// The Tokens rule start state linking to each lexer rule start state. 171 | class TokensStartState extends DecisionState { 172 | int get stateType => AtnState.TOKEN_START; 173 | } 174 | -------------------------------------------------------------------------------- /lib/src/atn/atn_type.dart: -------------------------------------------------------------------------------- 1 | part of antlr4dart;/// Represents the type of recognizer an ATN applies to. 2 | class AtnType { 3 | 4 | /// A lexer grammar. 5 | static const AtnType LEXER = const AtnType._internal('LEXER'); 6 | 7 | /// A parser grammar. 8 | static const AtnType PARSER = const AtnType._internal('PARSER'); 9 | 10 | static const Map values = const {0: LEXER, 1:PARSER}; 11 | 12 | final name; 13 | 14 | const AtnType._internal(this.name); 15 | 16 | String toString() => name; 17 | } 18 | -------------------------------------------------------------------------------- /lib/src/atn/ll1_analyser.dart: -------------------------------------------------------------------------------- 1 | part of antlr4dart; 2 | 3 | class Ll1Analyzer { 4 | 5 | /// Special value added to the lookahead sets to indicate that we hit 6 | /// a predicate during analysis if `seeThruPreds == false`. 7 | static const int HIT_PRED = Token.INVALID_TYPE; 8 | 9 | final Atn atn; 10 | 11 | Ll1Analyzer(this.atn); 12 | 13 | /// Calculates the SLL(1) expected lookahead set for each outgoing transition 14 | /// of an [AtnState]. 15 | /// 16 | /// The returned array has one element for each outgoing transition in `s`. 17 | /// If the closure from transition **i** leads to a semantic predicate before 18 | /// matching a symbol, the element at index **i** of the result will be `null`. 19 | /// 20 | /// [state] is the ATN state. 21 | /// 22 | /// Return the expected symbols for each outgoing transition of `s`. 23 | List getDecisionLookahead(AtnState state) { 24 | if (state == null) return null; 25 | List look = new List(state.numberOfTransitions); 26 | for (int alt = 0; alt < state.numberOfTransitions; alt++) { 27 | look[alt] = new IntervalSet(); 28 | Set lookBusy = new HashSet(); 29 | bool seeThruPreds = false; // fail to get lookahead upon pred 30 | _look(state.getTransition(alt).target, null, PredictionContext.EMPTY, 31 | look[alt], lookBusy, new BitSet(), seeThruPreds, false); 32 | // Wipe out lookahead for this alternative if we found nothing 33 | // or we had a predicate when we !seeThruPreds 34 | if (look[alt].length == 0 || look[alt].contains(HIT_PRED)) { 35 | look[alt] = null; 36 | } 37 | } 38 | return look; 39 | } 40 | 41 | /// Compute set of tokens that can follow [state] in the ATN in the specified 42 | /// [context]. 43 | /// 44 | /// If [context] is `null` and the end of the rule containing [state] is 45 | /// reached, [Token.EPSILON] is added to the result set. If [context] is not 46 | /// `null` and the end of the outermost rule is reached, [Token.EOF] is added 47 | /// to the result set. 48 | /// 49 | /// [state] is the ATN state. 50 | /// [stopState] is the ATN state to stop at. This can be a [BlockEndState] to 51 | /// detect epsilon paths through a closure. 52 | /// [context] is the complete parser context, or `null` if the context 53 | /// should be ignored. 54 | /// 55 | /// Return the set of tokens that can follow [state] in the ATN in the 56 | /// specified [context]. 57 | IntervalSet look(AtnState state, RuleContext context, [AtnState stopState]) { 58 | IntervalSet r = new IntervalSet(); 59 | bool seeThruPreds = true; // ignore preds; get all lookahead 60 | var lookContext = context != null 61 | ? new PredictionContext.fromRuleContext(state.atn, context) : null; 62 | _look(state, stopState, lookContext, r, 63 | new HashSet(), new BitSet(), seeThruPreds, true); 64 | return r; 65 | } 66 | 67 | /// Compute set of tokens that can follow [state] in the ATN in the 68 | /// specified [context]. 69 | /// 70 | /// If [context] is `null` and [stopState] or the end of the rule containing 71 | /// [state] is reached, [Token.EPSILON] is added to the result set. If 72 | /// [context] is not `null` and [addEof] is `true` and [stopState] or the 73 | /// end of the outermost rule is reached, [Token.EOF] is added to the result 74 | /// set. 75 | /// 76 | /// [state] is the ATN state. 77 | /// [stopState] is the ATN state to stop at. This can be a [BlockEndState] to 78 | /// detect epsilon paths through a closure. 79 | /// [context] is the outer context, or `null` if the outer context should 80 | /// not be used. 81 | /// [look] is the result lookahead set. 82 | /// [lookBusy] is a set used for preventing epsilon closures in the ATN 83 | /// from causing a stack overflow. Outside code should pass 84 | /// `new HashSet()` for this argument. 85 | /// [calledRuleStack] is A set used for preventing left recursion in the 86 | /// ATN from causing a stack overflow. Outside code should pass `new BitSet()` 87 | /// for this argument. 88 | /// [seeThruPreds] is `true` to true semantic predicates as implicitly `true` 89 | /// and "see through them", otherwise `false` to treat semantic predicates as 90 | /// opaque and add [HIT_PRED] to the result if one is encountered. 91 | /// [addEof] tells to add [Token.EOF] to the result if the end of the 92 | /// outermost context is reached. This parameter has no effect if [context] 93 | /// is `null`. 94 | void _look(AtnState state, 95 | AtnState stopState, 96 | PredictionContext context, 97 | IntervalSet look, 98 | Set lookBusy, 99 | BitSet calledRuleStack, 100 | bool seeThruPreds, bool addEof) { 101 | AtnConfig c = new AtnConfig(state, 0, context); 102 | if (!lookBusy.add(c)) return; 103 | if (state == stopState) { 104 | if (context == null) { 105 | look.addSingle(Token.EPSILON); 106 | return; 107 | } else if (context.isEmpty && addEof) { 108 | look.addSingle(Token.EOF); 109 | return; 110 | } 111 | } 112 | if (state is RuleStopState) { 113 | if ( context==null ) { 114 | look.addSingle(Token.EPSILON); 115 | return; 116 | } else if (context.isEmpty && addEof) { 117 | look.addSingle(Token.EOF); 118 | return; 119 | } 120 | if (context != PredictionContext.EMPTY ) { 121 | // run thru all possible stack tops in ctx 122 | for (int i = 0; i < context.length; i++) { 123 | AtnState returnState = atn.states[context.returnStateFor(i)]; 124 | bool removed = calledRuleStack.get(returnState.ruleIndex); 125 | try { 126 | calledRuleStack.set(returnState.ruleIndex); 127 | _look(returnState, stopState, context.parentFor(i), 128 | look, lookBusy, calledRuleStack, seeThruPreds, addEof); 129 | } finally { 130 | if (removed) { 131 | calledRuleStack.set(returnState.ruleIndex, true); 132 | } 133 | } 134 | } 135 | return; 136 | } 137 | } 138 | int n = state.numberOfTransitions; 139 | for (int i = 0; i < n; i++) { 140 | Transition transition = state.getTransition(i); 141 | if (transition.runtimeType == RuleTransition) { 142 | if (calledRuleStack.get(transition.target.ruleIndex)) continue; 143 | PredictionContext newContext = 144 | new SingletonPredictionContext.empty( 145 | context, (transition as RuleTransition).followState.stateNumber); 146 | try { 147 | calledRuleStack.set( 148 | (transition as RuleTransition).target.ruleIndex, true); 149 | _look(transition.target, stopState, newContext, 150 | look, lookBusy, calledRuleStack, seeThruPreds, addEof); 151 | } finally { 152 | calledRuleStack.clear((transition as RuleTransition).target.ruleIndex); 153 | } 154 | } else if (transition is AbstractPredicateTransition) { 155 | if (seeThruPreds) { 156 | _look(transition.target, stopState, context, 157 | look, lookBusy, calledRuleStack, seeThruPreds, addEof); 158 | } else { 159 | look.addSingle(HIT_PRED); 160 | } 161 | } else if (transition.isEpsilon) { 162 | _look(transition.target, stopState, context, look, 163 | lookBusy, calledRuleStack, seeThruPreds, addEof); 164 | } else if (transition.runtimeType == WildcardTransition) { 165 | look.addAll( 166 | IntervalSet.of(Token.MIN_USER_TOKEN_TYPE, atn.maxTokenType)); 167 | } else { 168 | IntervalSet set = transition.label; 169 | if (set != null) { 170 | if (transition is NotSetTransition) { 171 | set = set.complement( 172 | IntervalSet.of(Token.MIN_USER_TOKEN_TYPE, atn.maxTokenType)); 173 | } 174 | look.addAll(set); 175 | } 176 | } 177 | } 178 | } 179 | } -------------------------------------------------------------------------------- /lib/src/atn/semantic_context.dart: -------------------------------------------------------------------------------- 1 | part of antlr4dart; 2 | 3 | /// A tree structure used to record the semantic context in which 4 | /// an ATN configuration is valid. It's either a single predicate, 5 | /// a conjunction `p1 && p2`, or a sum of products `p1 || p2`. 6 | abstract class SemanticContext { 7 | 8 | static final SemanticContext NONE = new Predicate(); 9 | 10 | /// For context independent predicates, we evaluate them without a local 11 | /// context (i.e., null context). That way, we can evaluate them without 12 | /// having to create proper rule-specific context during prediction (as 13 | /// opposed to the parser, which creates them naturally). In a practical 14 | /// sense, this avoids a cast error from [RuleContext] to myruleContext. 15 | /// 16 | /// For context dependent predicates, we must pass in a local context so that 17 | /// references such as `$arg` evaluate properly as `localCtx.arg`. We only 18 | /// capture context dependent predicates in the context in which we begin 19 | /// prediction, so we passed in the outer context here in case of context 20 | /// dependent predicate evaluation. 21 | /// 22 | /// [parser] is the parser instance. 23 | /// [outerContext] is the current parser context object. 24 | bool eval(Recognizer parser, RuleContext outerContext); 25 | 26 | /// Evaluate the precedence predicates for the context and reduce the result. 27 | /// 28 | /// [parser] is the parser instance. 29 | /// [outerContext] is the current parser context object. 30 | /// 31 | /// Return the simplified semantic context after precedence predicates are 32 | /// evaluated, which will be one of the following values: 33 | /// * [NONE]: if the predicate simplifies to `true` after 34 | /// precedence predicates are evaluated. 35 | /// * `null`: if the predicate simplifies to `false` after precedence 36 | /// predicates are evaluated. 37 | /// * `this`: if the semantic context is not changed as a result of precedence 38 | /// predicate evaluation. 39 | /// * A non-`null` [SemanticContext]: the new simplified semantic context 40 | /// after precedence predicates are evaluated. 41 | SemanticContext evalPrecedence(Recognizer parser, RuleContext outerContext) { 42 | return this; 43 | } 44 | 45 | static SemanticContext and(SemanticContext a, SemanticContext b) { 46 | if (a == null || a == NONE) return b; 47 | if (b == null || b == NONE) return a; 48 | And result = new And(a, b); 49 | if (result.operands.length == 1) return result.operands[0]; 50 | return result; 51 | } 52 | 53 | /// See [ParserATNSimulator.predsForAmbigAlts]. 54 | static SemanticContext or(SemanticContext a, SemanticContext b) { 55 | if (a == null) return b; 56 | if (b == null) return a; 57 | if (a == NONE || b == NONE) return NONE; 58 | Or result = new Or(a, b); 59 | if (result.operands.length == 1) return result.operands[0]; 60 | return result; 61 | } 62 | 63 | static _filterPrecPredicates(Set iterable) { 64 | List result = null; 65 | List copy = new List.from(iterable); 66 | for (Iterator iterator = copy.iterator; iterator.moveNext();) { 67 | SemanticContext context = iterator.current; 68 | if (context is PrecedencePredicate) { 69 | if (result == null) result = new List(); 70 | result.add(context); 71 | iterable.remove(context); 72 | } 73 | } 74 | if (result == null) return []; 75 | return result; 76 | } 77 | 78 | static PrecedencePredicate _min(List predicates) { 79 | PrecedencePredicate min = predicates[0]; 80 | for(int i = 1; i < predicates.length; i++) { 81 | if (min.compareTo(predicates[i]) > 0) min = predicates[i]; 82 | } 83 | return min; 84 | } 85 | } 86 | 87 | /// This is the base class for semantic context "operators", which operate on 88 | /// a list of semantic context "operands". 89 | abstract class Operator extends SemanticContext { 90 | 91 | /// The operands for the semantic context operator. 92 | List get operands; 93 | } 94 | 95 | 96 | class Predicate extends SemanticContext { 97 | final int ruleIndex; 98 | final int predIndex; 99 | final bool isCtxDependent; 100 | 101 | Predicate([this.ruleIndex = -1, 102 | this.predIndex = -1, 103 | this.isCtxDependent = false]); 104 | 105 | bool eval(Recognizer parser, RuleContext outerContext) { 106 | RuleContext localctx = isCtxDependent ? outerContext : null; 107 | return parser.semanticPredicate(localctx, ruleIndex, predIndex); 108 | } 109 | 110 | int get hashCode { 111 | int hashCode = MurmurHash.initialize(); 112 | hashCode = MurmurHash.update(hashCode, ruleIndex); 113 | hashCode = MurmurHash.update(hashCode, predIndex); 114 | hashCode = MurmurHash.update(hashCode, isCtxDependent ? 1 : 0); 115 | hashCode = MurmurHash.finish(hashCode, 3); 116 | return hashCode; 117 | } 118 | 119 | bool operator==(Object other) { 120 | return other is Predicate 121 | && ruleIndex == other.ruleIndex 122 | && predIndex == other.predIndex 123 | && isCtxDependent == other.isCtxDependent; 124 | } 125 | 126 | String toString() => "{$ruleIndex:$predIndex}?"; 127 | } 128 | 129 | class And extends Operator { 130 | 131 | final List operands; 132 | 133 | And(SemanticContext a, SemanticContext b) 134 | : operands = new List () { 135 | Set opnds = new HashSet(); 136 | if (a is And) { 137 | opnds.addAll(a.operands); 138 | } else { 139 | operands.add(a); 140 | } 141 | if (b is And) { 142 | opnds.addAll(b.operands); 143 | } else { 144 | opnds.add(b); 145 | } 146 | var precPredicates = SemanticContext._filterPrecPredicates(opnds); 147 | if (!precPredicates.isEmpty) { 148 | // interested in the transition with the lowest precedence 149 | var reduced = SemanticContext._min(precPredicates); 150 | opnds.add(reduced); 151 | } 152 | operands.addAll(opnds); 153 | } 154 | 155 | int get hashCode => MurmurHash.calcHashCode(operands, runtimeType.hashCode); 156 | 157 | bool operator==(Object other) =>other is And && operands == other.operands; 158 | 159 | bool eval(Recognizer parser, RuleContext outerContext) { 160 | for (SemanticContext opnd in operands) { 161 | if (!opnd.eval(parser, outerContext)) return false; 162 | } 163 | return true; 164 | } 165 | 166 | SemanticContext evalPrecedence(Recognizer parser, RuleContext outerContext) { 167 | bool differs = false; 168 | List opnds = new List(); 169 | for (SemanticContext context in operands) { 170 | SemanticContext evaluated = context.evalPrecedence(parser, outerContext); 171 | differs = differs || (evaluated != context); 172 | if (evaluated == null) { 173 | // The AND context is false if any element is false 174 | return null; 175 | } else if (evaluated != SemanticContext.NONE) { 176 | // Reduce the result by skipping true elements 177 | opnds.add(evaluated); 178 | } 179 | } 180 | if (!differs) return this; 181 | if (opnds.isEmpty) { 182 | // all elements were true, so the AND context is true 183 | return SemanticContext.NONE; 184 | } 185 | SemanticContext result = opnds[0]; 186 | for (int i = 1; i < opnds.length; i++) { 187 | result = SemanticContext.and(result, opnds[i]); 188 | } 189 | return result; 190 | } 191 | 192 | String toString() => operands.join('&&'); 193 | } 194 | 195 | class Or extends Operator { 196 | 197 | final List operands; 198 | 199 | Or(SemanticContext a, SemanticContext b) : 200 | operands = new List () { 201 | var opnds = new HashSet(); 202 | if (a is Or) { 203 | opnds.addAll(a.operands); 204 | } else { 205 | opnds.add(a); 206 | } 207 | if (b is Or) { 208 | opnds.addAll(b.operands); 209 | } else { 210 | opnds.add(b); 211 | } 212 | operands.addAll(opnds); 213 | } 214 | 215 | int get hashCode => MurmurHash.calcHashCode(operands, runtimeType.hashCode); 216 | 217 | bool operator==(Object other) => other is Or && operands == other.operands; 218 | 219 | bool eval(Recognizer parser, RuleContext outerContext) { 220 | for (SemanticContext opnd in operands) { 221 | if (opnd.eval(parser, outerContext)) return true; 222 | } 223 | return false; 224 | } 225 | 226 | SemanticContext evalPrecedence(Recognizer parser, RuleContext outerContext) { 227 | bool differs = false; 228 | var opnds = new List(); 229 | for (SemanticContext context in operands) { 230 | SemanticContext evaluated = context.evalPrecedence(parser, outerContext); 231 | differs = differs || (evaluated != context); 232 | if (evaluated == SemanticContext.NONE) { 233 | // The OR context is true if any element is true 234 | return SemanticContext.NONE; 235 | } else if (evaluated != null) { 236 | // Reduce the result by skipping false elements 237 | opnds.add(evaluated); 238 | } 239 | } 240 | if (!differs) return this; 241 | if (opnds.isEmpty) { 242 | // all elements were false, so the OR context is false 243 | return null; 244 | } 245 | SemanticContext result = opnds[0]; 246 | for (int i = 1; i < opnds.length; i++) { 247 | result = SemanticContext.or(result, opnds[i]); 248 | } 249 | return result; 250 | } 251 | 252 | String toString() => operands.join("||"); 253 | } 254 | 255 | class PrecedencePredicate extends SemanticContext 256 | implements Comparable { 257 | 258 | final int precedence; 259 | 260 | PrecedencePredicate([this.precedence = 0]); 261 | 262 | int get hashCode => 31 + precedence; 263 | 264 | bool eval(Recognizer parser, RuleContext outerContext) { 265 | return parser.precedencePredicate(outerContext, precedence); 266 | } 267 | 268 | SemanticContext evalPrecedence(Recognizer parser, RuleContext outerContext) { 269 | return parser 270 | .precedencePredicate(outerContext, precedence) ? SemanticContext.NONE : null; 271 | } 272 | 273 | int compareTo(PrecedencePredicate other) => precedence - other.precedence; 274 | 275 | bool operator==(Object other) { 276 | return other is PrecedencePredicate && precedence == other.precedence; 277 | } 278 | 279 | String toString() => "{$precedence >= prec}?"; 280 | } 281 | -------------------------------------------------------------------------------- /lib/src/atn/transitions.dart: -------------------------------------------------------------------------------- 1 | part of antlr4dart; 2 | 3 | /// An ATN transition between any two ATN states. Subclasses define atom, set, 4 | /// epsilon, action, predicate, rule transitions. 5 | /// 6 | /// This is a one way link. It emanates from a state (usually via a list of 7 | /// transitions) and has a target state. 8 | /// 9 | /// Since we never have to change the ATN transitions once we construct it, 10 | /// we can fix these transitions as specific classes. The DFA transitions 11 | /// on the other hand need to update the labels as it adds transitions to 12 | /// the states. We'll use the term Edge for the DFA to distinguish them from 13 | /// ATN transitions. 14 | abstract class Transition { 15 | 16 | static const int EPSILON = 1; 17 | static const int RANGE = 2; 18 | static const int RULE = 3; 19 | static const int PREDICATE = 4; 20 | static const int ATOM = 5; 21 | static const int ACTION = 6; 22 | static const int SET = 7; 23 | static const int NOT_SET = 8; 24 | static const int WILDCARD = 9; 25 | static const int PRECEDENCE = 10; 26 | 27 | static const List serializationNames = 28 | const ["INVALID", "EPSILON", "RANGE", "RULE", "PREDICATE", "ATOM", 29 | "ACTION", "SET", "NOT_SET", "WILDCARD", "PRECEDENCE" 30 | ]; 31 | 32 | /// The target of this transition. 33 | AtnState target; 34 | 35 | Transition._internal(this.target) { 36 | if (target == null) { 37 | throw new ArgumentError(target); 38 | } 39 | } 40 | 41 | int get serializationType; 42 | 43 | /// Are we epsilon, action or sempred? 44 | bool get isEpsilon => false; 45 | 46 | IntervalSet get label => null; 47 | 48 | bool matches(int symbol, int minVocabSymbol, int maxVocabSymbol); 49 | } 50 | 51 | class RangeTransition extends Transition { 52 | final int from; 53 | final int to; 54 | 55 | RangeTransition(AtnState target, this.from, this.to) 56 | : super._internal(target); 57 | 58 | int get serializationType => Transition.RANGE; 59 | 60 | IntervalSet get label => IntervalSet.of(from, to); 61 | 62 | bool matches(int symbol, int minVocabSymbol, int maxVocabSymbol) { 63 | return symbol >= from && symbol <= to; 64 | } 65 | 66 | String toString() 67 | => "'${new String.fromCharCode(from)}'..'${new String.fromCharCode(to)}'"; 68 | } 69 | 70 | /// A transition containing a set of values. 71 | class SetTransition extends Transition { 72 | final IntervalSet set; 73 | 74 | SetTransition(AtnState target, [IntervalSet set]) 75 | : this.set = (set != null) ? set : IntervalSet.ofSingle(Token.INVALID_TYPE), 76 | super._internal(target); 77 | 78 | int get serializationType => Transition.SET; 79 | 80 | IntervalSet get label => set; 81 | 82 | bool matches(int symbol, int minVocabSymbol, int maxVocabSymbol) { 83 | return set.contains(symbol); 84 | } 85 | 86 | String toString() => set.toString(); 87 | } 88 | 89 | class WildcardTransition extends Transition { 90 | WildcardTransition(AtnState target) : super._internal(target); 91 | 92 | int get serializationType => Transition.WILDCARD; 93 | 94 | bool matches(int symbol, int minVocabSymbol, int maxVocabSymbol) { 95 | return symbol >= minVocabSymbol && symbol <= maxVocabSymbol; 96 | } 97 | 98 | String toString() => "."; 99 | } 100 | 101 | class RuleTransition extends Transition { 102 | 103 | final int precedence; 104 | 105 | /// Ptr to the rule definition object for this rule ref. 106 | final int ruleIndex; 107 | 108 | /// What node to begin computations following ref to rule. 109 | AtnState followState; 110 | 111 | RuleTransition(RuleStartState ruleStart, 112 | this.ruleIndex, 113 | this.precedence, 114 | this.followState) : super._internal(ruleStart); 115 | 116 | int get serializationType => Transition.RULE; 117 | 118 | bool get isEpsilon => true; 119 | 120 | bool matches(int symbol, int minVocabSymbol, int maxVocabSymbol) => false; 121 | } 122 | 123 | /// A tree of semantic predicates from the grammar AST if `label == `SEMPRED`. 124 | /// In the ATN, labels will always be exactly one predicate, but the DFA 125 | /// may have to combine a bunch of them as it collects predicates from 126 | /// multiple ATN configurations into a single DFA state. 127 | class PredicateTransition extends AbstractPredicateTransition { 128 | 129 | final int ruleIndex; 130 | final int predIndex; 131 | final bool isCtxDependent; 132 | 133 | PredicateTransition(AtnState target, 134 | this.ruleIndex, 135 | this.predIndex, 136 | this.isCtxDependent) : super(target); 137 | 138 | int get serializationType => Transition.PREDICATE; 139 | 140 | bool get isEpsilon => true; 141 | 142 | bool matches(int symbol, int minVocabSymbol, int maxVocabSymbol) => false; 143 | 144 | Predicate get predicate { 145 | return new Predicate(ruleIndex, predIndex, isCtxDependent); 146 | } 147 | 148 | String toString() => "pred_$ruleIndex:$predIndex"; 149 | } 150 | 151 | class NotSetTransition extends SetTransition { 152 | 153 | StringSource stringSource; 154 | 155 | NotSetTransition(AtnState target, IntervalSet set) : super(target, set); 156 | 157 | int get serializationType => Transition.NOT_SET; 158 | 159 | bool matches(int symbol, int minVocabSymbol, int maxVocabSymbol) { 160 | return symbol >= minVocabSymbol 161 | && symbol <= maxVocabSymbol 162 | && !super.matches(symbol, minVocabSymbol, maxVocabSymbol); 163 | } 164 | 165 | String toString() => "~${super.toString()}"; 166 | } 167 | 168 | abstract class AbstractPredicateTransition extends Transition { 169 | 170 | AbstractPredicateTransition(AtnState target) : super._internal(target); 171 | 172 | } 173 | 174 | class PrecedencePredicateTransition extends AbstractPredicateTransition { 175 | final int precedence; 176 | 177 | PrecedencePredicateTransition(AtnState target, this.precedence) 178 | : super(target); 179 | 180 | int get serializationType => Transition.PRECEDENCE; 181 | 182 | bool get isEpsilon => true; 183 | 184 | PrecedencePredicate get predicate => new PrecedencePredicate(precedence); 185 | 186 | bool matches(int symbol, int minVocabSymbol, int maxVocabSymbol) => false; 187 | 188 | String toString() => "$precedence >= _p"; 189 | } 190 | 191 | class ActionTransition extends Transition { 192 | final int ruleIndex; 193 | final int actionIndex; 194 | final bool isCtxDependent; 195 | 196 | ActionTransition(AtnState target, 197 | this.ruleIndex, 198 | [this.actionIndex = -1, 199 | this.isCtxDependent = false]) : super._internal(target); 200 | 201 | int get serializationType => Transition.ACTION; 202 | 203 | bool get isEpsilon => true; 204 | 205 | bool matches(int symbol, int minVocabSymbol, int maxVocabSymbol) => false; 206 | 207 | String toString() => "action_$ruleIndex:$actionIndex"; 208 | } 209 | 210 | class AtomTransition extends Transition { 211 | /// The token type or character value; or, signifies special label. 212 | final int especialLabel; 213 | 214 | AtomTransition(AtnState target, this.especialLabel) : super._internal(target); 215 | 216 | int get serializationType => Transition.ATOM; 217 | 218 | IntervalSet get label => IntervalSet.ofSingle(especialLabel); 219 | 220 | bool matches(int symbol, int minVocabSymbol, int maxVocabSymbol) { 221 | return especialLabel == symbol; 222 | } 223 | 224 | String toString() => "$especialLabel"; 225 | } 226 | 227 | class EpsilonTransition extends Transition { 228 | 229 | EpsilonTransition(AtnState target) : super._internal(target); 230 | 231 | int get serializationType => Transition.EPSILON; 232 | 233 | bool get isEpsilon => true; 234 | 235 | bool matches(int symbol, int minVocabSymbol, int maxVocabSymbol) => false; 236 | 237 | String toString() => "epsilon"; 238 | } 239 | -------------------------------------------------------------------------------- /lib/src/contexts.dart: -------------------------------------------------------------------------------- 1 | part of antlr4dart; 2 | 3 | typedef bool IsInstanceOf(RuleContext context); 4 | 5 | /// A rule context is a record of a single rule invocation. It knows which 6 | /// context invoked it, if any. If there is no parent context, the naturally 7 | /// the invoking state is not valid. The parent link provides a chain upwards 8 | /// from the current rule invocation to the root of the invocation tree, 9 | /// forming a stack. We actually carry no information about the rule associated 10 | /// with this context (except when parsing). 11 | /// 12 | /// The parent contexts are useful for computing lookahead sets and getting 13 | /// error information. 14 | /// 15 | /// These objects are used during parsing and prediction. For the special case 16 | /// of parsers and tree parsers, we use the subclass [ParserRuleContext]. 17 | /// 18 | class RuleContext implements RuleNode { 19 | 20 | static final ParserRuleContext EMPTY = new ParserRuleContext(); 21 | 22 | /// What context invoked this rule? 23 | RuleContext parent; 24 | 25 | /// What state invoked the rule associated with this context? 26 | /// 27 | /// If parent is `null`, this should be `-1`. 28 | int invokingState = -1; 29 | 30 | RuleContext([this.parent, this.invokingState = -1]); 31 | 32 | /// A context is empty if there is no invoking state; meaning nobody call 33 | /// current context. 34 | bool get isEmpty => invokingState == -1; 35 | 36 | Interval get sourceInterval => Interval.INVALID; 37 | 38 | RuleContext get ruleContext => this; 39 | 40 | RuleContext get payload => this; 41 | 42 | /// Return the combined text of all child nodes. This method only considers 43 | /// tokens which have been added to the parse tree. 44 | /// 45 | /// Since tokens on hidden channels (e.g. whitespace or comments) are not 46 | /// added to the parse trees, they will not appear in the output of this 47 | /// method. 48 | String get text { 49 | if (childCount == 0) return ""; 50 | StringBuffer sb = new StringBuffer(); 51 | for (int i = 0; i < childCount; i++) { 52 | sb.write(getChild(i).text); 53 | } 54 | return sb.toString(); 55 | } 56 | 57 | int get ruleIndex => -1; 58 | 59 | int get childCount => 0; 60 | 61 | int depth() { 62 | int depth = 0; 63 | RuleContext parent = this; 64 | while (parent != null) { 65 | parent = parent.parent; 66 | depth++; 67 | } 68 | return depth; 69 | } 70 | 71 | ParseTree getChild(int i) => null; 72 | 73 | dynamic accept(ParseTreeVisitor visitor) => visitor.visitChildren(this); 74 | 75 | /// If [asTree] is `true`, return a whole string tree, not just a node, 76 | /// in LISP format (root `child1..childN`) or just a node when this is 77 | /// a leaf. Otherwise, return a string representation this [RuleContext] 78 | /// surrounded by `[]`. 79 | /// 80 | /// [sourceOfRules] could be a [Recognizer] or a [List] of rule names. 81 | String toString([Recognizer sourceOfRules, bool asTree = true]) { 82 | return (sourceOfRules != null && asTree) 83 | ? Trees.toStringTree(this, sourceOfRules) 84 | : _toString(sourceOfRules); 85 | } 86 | 87 | String _toString([sourceOfRules, RuleContext stop]) { 88 | stop = (stop != null) ? stop : RuleContext.EMPTY; 89 | List ruleNames = (sourceOfRules is Recognizer) 90 | ? sourceOfRules.ruleNames 91 | : null; 92 | StringBuffer sb = new StringBuffer("["); 93 | RuleContext parent = this; 94 | while (parent != null && parent != stop) { 95 | if (ruleNames == null) { 96 | if (!parent.isEmpty) sb.write(parent.invokingState); 97 | } else { 98 | int ruleIndex = parent.ruleIndex; 99 | if (ruleIndex >= 0 && ruleIndex < ruleNames.length) { 100 | sb.write(ruleNames[ruleIndex]); 101 | } else { 102 | sb.write(ruleIndex); 103 | } 104 | } 105 | if (parent.parent != null 106 | && (ruleNames != null || !parent.parent.isEmpty)) { 107 | sb.write(" "); 108 | } 109 | parent = parent.parent; 110 | } 111 | sb.write("]"); 112 | return sb.toString(); 113 | } 114 | } 115 | 116 | /// This object is used by the [ParserInterpreter] and is the same as a regular 117 | /// [ParserRuleContext] except that we need to track the rule index of the 118 | /// current context so that we can build parse trees. 119 | class InterpreterRuleContext extends ParserRuleContext { 120 | final int ruleIndex; 121 | 122 | InterpreterRuleContext(ParserRuleContext parent, 123 | int invokingStateNumber, 124 | this.ruleIndex) : super(parent, invokingStateNumber); 125 | 126 | } 127 | 128 | /// A rule invocation record for parsing. 129 | /// 130 | /// Contains all of the information about the current rule not stored in the 131 | /// [RuleContext]. It handles parse tree children list, Any ATN state tracing, 132 | /// and the default values available for rule indications: start, stop, rule 133 | /// index, current alt number, current ATN state. 134 | /// 135 | /// Subclasses made for each rule and grammar track the parameters, return 136 | /// values, locals, and labels specific to that rule. These are the objects 137 | /// that are returned from rules. 138 | /// 139 | class ParserRuleContext extends RuleContext { 140 | 141 | /// If we are debugging or building a parse tree for a visitor, we need to 142 | /// track all of the tokens and rule invocations associated with this rule's 143 | /// context. 144 | /// 145 | /// This is empty for parsing w/o tree constr. operation because we don't 146 | /// the need to track the details about how we parse this rule. 147 | List children; 148 | 149 | Token start, stop; 150 | 151 | /// The exception which forced this rule to return. If the rule 152 | /// successfully completed, this is `null`. 153 | RecognitionException exception; 154 | 155 | ParserRuleContext([ParserRuleContext parent, int invokingStateNumber]) 156 | : super(parent, invokingStateNumber); 157 | 158 | ParserRuleContext.from(ParserRuleContext context) { 159 | parent = context.parent; 160 | invokingState = context.invokingState; 161 | start = context.start; 162 | stop = context.stop; 163 | } 164 | 165 | int get childCount => children != null ? children.length : 0; 166 | 167 | Interval get sourceInterval { 168 | return (start == null || stop == null) 169 | ? Interval.INVALID 170 | : Interval.of(start.tokenIndex, stop.tokenIndex); 171 | } 172 | 173 | void enterRule(ParseTreeListener listener) {} 174 | 175 | void exitRule(ParseTreeListener listener) {} 176 | 177 | /// Does not set parent link; other add methods do that. 178 | dynamic addChild(dynamic child) { 179 | if (child is Token) { 180 | child = new TerminalNode(child); 181 | child.parent = this; 182 | } 183 | if (children == null) children = new List(); 184 | children.add(child); 185 | return child; 186 | } 187 | 188 | /// Used by [Parser.enterOuterAlt] to toss out a [RuleContext] previously 189 | /// added as we entered a rule. If we have # label, we will need to remove 190 | /// generic rule context object. 191 | void removeLastChild() { 192 | if (children != null ) { 193 | children.removeLast(); 194 | } 195 | } 196 | 197 | ErrorNode addErrorNode(Token badToken) { 198 | var node = new ErrorNode(badToken); 199 | addChild(node); 200 | node.parent = this; 201 | return node; 202 | } 203 | 204 | ParseTree getChild(int i) { 205 | return children != null && i >= 0 206 | && i < children.length ? children[i] : null; 207 | } 208 | 209 | dynamic getChildAt(IsInstanceOf isInstanceOf, int i) { 210 | if (children == null || i < 0 || i >= children.length) return null; 211 | int pos = -1; // what element have we found with ctxType? 212 | for (ParseTree child in children) { 213 | if (isInstanceOf(child)) { 214 | pos++; 215 | if (pos == i) return child; 216 | } 217 | } 218 | return null; 219 | } 220 | 221 | TerminalNode getToken(int ttype, int i) { 222 | if (children == null || i < 0 || i >= children.length) return null; 223 | int pos = -1; // what token with ttype have we found? 224 | for (ParseTree child in children) { 225 | if (child is TerminalNode) { 226 | Token symbol = child.symbol; 227 | if (symbol.type == ttype) { 228 | pos++; 229 | if (pos == i) return child; 230 | } 231 | } 232 | } 233 | return null; 234 | } 235 | 236 | List getTokens(int ttype) { 237 | if (children == null) return []; 238 | List tokens = null; 239 | for (ParseTree child in children) { 240 | if (child is TerminalNode) { 241 | Token symbol = child.symbol; 242 | if (symbol.type == ttype) { 243 | if (tokens == null) { 244 | tokens = new List(); 245 | } 246 | tokens.add(child); 247 | } 248 | } 249 | } 250 | return tokens == null ? [] : tokens; 251 | } 252 | 253 | dynamic getRuleContext(IsInstanceOf isInstanceOf, int i) { 254 | return getChildAt(isInstanceOf, i); 255 | } 256 | 257 | List getRuleContexts(IsInstanceOf isInstanceOf) { 258 | if (children == null) return []; 259 | List contexts = null; 260 | for (ParseTree child in children) { 261 | if (isInstanceOf(child)) { 262 | if (contexts == null) { 263 | contexts = new List(); 264 | } 265 | contexts.add(child); 266 | } 267 | } 268 | return contexts == null ? [] : contexts; 269 | } 270 | 271 | // Used for rule context info debugging during parse-time, not so much 272 | // for ATN debugging. 273 | String _toInfoString(Parser recognizer) { 274 | List rules = recognizer.getRuleInvocationStack(this); 275 | rules = rules.reversed; 276 | return "ParserRuleContext$rules{start=$start, stop=$stop}"; 277 | } 278 | } 279 | 280 | -------------------------------------------------------------------------------- /lib/src/deprecation_fix/deprecated_parser_mixin.dart: -------------------------------------------------------------------------------- 1 | part of antlr4dart.deprecation_fix; 2 | /** 3 | * Mixin for back-compatibility of [Parser] with deprecation of [ErrorListener]. 4 | * 5 | * Functions very similarly to [DeprecatedParserMixin] except forwards 6 | * additional parser-specific events. If you are fixing a parser, use this. If 7 | * you are fixing a non-parser recognizer (e.g. lexer) user 8 | * [DeprecatedRecognizerMixin]. Do not use both. 9 | * 10 | * Usage: while users are encouraged to re-write code using [ErrorListener]s 11 | * with the [Stream]-based interface around [Parser].on*, this mixin class 12 | * allows old recognizers to work almost immediately. 13 | * 14 | * [deprecatedMixinInit] must be called before back-compatibility 15 | * works as expected. 16 | * 17 | * Old broken code 18 | * class MyLanguageSpecificParser extends Parser{ 19 | * MyLanguageSpecificParser(); 20 | * 21 | * //stuff involving funcitons removed from the Parser API 22 | * //throw NoSuchMethodErrors 23 | * } 24 | * 25 | * New code 26 | * class MyLanguageSpecificParser extends Parser with DeprecatedParserMixin{ 27 | * MyLanguageSpecificParser(){ 28 | * deprecatedParserMixinInit(); 29 | * //functions removed from the old Parser API now work as expected. 30 | * } 31 | * 32 | * //stuff involving functions removed from the old Parser API execute! 33 | * } 34 | */ 35 | abstract class DeprecatedParserMixin implements DeprecatedRecognizerMixin, Parser{ 36 | List _listeners = new List(); 37 | 38 | /** 39 | * Must be called after parser creation in order for deprecation fixes 40 | * to work. 41 | */ 42 | void deprecatedMixinInit({bool autoAddConsoleErrorListener: true}){ 43 | if (autoAddConsoleErrorListener){ 44 | _listeners.add(ConsoleErrorListener.INSTANCE); 45 | } 46 | //temporary fix. 47 | _subscribeListeners(); 48 | } 49 | 50 | @override 51 | Stream get onSyntaxError; 52 | 53 | /** 54 | * Sets up subscriptions to [errorListener]s. Since the forEach loop is 55 | * inside the subscription, this should allow newly added ErrorListeners 56 | * to still be called. 57 | */ 58 | void _subscribeListeners(){ 59 | onSyntaxError.listen((e) => errorListeners.forEach((el) => el.syntaxError( 60 | e.recognizer, e.offendingSymbol, e.line, e.charPositionInLine, 61 | e.message, e.exception))); 62 | onAmbiguity.listen((e) => errorListeners.forEach((el) => 63 | el.reportAmbiguity(e.recognizer, e.dfa, e.startIndex, 64 | e.stopIndex, e.exact, e.ambigAlts, e.configs))); 65 | onAttemptingFullContext.listen((e) => errorListeners.forEach((el) => 66 | el.reportAttemptingFullContext(e.recognizer, e.dfa, e.startIndex, 67 | e.stopIndex, e.ambigAlts, e.configs))); 68 | onContextSensitivity.listen((e) => errorListeners.forEach((el) => 69 | el.reportContextSensitivity(e.recognizer, e.dfa, e.startIndex, 70 | e.stopIndex, e.prediction, e.configs))); 71 | } 72 | 73 | List get errorListeners => _listeners; 74 | 75 | ErrorListener get errorListenerDispatch { 76 | return new ProxyErrorListener(errorListeners); 77 | } 78 | 79 | /** 80 | * Throws [NullThrownError] if [listener] is `null`. 81 | * Deprecated. Use [onError.listen](void f(Error e)) in new code, where 82 | * f should handle each of the error types in src/errors.dart 83 | */ 84 | void addErrorListener(ErrorListener listener) { 85 | if (listener == null) throw new NullThrownError(); 86 | _listeners.add(listener); 87 | } 88 | 89 | /** 90 | * Deprecated. Save [StreamSubscription]s from [onError.listen] and 91 | * [StreamSubscription.cancel] them when they are not longer required. 92 | */ 93 | void removeErrorListener(ErrorListener listener) { 94 | _listeners.remove(listener); 95 | } 96 | 97 | /** 98 | * Deprecated. No one-for-one replacement, though if all [StreamSubscription]s 99 | * are saved then they can be [StreamSubscription.cancel]ed individually. 100 | * 101 | * Note this function does not cancel subscriptions made through the new 102 | * [Stream] interface. 103 | */ 104 | void removeErrorListeners() { 105 | _listeners.clear(); 106 | } 107 | } -------------------------------------------------------------------------------- /lib/src/deprecation_fix/deprecated_recognizer_mixin.dart: -------------------------------------------------------------------------------- 1 | part of antlr4dart.deprecation_fix; 2 | /** 3 | * Mixin for back-compatibility of Recognizer with deprecation of 4 | * [ErrorListener]. 5 | * 6 | * Functions very similarly to [DeprecatedParserMixin] except only forwards 7 | * [SyntaxError]s. If you are fixing a parser, use the parse mixin. If you are 8 | * fixing a non-parser recognizer (e.g. lexer) user this mixin. Do no use both. 9 | * 10 | * Usage: while users are encouraged to re-write code using [ErrorListener]s 11 | * with the [Stream]-based interface around [Recognizer].on*, this mixin class 12 | * allows old recognizers to work almost immediately. 13 | * 14 | * [deprecatedMixinInit] must be called before back-compatibility 15 | * works as expected. 16 | * 17 | * Old broken code 18 | * class MyLanguageSpecificRecognizer extends Recognizer{ 19 | * MyLanguageSpecificRecognizer(); 20 | * 21 | * //stuff involving funcitons removed from the Recognizer API 22 | * //throw NoSuchMethodErrors 23 | * } 24 | * 25 | * New code 26 | * class MyLanguageSpecificRecognizer extends Recognizer with DeprecatedRecognizerMixin{ 27 | * MyLanguageSpecificRecognizer(){ 28 | * deprecatedRecognizerMixinInit(); 29 | * //functions removed from the Recognizer API now work as expected. 30 | * } 31 | * 32 | * //stuff involving functions removed from the Recognizer API execute! 33 | * } 34 | */ 35 | abstract class DeprecatedRecognizerMixin implements Recognizer{ 36 | List _listeners = new List(); 37 | 38 | /** 39 | * Must be called after recognizer creation in order for deprecation fixes 40 | * to work. 41 | */ 42 | void deprecatedMixinInit({bool autoAddConsoleErrorListener: true}){ 43 | if (autoAddConsoleErrorListener){ 44 | _listeners.add(ConsoleErrorListener.INSTANCE); 45 | } 46 | //temporary fix. 47 | _subscribeListeners(); 48 | } 49 | 50 | /** 51 | * Sets up subscriptions to [errorListener]s. Since the forEach loop is 52 | * inside the subscription, this should allow newly added ErrorListeners 53 | * to still be called. 54 | */ 55 | void _subscribeListeners(){ 56 | onSyntaxError.listen((e) => errorListeners.forEach((el) => el.syntaxError( 57 | e.recognizer, e.offendingSymbol, e.line, e.charPositionInLine, 58 | e.message, e.exception))); 59 | } 60 | 61 | List get errorListeners => _listeners; 62 | 63 | ErrorListener get errorListenerDispatch { 64 | return new ProxyErrorListener(errorListeners); 65 | } 66 | 67 | /** 68 | * Throws [NullThrownError] if [listener] is `null`. 69 | * Deprecated. Use [onError.listen](void f(Error e)) in new code, where 70 | * f should handle each of the error types in src/errors.dart 71 | */ 72 | void addErrorListener(ErrorListener listener) { 73 | if (listener == null) throw new NullThrownError(); 74 | _listeners.add(listener); 75 | } 76 | 77 | /** 78 | * Deprecated. Save [StreamSubscription]s from [onError.listen] and 79 | * [StreamSubscription.cancel] them when they are not longer required. 80 | */ 81 | void removeErrorListener(ErrorListener listener) { 82 | _listeners.remove(listener); 83 | } 84 | 85 | /** 86 | * Deprecated. No one-for-one replacement, though if all [StreamSubscription]s 87 | * are saved then they can be [StreamSubscription.cancel]ed individually. 88 | */ 89 | void removeErrorListeners() { 90 | _listeners.clear(); 91 | } 92 | } -------------------------------------------------------------------------------- /lib/src/deprecation_fix/deprecation_fix.dart: -------------------------------------------------------------------------------- 1 | @deprecated 2 | library antlr4dart.deprecation_fix; 3 | 4 | import 'package:antlr4dart/antlr4dart.dart'; 5 | 6 | import 'dart:async'; 7 | 8 | part 'error_listener.dart'; 9 | part 'deprecated_parser_mixin.dart'; 10 | part 'deprecated_recognizer_mixin.dart'; -------------------------------------------------------------------------------- /lib/src/deprecation_fix/error_listener.dart: -------------------------------------------------------------------------------- 1 | part of antlr4dart.deprecation_fix; 2 | 3 | /** 4 | * How to emit recognition errors. 5 | * 6 | * Use [Recognizer.onError].listen instead. See [Recognizer] documentation for 7 | * details on removing deprecated code. 8 | */ 9 | class ErrorListener { 10 | /// Upon syntax error, notify any interested parties. This is not how to 11 | /// recover from errors or compute error messages. [ErrorStrategy] 12 | /// specifies how to recover from syntax errors and how to compute error 13 | /// messages. This listener's job is simply to emit a computed message, 14 | /// though it has enough information to create its own message in many cases. 15 | /// 16 | /// The [RecognitionException] is non-null for all syntax errors except 17 | /// when we discover mismatched token errors that we can recover from 18 | /// in-line, without returning from the surrounding rule (via the single 19 | /// token insertion and deletion mechanism). 20 | /// 21 | /// [recognizer] is the parser where got the error. From this object, you can 22 | /// access the context as well as the input source. 23 | /// [offendingSymbol] is the offending token in the input token source, 24 | /// unless recognizer is a lexer (then it's `null`). If no viable alternative 25 | /// error, [exception] has token at which we started production for the 26 | /// decision. 27 | /// [line] is the line number in the input where the error occurred. 28 | /// [charPositionInLine] is the character position within that line where 29 | /// the error occurred. 30 | /// [message] is the message to emit. 31 | /// [exception] is the exception generated by the parser that led to the 32 | /// reporting of an error. It is `null` in the case where the parser was 33 | /// able to recover in line without exiting the surrounding rule. 34 | void syntaxError(Recognizer recognizer, 35 | Object offendingSymbol, 36 | int line, 37 | int charPositionInLine, 38 | String message, 39 | RecognitionException exception) {} 40 | 41 | /// This method is called by the parser when a full-context prediction 42 | /// results in an ambiguity. 43 | /// 44 | /// When [exact] is `true`, **all** of the alternatives in [ambigAlts] are 45 | /// viable, i.e. this is reporting an exact ambiguity. 46 | /// [exact] is `false`, **at least two** of the alternatives in [ambigAlts] 47 | /// are viable for the current input, but the prediction algorithm terminated 48 | /// as soon as it determined that at least the **minimum** alternative in 49 | /// [ambigAlts] is viable. 50 | /// 51 | /// When the [PredictionMode.LL_EXACT_AMBIG_DETECTION] prediction mode 52 | /// is used, the parser is required to identify exact ambiguities so 53 | /// [exact] will always be `true`. 54 | /// 55 | /// This method is not used by lexers. 56 | /// 57 | /// [recognizer] is the parser instance. 58 | /// [dfa] is the DFA for the current decision. 59 | /// [startIndex] is the input index where the decision started. 60 | /// [stopIndex] is the input input where the ambiguity is reported. 61 | /// [exact] is `true` if the ambiguity is exactly known, otherwise `false`. 62 | /// This is always `true` when [PredictionMode.LL_EXACT_AMBIG_DETECTION] 63 | /// is used. 64 | /// [ambigAlts] is the potentially ambiguous alternatives. 65 | /// [configs] is the ATN configuration set where the ambiguity was 66 | /// determined. 67 | void reportAmbiguity(Parser recognizer, 68 | Dfa dfa, 69 | int startIndex, 70 | int stopIndex, 71 | bool exact, 72 | BitSet ambigAlts, 73 | AtnConfigSet configs) {} 74 | 75 | /// This method is called when an SLL conflict occurs and the parser is about 76 | /// to use the full context information to make an LL decision. 77 | /// 78 | /// If one or more configurations in [configs] contains a semantic 79 | /// predicate, the predicates are evaluated before this method is called. 80 | /// The subset of alternatives which are still viable after predicates are 81 | /// evaluated is reported in [conflictingAlts]. 82 | /// 83 | /// This method is not used by lexers. 84 | /// 85 | /// [recognizer] is the parser instance. 86 | /// [dfa] is the DFA for the current decision. 87 | /// [startIndex] is the input index where the decision started. 88 | /// [stopIndex] is the input index where the SLL conflict occurred. 89 | /// [conflictingAlts] is the specific conflicting alternatives. If this is 90 | /// `null`, the conflicting alternatives are all alternatives represented 91 | /// in [configs]. 92 | /// [configs] is the ATN configuration set where the SLL conflict was 93 | /// detected. 94 | void reportAttemptingFullContext(Parser recognizer, 95 | Dfa dfa, 96 | int startIndex, 97 | int stopIndex, 98 | BitSet conflictingAlts, 99 | AtnConfigSet configs) {} 100 | 101 | /// This method is called by the parser when a full-context prediction has a 102 | /// unique result. 103 | /// 104 | /// For prediction implementations that only evaluate full-context 105 | /// predictions when an SLL conflict is found (including the default 106 | /// [ParserAtnSimulator] implementation), this method reports cases 107 | /// where SLL conflicts were resolved to unique full-context predictions, 108 | /// i.e. the decision was context-sensitive. This report does not necessarily 109 | /// indicate a problem, and it may appear even in completely unambiguous 110 | /// grammars. 111 | /// 112 | /// [configs] may have more than one represented alternative if the 113 | /// full-context prediction algorithm does not evaluate predicates before 114 | /// beginning the full-context prediction. In all cases, the final prediction 115 | /// is passed as the [prediction] argument. 116 | /// 117 | /// This method is not used by lexers. 118 | /// 119 | /// [recognizer] is the parser instance. 120 | /// [dfa] is the DFA for the current decision. 121 | /// [startIndex] the input index where the decision started. 122 | /// [stopIndex] is the input index where the context sensitivity was 123 | /// finally determined. 124 | /// [prediction] is the unambiguous result of the full-context prediction. 125 | /// [configs] is the ATN configuration set where the unambiguous prediction 126 | /// was determined. 127 | void reportContextSensitivity(Parser recognizer, 128 | Dfa dfa, 129 | int startIndex, 130 | int stopIndex, 131 | int prediction, 132 | AtnConfigSet configs) {} 133 | } 134 | 135 | class ConsoleErrorListener extends ErrorListener { 136 | 137 | static final ConsoleErrorListener INSTANCE = new ConsoleErrorListener(); 138 | 139 | void syntaxError(Recognizer recognizer, 140 | Object offendingSymbol, 141 | int line, 142 | int charPositionInLine, 143 | String msg, 144 | RecognitionException e) { 145 | print("line $line:$charPositionInLine $msg"); 146 | } 147 | } 148 | 149 | //// This implementation of [ErrorListener] dispatches all calls to a 150 | //// collection of delegate listeners. This reduces the effort required 151 | //// to support multiple listeners. 152 | class ProxyErrorListener implements ErrorListener { 153 | final Iterable delegates; 154 | 155 | ProxyErrorListener(this.delegates) { 156 | assert(delegates != null); 157 | } 158 | 159 | void syntaxError(Recognizer recognizer, 160 | Object offendingSymbol, 161 | int line, 162 | int charPositionInLine, 163 | String message, 164 | RecognitionException exception) { 165 | delegates.forEach((delegate) { 166 | delegate.syntaxError(recognizer, 167 | offendingSymbol, 168 | line, 169 | charPositionInLine, 170 | message, 171 | exception); 172 | }); 173 | } 174 | 175 | void reportAmbiguity(Parser recognizer, 176 | Dfa dfa, 177 | int startIndex, 178 | int stopIndex, 179 | bool exact, 180 | BitSet ambigAlts, 181 | AtnConfigSet configs) { 182 | delegates.forEach((delegate) { 183 | delegate.reportAmbiguity(recognizer, 184 | dfa, 185 | startIndex, 186 | stopIndex, 187 | exact, 188 | ambigAlts, 189 | configs); 190 | }); 191 | } 192 | 193 | void reportAttemptingFullContext(Parser recognizer, 194 | Dfa dfa, 195 | int startIndex, 196 | int stopIndex, 197 | BitSet conflictingAlts, 198 | AtnConfigSet configs) { 199 | delegates.forEach((delegate) { 200 | delegate.reportAttemptingFullContext(recognizer, 201 | dfa, 202 | startIndex, 203 | stopIndex, 204 | conflictingAlts, 205 | configs); 206 | }); 207 | } 208 | 209 | void reportContextSensitivity(Parser recognizer, 210 | Dfa dfa, 211 | int startIndex, 212 | int stopIndex, 213 | int prediction, 214 | AtnConfigSet configs) { 215 | delegates.forEach((delegate) { 216 | delegate.reportContextSensitivity(recognizer, 217 | dfa, 218 | startIndex, 219 | stopIndex, 220 | prediction, 221 | configs); 222 | }); 223 | } 224 | } -------------------------------------------------------------------------------- /lib/src/dfa.dart: -------------------------------------------------------------------------------- 1 | part of antlr4dart; 2 | 3 | class Dfa { 4 | 5 | // true if this Dfa is for a precedence decision; otherwise, 6 | // false. This is the backing field for isPrecedenceDfa. 7 | bool _isPrecedenceDfa = false; 8 | 9 | /// A set of all DFA states. Use [Map] so we can get old state back 10 | /// ([Set] only allows you to see if it's there). 11 | final Map states = new HashMap(); 12 | 13 | DfaState s0; 14 | 15 | final int decision; 16 | 17 | /// From which ATN state did we create this DFA? 18 | final DecisionState atnStartState; 19 | 20 | Dfa(this.atnStartState, [this.decision]); 21 | 22 | /// Return a list of all states in this DFA, ordered by state number. 23 | List get orderedStates { 24 | return new List.from(states.keys) 25 | ..sort((o1, o2) => o1.stateNumber - o2.stateNumber); 26 | } 27 | 28 | /// Gets whether this Dfa is a precedence Dfa. 29 | /// 30 | /// Precedence DFAs use a special start state [s0] which is not stored 31 | /// in [states]. The [DfaState.edges] list for this start state contains 32 | /// outgoing edges supplying individual start states corresponding to 33 | /// specific precedence values. 34 | /// 35 | /// Return `true` if this is a precedence DFA; otherwise, `false`. 36 | bool get isPrecedenceDfa => _isPrecedenceDfa; 37 | 38 | /// Sets whether this is a precedence Dfa. 39 | /// 40 | /// If the specified value differs from the current Dfa configuration, the 41 | /// following actions are taken. Otherwise no changes are made to the 42 | /// current DFA. 43 | /// 44 | /// [states] map is cleared. 45 | /// 46 | /// If [isPrecedenceDfa] is `false`, the initial state [s0] is set to `null`. 47 | /// Otherwise, it is initialized to a new [DfaState] with an empty outgoing 48 | /// [DfaState.edges] list to store the start states for individual precedence 49 | /// values. 50 | /// 51 | /// Param [isPrecedenceDfa] is `true` if this is a precedence Dfa. Otherwise, 52 | /// `false`. 53 | void set isPrecedenceDfa(bool isPrecedenceDfa) { 54 | if (_isPrecedenceDfa != isPrecedenceDfa) { 55 | states.clear(); 56 | s0 = isPrecedenceDfa ? 57 | (new DfaState.config(new AtnConfigSet()) 58 | ..edges = new List() 59 | ..isAcceptState = false 60 | ..requiresFullContext = false) 61 | : null; 62 | _isPrecedenceDfa = isPrecedenceDfa; 63 | } 64 | } 65 | 66 | /// Get the start state for a specific precedence value. 67 | /// 68 | /// [precedence] is the current precedence. 69 | /// 70 | /// Return the start state corresponding to the specified precedence, or 71 | /// `null` if no start state exists for the specified precedence. 72 | /// 73 | /// A [StateError] occurs when this is not a precedence Dfa. 74 | DfaState getPrecedenceStartStateFor(int precedence) { 75 | if (!isPrecedenceDfa) { 76 | throw new StateError( 77 | "Only precedence DFAs may contain a precedence start state."); 78 | } 79 | // s0.edges is never null for a precedence Dfa 80 | return (precedence < 0 81 | || precedence >= s0.edges.length) ? null : s0.edges[precedence]; 82 | } 83 | 84 | /// Set the start state for a specific precedence value. 85 | /// 86 | /// [precedence] is the current precedence. 87 | /// [startState] is the start state corresponding to the specified 88 | /// precedence. 89 | /// 90 | /// A [StateError] occurs when his is not a precedence Dfa. 91 | void setPrecedenceStartStateFor(int precedence, DfaState startState) { 92 | if (!isPrecedenceDfa) { 93 | throw new StateError( 94 | "Only precedence DFAs may contain a precedence start state."); 95 | } 96 | if (precedence < 0) return; 97 | // When the DFA is turned into a precedence DFA, s0 will be initialized 98 | // once and not updated again s0.edges is never null for a precedence DFA. 99 | if (precedence >= s0.edges.length) { 100 | int n = precedence - s0.edges.length; 101 | for (int i = 0; i <= n; i++) s0.edges.add(null); 102 | } 103 | s0.edges[precedence] = startState; 104 | } 105 | 106 | String toString([List tokenNames]) { 107 | return (s0 == null) ? "" : ((tokenNames != null) 108 | ? new DfaSerializer(this,tokenNames) 109 | : new LexerDfaSerializer(this)).toString(); 110 | } 111 | } 112 | 113 | /// A DFA state represents a set of possible ATN configurations. 114 | /// 115 | /// As Aho, Sethi, Ullman p. 117 says "The DFA uses its state 116 | /// to keep track of all possible states the ATN can be in after 117 | /// reading each input symbol. That is to say, after reading 118 | /// input a1a2..an, the DFA is in a state that represents the 119 | /// subset T of the states of the ATN that are reachable from the 120 | /// ATN's start state along some path labeled a1a2..an." 121 | /// 122 | /// In conventional NFA->DFA conversion, therefore, the subset T 123 | /// would be a [BitSet] representing the set of states the 124 | /// ATN could be in. 125 | /// 126 | /// We need to track the alt predicted by each state as well, however. 127 | /// More importantly, we need to maintain a stack of states, tracking the 128 | /// closure operations as they jump from rule to rule, emulating rule 129 | /// invocations (method calls). 130 | /// 131 | /// I have to add a stack to simulate the proper lookahead sequences for 132 | /// the underlying LL grammar from which the ATN was derived. 133 | /// 134 | /// I use a set of [AtnConfig] objects not simple states. An [AtnConfig] 135 | /// is both a state (ala normal conversion) and a [RuleContext] describing 136 | /// the chain of rules (if any) followed to arrive at that state. 137 | /// 138 | /// A DFA state may have multiple references to a particular state, 139 | /// but with different ATN contexts (with same or different alts) 140 | /// meaning that state was reached via a different set of rule invocations. 141 | class DfaState { 142 | 143 | int stateNumber = -1; 144 | 145 | AtnConfigSet configs = new AtnConfigSet(); 146 | 147 | /// `edges[symbol]` points to target of symbol. 148 | /// 149 | /// Shift up by 1 so (-1) [Token.EOF] maps to `edges[0]`. 150 | List edges; 151 | 152 | bool isAcceptState = false; 153 | 154 | /// If accept state, what ttype do we match or alt do we predict? 155 | /// 156 | /// This is set to [Atn.INVALID_ALT_NUMBER] when `[predicates] != null` 157 | /// or [requiresFullContext]. 158 | int prediction; 159 | 160 | LexerActionExecutor lexerActionExecutor; 161 | 162 | /// Indicates that this state was created during SLL prediction that 163 | /// discovered a conflict between the configurations in the state. 164 | /// 165 | /// Future [ParserAtnSimulator.execAtn] invocations immediately jumped doing 166 | /// full context prediction if this field is `true`. 167 | bool requiresFullContext = false; 168 | 169 | /// During SLL parsing, this is a list of predicates associated with the 170 | /// ATN configurations of the DFA state. 171 | /// 172 | /// When we have predicates, [requiresFullContext] is `false` since full 173 | /// context prediction evaluates predicates on-the-fly. If this is not `null`, 174 | /// then [prediction] is [Atn.INVALID_ALT_NUMBER]. 175 | /// 176 | /// We only use these for non-[requiresFullContext] but conflicting states. 177 | /// That means we know from the context (it's $ or we don't dip into outer 178 | /// context) that it's an ambiguity not a conflict. 179 | /// 180 | /// This list is computed by the [ParserAtnSimulator]. 181 | List<_PredPrediction> predicates; 182 | 183 | DfaState([this.stateNumber]); 184 | 185 | DfaState.config(this.configs); 186 | 187 | /// Get the set of all alts mentioned by all ATN configurations in this 188 | /// DFA state. 189 | Set get altSet { 190 | Set alts = new HashSet(); 191 | if (configs != null) { 192 | for (AtnConfig c in configs) alts.add(c.alt); 193 | } 194 | return (alts.isEmpty) ? null : alts; 195 | } 196 | 197 | int get hashCode { 198 | int hash = MurmurHash.initialize(7); 199 | hash = MurmurHash.update(hash, configs.hashCode); 200 | return MurmurHash.finish(hash, 1); 201 | } 202 | 203 | /// Two [DfaState] instances are equal if their ATN configuration sets 204 | /// are the same. 205 | /// 206 | /// This method is used to see if a state already exists. 207 | /// 208 | /// Because the number of alternatives and number of ATN configurations are 209 | /// finite, there is a finite number of DFA states that can be processed. 210 | /// This is necessary to show that the algorithm terminates. 211 | /// 212 | /// Cannot test the DFA state numbers here because in 213 | /// [ParserAtnSimulator.addDfaState] we need to know if any other state 214 | /// exists that has this exact set of ATN configurations. The [stateNumber] 215 | /// is irrelevant. 216 | bool operator==(Object other) { 217 | return other is DfaState && configs == other.configs; 218 | } 219 | 220 | String toString() { 221 | StringBuffer sb = new StringBuffer() 222 | ..write(stateNumber) 223 | ..write(":") 224 | ..write(configs); 225 | if (isAcceptState) { 226 | sb 227 | ..write("=>") 228 | ..write(predicates != null ? predicates : prediction); 229 | } 230 | return sb.toString(); 231 | } 232 | } 233 | 234 | /// Map a predicate to a predicted alternative. 235 | class _PredPrediction { 236 | // never null; at least SemanticContext.NONE 237 | SemanticContext pred; 238 | int alt; 239 | _PredPrediction(this.pred, this.alt); 240 | String toString() => "($pred, $alt)"; 241 | } 242 | 243 | 244 | /// A DFA walker that knows how to dump them to serialized strings. 245 | class DfaSerializer { 246 | final Dfa dfa; 247 | final List tokenNames; 248 | 249 | DfaSerializer(this.dfa, this.tokenNames); 250 | 251 | String toString() { 252 | if (dfa.s0 == null) return null; 253 | StringBuffer sb = new StringBuffer(); 254 | List states = dfa.orderedStates; 255 | for (DfaState s in states) { 256 | int n = 0; 257 | if (s.edges != null) n = s.edges.length; 258 | int upperBound = pow(2, 53) - 1; 259 | for (int i = 0; i < n; i++) { 260 | DfaState t = s.edges[i]; 261 | if (t != null && t.stateNumber < upperBound) { 262 | sb..write(_stateStringFor(s)) 263 | ..write("-") 264 | ..write(_edgeLabelFor(i)) 265 | ..write("->") 266 | ..writeln(_stateStringFor(t)); 267 | } 268 | } 269 | } 270 | return sb.toString(); 271 | } 272 | 273 | String _edgeLabelFor(int i) { 274 | return (i == 0) 275 | ? "EOF" 276 | : (tokenNames != null) 277 | ? tokenNames[i-1] 278 | : new String.fromCharCode(i); 279 | } 280 | 281 | String _stateStringFor(DfaState s) { 282 | StringBuffer sb = new StringBuffer(s.isAcceptState ? ':' : '') 283 | ..write('s') 284 | ..write(s.stateNumber) 285 | ..write(s.requiresFullContext ? '^' : ''); 286 | if (s.isAcceptState) { 287 | sb 288 | ..write('=>') 289 | ..write(s.predicates != null ? s.predicates : s.prediction); 290 | } 291 | return sb.toString(); 292 | } 293 | } 294 | 295 | class LexerDfaSerializer extends DfaSerializer { 296 | LexerDfaSerializer(Dfa dfa) : super(dfa, null); 297 | String _edgeLabelFor(int i) => "'${new String.fromCharCode(i)}'"; 298 | } 299 | -------------------------------------------------------------------------------- /lib/src/events.dart: -------------------------------------------------------------------------------- 1 | part of antlr4dart; 2 | 3 | abstract class RecognizerEvent{ 4 | final Recognizer recognizer; 5 | RecognizerEvent(this.recognizer); 6 | } 7 | 8 | abstract class ParserEvent extends RecognizerEvent{ 9 | @override 10 | Parser get recognizer; 11 | 12 | final Dfa dfa; 13 | final int startIndex; 14 | final int stopIndex; 15 | final AtnConfigSet configs; 16 | 17 | ParserEvent(Parser recognizer, this.dfa, this.startIndex, this.stopIndex, 18 | this.configs): super(recognizer); 19 | } 20 | 21 | /// [Error] emitted from a recognizer - generally a [Lexer]. If receiving from 22 | /// a [Parser], this error is usually a [ParserSyntaxError]. 23 | /// 24 | /// The [RecognitionException] is non-null for all syntax errors except 25 | /// when we discover mismatched token errors that we can recover from 26 | /// in-line, without returning from the surrounding rule (via the single 27 | /// token insertion and deletion mechanism). 28 | /// 29 | /// [recognizer] is the recognizer where got the error. From this object, you 30 | /// can access the context as well as the input source. 31 | /// If no viable alternative error, [exception] has token at which we started 32 | /// production for the decision. 33 | /// [line] is the line number in the input where the error occurred. 34 | /// [charPositionInLine] is the character position within that line where 35 | /// the error occurred. 36 | /// [message] is the message to emit. 37 | /// [exception] is the exception generated by the parser that led to the 38 | /// reporting of an error. It is `null` in the case where the parser was 39 | /// able to recover in line without exiting the surrounding rule. 40 | class SyntaxError extends Error implements RecognizerEvent{ 41 | final Recognizer recognizer; 42 | final int line; 43 | final int charPositionInLine; 44 | final String message; 45 | final RecognitionException exception; 46 | 47 | /** 48 | * [ParserSyntaxError]s are a subclass with an [offendingSymbol]. 49 | * These errors likely arose from a [Lexer], which is responsible for 50 | * creating the [Token]s itself, so you should not rely on this [Token] being 51 | * anything except null. If you know this came from a [ParserSyntaxError] 52 | * (which has a non-deprecated offendingSymbol getter), recast this. 53 | */ 54 | @deprecated 55 | Token get offendingSymbol => null; 56 | 57 | SyntaxError( 58 | this.recognizer, 59 | this.line, 60 | this.charPositionInLine, 61 | this.message, 62 | this.exception 63 | ); 64 | 65 | @override 66 | String toString() => "Syntax error, line $line:$charPositionInLine $message"; 67 | } 68 | 69 | /// Specialized [SyntaxError] emitted from a [Parser]. 70 | /// 71 | /// Largely the same, except [offendingSymbol] is the offending token in the 72 | /// input token source. 73 | class ParserSyntaxError extends Error implements SyntaxError{ 74 | final Recognizer recognizer; 75 | 76 | /** 77 | * [ParserSyntaxError]s implement [SyntaxError] along with a reference to the 78 | * [offendingSymbol]. 79 | */ 80 | final Token offendingSymbol; 81 | int get line => offendingSymbol.line; 82 | int get charPositionInLine => offendingSymbol.charPositionInLine; 83 | final String message; 84 | final RecognitionException exception; 85 | 86 | ParserSyntaxError( 87 | this.recognizer, 88 | this.offendingSymbol, 89 | this.message, 90 | this.exception 91 | ); 92 | 93 | @override 94 | String toString() => "Syntax error, line $line:$charPositionInLine $message"; 95 | } 96 | 97 | /// These events are sent by the parser when a full-context prediction 98 | /// results in an ambiguity. 99 | /// 100 | /// When [exact] is `true`, **all** of the alternatives in [ambigAlts] are 101 | /// viable, i.e. this is reporting an exact ambiguity. 102 | /// [exact] is `false`, **at least two** of the alternatives in [ambigAlts] 103 | /// are viable for the current input, but the prediction algorithm terminated 104 | /// as soon as it determined that at least the **minimum** alternative in 105 | /// [ambigAlts] is viable. 106 | /// 107 | /// When the [PredictionMode.LL_EXACT_AMBIG_DETECTION] prediction mode 108 | /// is used, the parser is required to identify exact ambiguities so 109 | /// [exact] will always be `true`. 110 | /// 111 | /// This method is not used by lexers. 112 | /// 113 | /// [recognizer] is the parser instance. 114 | /// [dfa] is the DFA for the current decision. 115 | /// [startIndex] is the input index where the decision started. 116 | /// [stopIndex] is the input input where the ambiguity is reported. 117 | /// [exact] is `true` if the ambiguity is exactly known, otherwise `false`. 118 | /// This is always `true` when [PredictionMode.LL_EXACT_AMBIG_DETECTION] 119 | /// is used. 120 | /// [ambigAlts] is the potentially ambiguous alternatives. 121 | /// [configs] is the ATN configuration set where the ambiguity was 122 | /// determined. 123 | class AmbiguityEvent extends ParserEvent{ 124 | final bool exact; 125 | final BitSet ambigAlts; 126 | 127 | AmbiguityEvent( 128 | Parser recognizer, 129 | Dfa dfa, 130 | int startIndex, 131 | int stopIndex, 132 | this.exact, 133 | this.ambigAlts, 134 | AtnConfigSet configs 135 | ): super(recognizer, dfa, startIndex, stopIndex, configs); 136 | } 137 | 138 | /// These events are fired when an SLL conflict occurs and the parser is about 139 | /// to use the full context information to make an LL decision. 140 | /// 141 | /// If one or more configurations in [configs] contains a semantic 142 | /// predicate, the predicates are evaluated before this method is called. 143 | /// The subset of alternatives which are still viable after predicates are 144 | /// evaluated is reported in [conflictingAlts]. 145 | /// 146 | /// This method is not used by lexers. 147 | /// 148 | /// [recognizer] is the parser instance. 149 | /// [dfa] is the DFA for the current decision. 150 | /// [startIndex] is the input index where the decision started. 151 | /// [stopIndex] is the input index where the SLL conflict occurred. 152 | /// [conflictingAlts] is the specific conflicting alternatives. If this is 153 | /// `null`, the conflicting alternatives are all alternatives represented 154 | /// in [configs]. 155 | /// [configs] is the ATN configuration set where the SLL conflict was 156 | /// detected. 157 | class AttemptingFullContextEvent extends ParserEvent{ 158 | final BitSet ambigAlts; 159 | 160 | AttemptingFullContextEvent( 161 | Parser recognizer, 162 | Dfa dfa, 163 | int startIndex, 164 | int stopIndex, 165 | this.ambigAlts, 166 | AtnConfigSet configs 167 | ): super(recognizer, dfa, startIndex, stopIndex, configs); 168 | } 169 | 170 | /// These events are fired by the parser when a full-context prediction has a 171 | /// unique result. 172 | /// 173 | /// For prediction implementations that only evaluate full-context 174 | /// predictions when an SLL conflict is found (including the default 175 | /// [ParserAtnSimulator] implementation), this method reports cases 176 | /// where SLL conflicts were resolved to unique full-context predictions, 177 | /// i.e. the decision was context-sensitive. This report does not necessarily 178 | /// indicate a problem, and it may appear even in completely unambiguous 179 | /// grammars. 180 | /// 181 | /// [configs] may have more than one represented alternative if the 182 | /// full-context prediction algorithm does not evaluate predicates before 183 | /// beginning the full-context prediction. In all cases, the final prediction 184 | /// is passed as the [prediction] argument. 185 | /// 186 | /// This method is not used by lexers. 187 | /// 188 | /// [recognizer] is the parser instance. 189 | /// [dfa] is the DFA for the current decision. 190 | /// [startIndex] the input index where the decision started. 191 | /// [stopIndex] is the input index where the context sensitivity was 192 | /// finally determined. 193 | /// [prediction] is the unambiguous result of the full-context prediction. 194 | /// [configs] is the ATN configuration set where the unambiguous prediction 195 | /// was determined. 196 | class ContextSensitivityEvent extends ParserEvent{ 197 | final int prediction; 198 | 199 | ContextSensitivityEvent( 200 | Parser recognizer, 201 | Dfa dfa, 202 | int startIndex, 203 | int stopIndex, 204 | this.prediction, 205 | AtnConfigSet configs 206 | ): super(recognizer, dfa, startIndex, stopIndex, configs); 207 | } -------------------------------------------------------------------------------- /lib/src/exceptions.dart: -------------------------------------------------------------------------------- 1 | part of antlr4dart; 2 | 3 | /// The root of the antlr4dart exception hierarchy. 4 | /// 5 | /// In general, antlr4dart tracks just 3 kinds of errors: prediction errors, 6 | /// failed predicate errors, and mismatched input errors. In each case, the 7 | /// parser knows where it is in the input, where it is in the ATN, the rule 8 | /// invocation stack, and what kind of problem occurred. 9 | class RecognitionException implements Exception { 10 | 11 | /// The [Recognizer] where this exception occurred. 12 | /// 13 | /// If the recognizer is not available, this is `null`. 14 | final Recognizer recognizer; 15 | 16 | /// The [RuleContext] at the time this exception was thrown. 17 | final RuleContext context; 18 | 19 | /// The input source which is the symbol source for the recognizer where 20 | /// this exception was thrown. 21 | final InputSource inputSource; 22 | 23 | /// The current [Token] when an error occurred. Since not all sources 24 | /// support accessing symbols by index, we have to track the [Token] 25 | /// instance itself. 26 | Token offendingToken; 27 | 28 | /// The ATN state number the parser was in at the time the error 29 | /// occurred. 30 | /// 31 | /// For [NoViableAltException] and [LexerNoViableAltException] exceptions, 32 | /// this is the [DecisionState] number. For others, it is the state whose 33 | /// outgoing edge we couldn't match. 34 | /// 35 | /// If the state number is not known, this will be `-1`. 36 | int offendingState = -1; 37 | 38 | String message; 39 | 40 | RecognitionException(this.recognizer, 41 | this.inputSource, 42 | this.context, 43 | [this.message]) { 44 | offendingState = (recognizer != null) ? recognizer.state : null; 45 | } 46 | 47 | /// Gets the set of input symbols which could potentially follow the 48 | /// previously matched symbol at the time this exception was thrown. 49 | /// 50 | /// If the set of expected tokens is not known and could not be computed, 51 | /// this method returns `null`. 52 | /// 53 | /// Return the set of token types that could potentially follow the current 54 | /// state in the ATN, or `null` if the information is not available. 55 | IntervalSet get expectedTokens { 56 | if (recognizer != null) { 57 | return recognizer.atn.getExpectedTokens(offendingState, context); 58 | } 59 | return null; 60 | } 61 | } 62 | 63 | /// A semantic predicate failed during validation. 64 | /// 65 | /// Validation of predicates occurs when normally parsing the alternative 66 | /// just like matching a token. 67 | /// 68 | /// Disambiguating predicate evaluation occurs when we test a predicate during 69 | /// prediction. 70 | class FailedPredicateException extends RecognitionException { 71 | 72 | final int ruleIndex; 73 | final int predicateIndex; 74 | final String predicate; 75 | 76 | FailedPredicateException(Parser recognizer, [String predicate, String message]) 77 | : predicate = predicate, 78 | ruleIndex = _ruleIndex(recognizer), 79 | predicateIndex = _predicateIndex(recognizer), 80 | super(recognizer, 81 | recognizer.inputSource, 82 | recognizer.context, 83 | _formatMessage(predicate, message)){ 84 | offendingToken = recognizer.currentToken; 85 | } 86 | 87 | static int _predicateIndex(Parser recognizer) { 88 | AtnState atnState = recognizer.interpreter.atn.states[recognizer.state]; 89 | AbstractPredicateTransition transition = atnState.getTransition(0); 90 | return (transition is PredicateTransition) ? transition.predIndex : 0; 91 | } 92 | 93 | static int _ruleIndex(Parser recognizer) { 94 | AtnState atnState = recognizer.interpreter.atn.states[recognizer.state]; 95 | AbstractPredicateTransition transition = atnState.getTransition(0); 96 | return (transition is PredicateTransition) ? transition.ruleIndex : 0; 97 | } 98 | 99 | static String _formatMessage(String predicate, String message) { 100 | return (message != null) ? message : "failed predicate: {$predicate}?"; 101 | } 102 | } 103 | 104 | /// This signifies any kind of mismatched input exceptions such as when the 105 | /// current input does not match the expected token. 106 | class InputMismatchException extends RecognitionException { 107 | InputMismatchException(Parser recognizer) 108 | : super(recognizer, recognizer.inputSource, recognizer.context) { 109 | offendingToken = recognizer.currentToken; 110 | } 111 | } 112 | 113 | /// Indicates that the parser could not decide which of two or more paths 114 | /// to take based upon the remaining input. 115 | /// 116 | /// It tracks the starting token of the offending input and also knows where 117 | /// the parser was in the various paths when the error. 118 | class NoViableAltException extends RecognitionException { 119 | 120 | /// Which configurations did we try at input.index that couldn't match 121 | /// `input.lookToken(1)`? 122 | final AtnConfigSet deadEndConfigs; 123 | 124 | /// The token object at the start index; the input source might not be 125 | /// buffering tokens so get a reference to it. (At the time the error 126 | /// occurred, of course the source needs to keep a buffer all of the tokens 127 | /// but later we might not have access to those.) 128 | final Token startToken; 129 | 130 | NoViableAltException(Parser recognizer, 131 | {TokenSource inputSource, 132 | Token startToken, 133 | Token offendingToken, 134 | ParserRuleContext context, 135 | this.deadEndConfigs}) 136 | : startToken = _getObject(startToken, recognizer.currentToken), 137 | super(recognizer, 138 | _getObject(inputSource, recognizer.inputSource), 139 | _getObject(context, recognizer.context)) { 140 | this.offendingToken = _getObject(offendingToken, recognizer.currentToken); 141 | } 142 | 143 | static dynamic _getObject(o1, o2) => o1 != null ? o1 : o2; 144 | } 145 | 146 | class LexerNoViableAltException extends RecognitionException { 147 | 148 | /// Matching attempted at what input index? 149 | final int startIndex; 150 | 151 | /// Which configurations did we try at `input.index` that couldn't match 152 | /// `input.lookAhead(1)`? 153 | final AtnConfigSet deadEndConfigs; 154 | 155 | LexerNoViableAltException(Lexer lexer, 156 | StringSource input, 157 | this.startIndex, 158 | this.deadEndConfigs) 159 | : super(lexer, input, null); 160 | 161 | 162 | String toString() { 163 | String symbol = ""; 164 | if (startIndex >= 0 && startIndex < inputSource.length) { 165 | var interval = Interval.of(startIndex,startIndex); 166 | symbol = (inputSource as StringSource).getText(interval); 167 | symbol = symbol.replaceAll('\t', "\\t"); 168 | symbol = symbol.replaceAll('\n', "\\n"); 169 | symbol = symbol.replaceAll('\r', "\\r"); 170 | } 171 | return "$LexerNoViableAltException('$symbol')"; 172 | } 173 | } 174 | -------------------------------------------------------------------------------- /lib/src/input_source.dart: -------------------------------------------------------------------------------- 1 | part of antlr4dart; 2 | 3 | /// A simple source of symbols whose values are represented as integers. This 4 | /// interface provides __marked ranges__ with support for a minimum level 5 | /// of buffering necessary to implement arbitrary lookahead during prediction. 6 | /// For more information on marked ranges, see [mark]. 7 | /// 8 | /// **Initializing Methods:** Some methods in this interface have 9 | /// unspecified behavior if no call to an initializing method has occurred after 10 | /// the source was constructed. The following is a list of initializing methods: 11 | /// 12 | /// * [lookAhead] 13 | /// * [consume] 14 | /// * [length] 15 | abstract class InputSource { 16 | 17 | /// The value returned by [sourceName] when the actual name of the 18 | /// underlying source is not known. 19 | static const String UNKNOWN_SOURCE_NAME = ""; 20 | 21 | /// Returns the total number of symbols in the source, including a single EOF 22 | /// symbol. 23 | int get length; 24 | 25 | /// Gets the name of the underlying symbol source. 26 | /// 27 | /// This method returns a non-null, non-empty string. If such a name is not 28 | /// known, this method returns [UNKNOWN_SOURCE_NAME]. 29 | String get sourceName; 30 | 31 | /// A mark provides a guarantee that [seek] operations will be valid over 32 | /// a "marked range" extending from the index where [mark] was called to 33 | /// the current [index]. This allows the use of input sources by specifying 34 | /// the minimum buffering requirements to support arbitrary lookahead during 35 | /// prediction. 36 | /// 37 | /// The returned mark is an opaque handle (type `int`) which is passed 38 | /// to [release] when the guarantees provided by the marked range are no 39 | /// longer necessary. When access to [mark]/[release] are nested, the 40 | /// marks must be released in reverse order of which they were obtained. 41 | /// Since marked regions are used during performance-critical sections of 42 | /// prediction, the specific behavior of invalid usage is unspecified (i.e. 43 | /// a mark is not released, or a mark is released twice, or marks are not 44 | /// released in reverse order from which they were created). 45 | /// 46 | /// The behavior of this method is unspecified if no call to an [InputSource] 47 | /// initializing method has occurred after this source was constructed. 48 | /// 49 | /// This method does not change the current position in the input source. 50 | /// 51 | /// The following example shows the use of [mark], [release], [index], and 52 | /// [seek] as part of an operation to safely work within a marked region, 53 | /// then restore the source position to its original value and release the mark. 54 | /// 55 | /// InputSource source = ...; 56 | /// int index = -1; 57 | /// int mark = source.mark; 58 | /// try { 59 | /// index = source.index; 60 | /// // perform work here... 61 | /// } finally { 62 | /// if (index != -1) { 63 | /// source.seek(index); 64 | /// } 65 | /// source.release(mark); 66 | /// } 67 | /// 68 | /// Return an opaque marker which should be passed to [release] when the 69 | /// marked range is no longer required. 70 | int get mark; 71 | 72 | /// Return the index into the source of the input symbol referred to by 73 | /// `lookAhead(1)`. 74 | /// 75 | /// The behavior of this method is unspecified if no call to an [InputSource] 76 | /// initializing method has occurred after this source was constructed. 77 | int get index; 78 | 79 | /// Consumes the current symbol in the source. This method has the following 80 | /// effects: 81 | /// 82 | /// * **Forward movement:** The value of [index] before calling this method 83 | /// is less than the value of [index] after calling this method. 84 | /// * **Ordered lookahead:** The value of `lookAhead(1)` before calling this 85 | /// method becomes the value of `lookAhead(1)` after calling this method. 86 | /// 87 | /// Note that calling this method does not guarantee that [index] is 88 | /// incremented by exactly `1`, as that would preclude the ability to 89 | /// implement filtering sources (e.g. [CommonTokenSource] which distinguishes 90 | /// between "on-channel" and "off-channel" tokens). 91 | /// 92 | /// A [StateError] occurs when an attempt is made to consume the the end of 93 | /// the source (i.e. if `lookAhead(1) == EOF` before calling [consume]). 94 | void consume(); 95 | 96 | /// Gets the value of the symbol at offset `i` from the current position. 97 | /// When `i == 1`, this method returns the value of the current symbol in 98 | /// the source (which is the next symbol to be consumed). When `i == -1`, 99 | /// this method returns the value of the previously read symbol in the 100 | /// source. It is not valid to call this method with `i == 0`, but the 101 | /// specific behavior is unspecified because this method is frequently 102 | /// called from performance-critical code. 103 | /// 104 | /// This method is guaranteed to succeed if any of the following are true: 105 | /// 106 | /// * `i > 0` 107 | /// * `i == -1` and [index] returns a value greater than the value of 108 | /// [index] after the source was constructed and `lookAhead(1)` was 109 | /// called in that order. Specifying the current `index` relative to 110 | /// the index after the source was created allows for filtering 111 | /// implementations that do not return every symbol from the underlying 112 | /// source. Specifying the call to `lookAhead(1)` allows for lazily 113 | /// initialized sources. 114 | /// * `lookAhead(i)` refers to a symbol consumed within a marked region 115 | /// that has not yet been released. 116 | /// 117 | /// 118 | /// If `i` represents a position at or beyond the end of the source, 119 | /// this method returns [EOF]. 120 | /// 121 | /// The return value is unspecified if `i < 0` and fewer than `-i` calls 122 | /// to [consume] have occurred from the beginning of the source before 123 | /// calling this method. 124 | /// 125 | /// An [UnsupportedError] occurs when the source does not support retrieving 126 | /// the value of the specified symbol. 127 | int lookAhead(int i); 128 | 129 | /// This method releases a marked range created by a call to [mark]. Calls to 130 | /// [release] must appear in the reverse order of the corresponding access to 131 | /// [mark]. If a mark is released twice, or if marks are not released in 132 | /// reverse order of the corresponding access to [mark], the behavior is 133 | /// unspecified. 134 | /// 135 | /// For more information and an example, see [mark]. 136 | /// 137 | /// [marker] is a marker returned by a call to [mark]. 138 | void release(int marker); 139 | 140 | /// Set the input cursor to the position indicated by [index]. If the 141 | /// specified index lies past the end of the source, the operation behaves as 142 | /// though [index] was the index of the EOF symbol. After this method 143 | /// returns without throwing an exception, the at least one of the following 144 | /// will be true: 145 | /// 146 | /// * [index](InputSource.index) will give access to the index of the first 147 | /// symbol appearing at or after the specified [index]. Specifically, 148 | /// implementations which filter their sources should automatically adjust 149 | /// [index] forward the minimum amount required for the operation to 150 | /// target a non-ignored symbol; 151 | /// * `lookAhead(1)` returns [EOF]. 152 | /// 153 | /// This operation is guaranteed to not throw an exception if [index] 154 | /// lies within a marked region. For more information on marked regions, see 155 | /// [mark]. The behavior of this method is unspecified if no call to 156 | /// an [InputSource] initializing method has occurred after this source 157 | /// was constructed. 158 | /// 159 | /// [index] is the absolute index to seek to. 160 | /// 161 | /// An [ArgumentError] occurs when the [index] is less than `0`. 162 | /// An [UnsupportedError] occurs when the source does not support seeking 163 | /// to the specified index. 164 | void seek(int index); 165 | } 166 | 167 | class StringSource implements InputSource { 168 | 169 | // The data being scanned 170 | List _data; 171 | 172 | // 0..n-1 index into string of next char 173 | int _index = 0; 174 | 175 | /// Line number `1..n` within the input. 176 | int line = 1; 177 | 178 | /// What is name or source? 179 | String sourceName; 180 | 181 | /// Copy data in string to a local int list. 182 | StringSource(String input) { 183 | _data = input.codeUnits; 184 | } 185 | 186 | /// The current input symbol index `0..n` where n indicates the last symbol 187 | /// has been read. 188 | /// 189 | /// The index is the index of char to be returned from `lookAhead(1)`. 190 | int get index => _index; 191 | 192 | int get length => _data.length; 193 | 194 | /// mark/release do nothing; we have entire buffer. 195 | int get mark => -1; 196 | 197 | /// Reset the source so that it's in the same state it was when the object 198 | /// was created *except* the data list is not touched. 199 | void reset() { 200 | _index = 0; 201 | } 202 | 203 | void consume() { 204 | if (_index >= _data.length) { 205 | assert(lookAhead(1) == Token.EOF); 206 | throw new StateError("cannot consume EOF"); 207 | } 208 | _index++; 209 | } 210 | 211 | int lookAhead(int i) { 212 | if (i == 0) return 0; // undefined 213 | if (i < 0) { 214 | // e.g., translate lookAhead(-1) to use offset i = 0; then data[p - 1] 215 | i++; 216 | // invalid; no char before first char 217 | if ((_index + i - 1) < 0) return Token.EOF; 218 | } 219 | if ((_index + i - 1) >= _data.length) return Token.EOF; 220 | return _data[_index + i - 1]; 221 | } 222 | 223 | int lookToken(int i) => lookAhead(i); 224 | 225 | void release(int marker) {} 226 | 227 | void seek(int index) { 228 | if (index <= _index) { 229 | _index = index; // just jump; don't update source state (line, ...) 230 | return; 231 | } 232 | // seek forward, consume until p hits index or n (whichever comes first) 233 | index = min(index, _data.length); 234 | while (_index < index) consume(); 235 | } 236 | 237 | /// This method returns the text for a range of characters within this input 238 | /// source. 239 | /// 240 | /// This method is guaranteed to not throw an exception if the specified 241 | /// [interval] lies entirely within a marked range. For more information 242 | /// about marked ranges, see [InputSource.mark]. 243 | /// 244 | /// [interval] is an interval within the source. 245 | /// 246 | /// Return the text of the specified interval. 247 | /// 248 | /// A [NullThrownError] occurs when [interval] is `null`. 249 | /// An [ArgumentError] occurs when `interval.a < 0`, or 250 | /// `interval.b < interval.a - 1`, or `interval.b` lies at or past the end 251 | /// of the source. 252 | /// An [UnsupportedError] occurs when the source does not support getting 253 | /// the text of the specified interval. 254 | String getText(Interval interval) { 255 | int start = interval._a; 256 | int stop = interval._b; 257 | if (stop >= _data.length) stop = _data.length - 1; 258 | int count = stop - start + 1; 259 | if (start >= _data.length) return ""; 260 | return new String.fromCharCodes(_data.getRange(start, start + count)); 261 | } 262 | 263 | String toString() => new String.fromCharCodes(_data); 264 | } 265 | -------------------------------------------------------------------------------- /lib/src/interpreters.dart: -------------------------------------------------------------------------------- 1 | part of antlr4dart; 2 | 3 | class LexerInterpreter extends Lexer { 4 | final String grammarFileName; 5 | final Atn atn; 6 | 7 | final List tokenNames; 8 | final List ruleNames; 9 | final List modeNames; 10 | 11 | final List decisionToDfa; 12 | final sharedContextCache = new PredictionContextCache(); 13 | 14 | LexerInterpreter(this.grammarFileName, 15 | this.tokenNames, 16 | this.ruleNames, 17 | this.modeNames, 18 | Atn atn, 19 | StringSource input) : super(input), 20 | decisionToDfa = new List(atn.numberOfDecisions), 21 | this.atn = atn { 22 | if (atn.grammarType != AtnType.LEXER) { 23 | throw new ArgumentError("The ATN must be a lexer ATN."); 24 | } 25 | for (int i = 0; i < decisionToDfa.length; i++) { 26 | decisionToDfa[i] = new Dfa(atn.getDecisionState(i), i); 27 | } 28 | interpreter = new LexerAtnSimulator( 29 | atn, decisionToDfa, sharedContextCache, this); 30 | } 31 | } 32 | 33 | /// A parser simulator that mimics what ANTLR's generated parser code does. 34 | /// A [ParserAtnSimulator] is used to make predictions via 35 | /// [Parser.adaptivePredict] but this class moves a pointer through the ATN to 36 | /// simulate parsing. [ParserAtnSimulator] just makes us efficient rather than 37 | /// having to backtrack, for example. 38 | /// 39 | /// This properly creates parse trees even for left recursive rules. 40 | /// 41 | /// We rely on the left recursive rule invocation and special predicate 42 | /// transitions to make left recursive rules work. 43 | /// 44 | class ParserInterpreter extends Parser { 45 | final String grammarFileName; 46 | final Atn atn; 47 | final BitSet pushRecursionContextStates; 48 | 49 | final List decisionToDfa; 50 | final sharedContextCache = new PredictionContextCache(); 51 | 52 | final List tokenNames; 53 | final List ruleNames; 54 | 55 | final parentContextStack = new List>(); 56 | 57 | ParserInterpreter(this.grammarFileName, 58 | this.tokenNames, 59 | this.ruleNames, 60 | Atn atn, 61 | TokenSource input) 62 | : decisionToDfa = new List(atn.numberOfDecisions), 63 | pushRecursionContextStates = new BitSet(), 64 | this.atn = atn, 65 | super(input) { 66 | for (int i = 0; i < decisionToDfa.length; i++) { 67 | decisionToDfa[i] = new Dfa(atn.getDecisionState(i), i); 68 | } 69 | // identify the ATN states where pushNewRecursionContext must be called 70 | for (AtnState state in atn.states) { 71 | if (state is! StarLoopEntryState) continue; 72 | if ((state as StarLoopEntryState).precedenceRuleDecision) { 73 | pushRecursionContextStates.set(state.stateNumber, true); 74 | } 75 | } 76 | // get atn simulator that knows how to do predictions 77 | interpreter = new ParserAtnSimulator( 78 | this, atn, decisionToDfa, sharedContextCache); 79 | } 80 | 81 | /// Begin parsing at [startRuleIndex]. 82 | ParserRuleContext parse(int startRuleIndex) { 83 | RuleStartState startRuleStartState = atn.ruleToStartState[startRuleIndex]; 84 | var rootContext = new InterpreterRuleContext( 85 | null, AtnState.INVALID_STATE_NUMBER, startRuleIndex); 86 | if (startRuleStartState.isPrecedenceRule) { 87 | enterRecursionRule(rootContext, 88 | startRuleStartState.stateNumber, startRuleIndex, 0); 89 | } else { 90 | enterRule(rootContext, startRuleStartState.stateNumber, startRuleIndex); 91 | } 92 | while (true) { 93 | AtnState p = _getAtnState(); 94 | switch (p.stateType) { 95 | case AtnState.RULE_STOP : 96 | // pop; return from rule 97 | if (context.isEmpty) { 98 | if (startRuleStartState.isPrecedenceRule) { 99 | ParserRuleContext result = context; 100 | var parentContext = parentContextStack.removeLast(); 101 | unrollRecursionContexts(parentContext.a); 102 | return result; 103 | } else { 104 | exitRule(); 105 | return rootContext; 106 | } 107 | } 108 | _visitRuleStopState(p); 109 | break; 110 | default : 111 | _visitState(p); 112 | break; 113 | } 114 | } 115 | } 116 | 117 | void enterRecursionRule(ParserRuleContext localctx, 118 | int state, 119 | int ruleIndex, 120 | int precedence) { 121 | parentContextStack.add( 122 | new Pair(context, localctx.invokingState)); 123 | super.enterRecursionRule(localctx, state, ruleIndex, precedence); 124 | } 125 | 126 | AtnState _getAtnState() => atn.states[state]; 127 | 128 | void _visitState(AtnState antState) { 129 | int edge; 130 | if (antState.numberOfTransitions > 1) { 131 | edge = interpreter.adaptivePredict(_input, 132 | (antState as DecisionState).decision, context); 133 | } else { 134 | edge = 1; 135 | } 136 | var transition = antState.getTransition(edge - 1); 137 | switch (transition.serializationType) { 138 | case Transition.EPSILON: 139 | if (pushRecursionContextStates.get(antState.stateNumber) 140 | && (transition.target is! LoopEndState)) { 141 | var ctx = new InterpreterRuleContext(parentContextStack.last.a, 142 | parentContextStack.last.b, context.ruleIndex); 143 | pushNewRecursionContext(ctx, 144 | atn.ruleToStartState[antState.ruleIndex].stateNumber, 145 | context.ruleIndex); 146 | } 147 | break; 148 | case Transition.ATOM: 149 | match(transition.especialLabel); 150 | break; 151 | case Transition.RANGE: 152 | case Transition.SET: 153 | case Transition.NOT_SET: 154 | if (!transition.matches( 155 | _input.lookAhead(1), Token.MIN_USER_TOKEN_TYPE, 65535)) { 156 | errorHandler.recoverInline(this); 157 | } 158 | matchWildcard(); 159 | break; 160 | case Transition.WILDCARD: 161 | matchWildcard(); 162 | break; 163 | case Transition.RULE: 164 | RuleStartState ruleStartState = transition.target; 165 | int ruleIndex = ruleStartState.ruleIndex; 166 | var ctx = new InterpreterRuleContext( 167 | context, antState.stateNumber, ruleIndex); 168 | if (ruleStartState.isPrecedenceRule) { 169 | enterRecursionRule(ctx, ruleStartState.stateNumber, 170 | ruleIndex, transition.precedence); 171 | } else { 172 | enterRule(ctx, transition.target.stateNumber, ruleIndex); 173 | } 174 | break; 175 | case Transition.PREDICATE: 176 | PredicateTransition predicateTransition = transition; 177 | if (!semanticPredicate(context, predicateTransition.ruleIndex, 178 | predicateTransition.predIndex)) { 179 | throw new FailedPredicateException(this); 180 | } 181 | break; 182 | case Transition.ACTION: 183 | ActionTransition actionTransition = transition; 184 | action(context, actionTransition.ruleIndex, actionTransition.actionIndex); 185 | break; 186 | case Transition.PRECEDENCE: 187 | if (!precedencePredicate(context, transition.precedence)) { 188 | throw new FailedPredicateException(this, 189 | "precpred(context, ${transition.precedence})"); 190 | } 191 | break; 192 | default: 193 | throw new UnsupportedError("Unrecognized ATN transition type."); 194 | } 195 | state = transition.target.stateNumber; 196 | } 197 | 198 | void _visitRuleStopState(AtnState p) { 199 | RuleStartState ruleStartState = atn.ruleToStartState[p.ruleIndex]; 200 | if (ruleStartState.isPrecedenceRule) { 201 | var parentContext = parentContextStack.removeLast(); 202 | unrollRecursionContexts(parentContext.a); 203 | state = parentContext.b; 204 | } else { 205 | exitRule(); 206 | } 207 | RuleTransition ruleTransition = atn.states[state].getTransition(0); 208 | state = ruleTransition.followState.stateNumber; 209 | } 210 | } 211 | -------------------------------------------------------------------------------- /lib/src/lexer.dart: -------------------------------------------------------------------------------- 1 | part of antlr4dart; 2 | 3 | /// A lexer is recognizer that draws input symbols from a character source. 4 | /// lexer grammars result in a subclass of this object. A Lexer object 5 | /// uses simplified [AtnSimulator.match] and error recovery mechanisms in 6 | /// the interest of speed. 7 | abstract class Lexer extends Recognizer 8 | implements TokenProvider { 9 | 10 | static const int DEFAULT_MODE = 0; 11 | static const int MORE = -2; 12 | static const int SKIP = -3; 13 | static const int MIN_CHAR_VALUE = 0; 14 | static const int MAX_CHAR_VALUE = 65534; 15 | 16 | final StreamController _syntaxErrorController = 17 | new StreamController.broadcast(sync: true); 18 | 19 | Stream get onSyntaxError => _syntaxErrorController.stream; 20 | 21 | // You can set the text for the current token to override what is in 22 | // the input char buffer. 23 | String _text; 24 | Pair _tokenFacSourcePair; 25 | StringSource _input; 26 | List _modeStack = new List(); 27 | 28 | /// How to create token objects. 29 | TokenFactory tokenFactory = CommonTokenFactory.DEFAULT; 30 | 31 | /// The goal of all lexer rules/methods is to create a token object. 32 | /// This is an instance variable as multiple rules may collaborate to 33 | /// create a single token. [nextToken] will return this object after 34 | /// matching lexer rule(s). If you subclass to allow multiple token 35 | /// emissions, then set this to the last token to be matched or 36 | /// something nonnull so that the auto token emit mechanism will not 37 | /// emit another token. 38 | Token token; 39 | 40 | /// What character index in the source did the current token start at? 41 | /// Needed, for example, to get the text for current token. Set at 42 | /// the start of [nextToken]. 43 | int tokenStartCharIndex = -1; 44 | 45 | /// The line on which the first character of the token resides. 46 | int tokenStartLine; 47 | 48 | /// The character position of first character within the line. 49 | int tokenStartCharPositionInLine; 50 | 51 | /// Once we see `EOF` on char source, next token will be `EOF`. 52 | /// If you have `DONE : EOF;` then you see `DONE EOF`. 53 | bool hitEof = false; 54 | 55 | /// The channel number for the current token. 56 | int channel; 57 | 58 | /// The token type for the current token. 59 | int type; 60 | 61 | int mode = DEFAULT_MODE; 62 | 63 | Lexer(this._input) { 64 | _tokenFacSourcePair = new Pair(this, _input); 65 | } 66 | 67 | /// Set the char source and reset the lexer. 68 | void set inputSource(InputSource source) { 69 | _input = null; 70 | _tokenFacSourcePair = new Pair(this, _input); 71 | reset(); 72 | _input = source; 73 | _tokenFacSourcePair = new Pair(this, _input); 74 | } 75 | 76 | StringSource get inputSource => _input; 77 | 78 | String get sourceName => _input.sourceName; 79 | 80 | int get line => interpreter.line; 81 | 82 | int get charPositionInLine => interpreter.charPositionInLine; 83 | 84 | void set line(int line) { 85 | interpreter.line = line; 86 | } 87 | 88 | void set charPositionInLine(int charPositionInLine) { 89 | interpreter.charPositionInLine = charPositionInLine; 90 | } 91 | 92 | /// What is the index of the current character of lookahead? 93 | int get charIndex => _input.index; 94 | 95 | /// Return the text matched so far for the current token or any 96 | /// text override. 97 | String get text => _text != null ? _text : interpreter.getText(_input); 98 | 99 | /// Set the complete text of this token; it wipes any previous changes to 100 | /// the text. 101 | void set text(String text) { 102 | _text = text; 103 | } 104 | 105 | List get modeNames => null; 106 | 107 | /// Used to print out token names like ID during debugging and 108 | /// error reporting. The generated parsers implement a method 109 | /// that overrides this to point to their `tokenNames`. 110 | List get tokenNames => null; 111 | 112 | /// Return a list of all Token objects in input char source. 113 | /// Forces load of all tokens. Does not include EOF token. 114 | List get allTokens { 115 | List tokens = new List(); 116 | Token token = nextToken(); 117 | while (token.type != Token.EOF) { 118 | tokens.add(token); 119 | token = nextToken(); 120 | } 121 | return tokens; 122 | } 123 | 124 | void reset() { 125 | // wack Lexer state variables 126 | if (_input != null) _input.seek(0); // rewind the input 127 | token = null; 128 | type = Token.INVALID_TYPE; 129 | channel = Token.DEFAULT_CHANNEL; 130 | tokenStartCharIndex = -1; 131 | tokenStartCharPositionInLine = -1; 132 | tokenStartLine = -1; 133 | text = null; 134 | hitEof = false; 135 | mode = DEFAULT_MODE; 136 | _modeStack.clear(); 137 | interpreter.reset(); 138 | } 139 | 140 | /// Return a token from this source; i.e., match a token on the input source. 141 | Token nextToken() { 142 | if (_input == null) { 143 | throw new StateError("nextToken requires a non-null input source."); 144 | } 145 | // Mark start location in char source so unbuffered sources are 146 | // guaranteed at least have text of current token 147 | int tokenStartMarker = _input.mark; 148 | try { 149 | outer: while (true) { 150 | if (hitEof) { 151 | emitEof(); 152 | return token; 153 | } 154 | token = null; 155 | channel = Token.DEFAULT_CHANNEL; 156 | tokenStartCharIndex = _input.index; 157 | tokenStartCharPositionInLine = interpreter.charPositionInLine; 158 | tokenStartLine = interpreter.line; 159 | text = null; 160 | do { 161 | type = Token.INVALID_TYPE; 162 | int ttype; 163 | try { 164 | ttype = interpreter.match(_input, mode); 165 | } on LexerNoViableAltException catch (e) { 166 | notifyListeners(e); // report error 167 | recover(e); 168 | ttype = SKIP; 169 | } 170 | if (_input.lookAhead(1) == Token.EOF) hitEof = true; 171 | if (type == Token.INVALID_TYPE) type = ttype; 172 | if (type == SKIP) continue outer; 173 | } while (type == MORE); 174 | if (token == null) emit(); 175 | return token; 176 | } 177 | } finally { 178 | // make sure we release marker after match or 179 | // unbuffered char source will keep buffering 180 | _input.release(tokenStartMarker); 181 | } 182 | } 183 | 184 | /// Instruct the lexer to skip creating a token for current lexer rule 185 | /// and look for another token. [nextToken] knows to keep looking when 186 | /// a lexer rule finishes with token set to `SKIP_TOKEN`. Recall that 187 | /// if `token == null` at end of any token rule, it creates one for you 188 | /// and emits it. 189 | void skip() { 190 | type = SKIP; 191 | } 192 | 193 | void more() { 194 | type = MORE; 195 | } 196 | 197 | void pushMode(int m) { 198 | _modeStack.add(mode); 199 | mode = m; 200 | } 201 | 202 | int popMode() { 203 | if (_modeStack.isEmpty) throw new StateError(''); 204 | mode = _modeStack.removeLast(); 205 | return mode; 206 | } 207 | 208 | /// By default does not support multiple emits per [nextToken] invocation 209 | /// for efficiency reasons. Subclass and override this method, nextToken, 210 | /// and [token](Lexer.token) (to push tokens into a list and pull from that 211 | /// list rather than a single variable as this implementation does). 212 | void emitToken(Token token) { 213 | this.token = token; 214 | } 215 | 216 | /// The standard method called to automatically emit a token at the 217 | /// outermost lexical rule. The token object should point into the 218 | /// char buffer `start..stop`. If there is a text override in 'text', 219 | /// use that to set the token's text. Override this method to emit 220 | /// custom [Token] objects or provide a new factory. 221 | Token emit() { 222 | Token token = tokenFactory(_tokenFacSourcePair, 223 | type, 224 | _text, 225 | channel, 226 | tokenStartCharIndex, 227 | charIndex - 1, 228 | tokenStartLine, 229 | tokenStartCharPositionInLine); 230 | emitToken(token); 231 | return token; 232 | } 233 | 234 | Token emitEof() { 235 | int pos = charPositionInLine; 236 | // The character position for EOF is one beyond the position of 237 | // the previous token's last character 238 | if (token != null) { 239 | int n = token.stopIndex - token.startIndex + 1; 240 | pos = token.charPositionInLine + n; 241 | } 242 | Token eof = tokenFactory(_tokenFacSourcePair, 243 | Token.EOF, 244 | null, 245 | Token.DEFAULT_CHANNEL, 246 | _input.index, 247 | _input.index - 1, 248 | line, 249 | pos); 250 | emitToken(eof); 251 | return eof; 252 | } 253 | 254 | /// Lexers can normally match any char in it's vocabulary after matching 255 | /// a token, so do the easy thing and just kill a character and hope 256 | /// it all works out. You can instead use the rule invocation stack 257 | /// to do sophisticated error recovery if you are in a fragment rule. 258 | void recover(dynamic e) { 259 | if (e is LexerNoViableAltException) { 260 | if (_input.lookAhead(1) != Token.EOF) { 261 | // skip a char and try again 262 | interpreter.consume(_input); 263 | } 264 | return; 265 | } if (e is RecognitionException) { 266 | _input.consume(); 267 | } 268 | } 269 | 270 | void notifyListeners(LexerNoViableAltException exception) { 271 | String text = _input.getText(Interval.of(tokenStartCharIndex, _input.index)); 272 | String msg = "token recognition error at: '${getErrorDisplay(text)}'"; 273 | _syntaxErrorController.add(new SyntaxError(this, 274 | tokenStartLine, tokenStartCharPositionInLine, msg, exception)); 275 | } 276 | 277 | String getErrorDisplay(String s) => getEscapedErrorDisplay(s); 278 | 279 | ///Use correctly spelled function [getEscapedErrorDisplay] instead. 280 | @deprecated 281 | String getScapedErrorDisplay(String s) => getEscapedErrorDisplay(s); 282 | 283 | String getEscapedErrorDisplay(String s) { 284 | s = s.replaceAll('\n', '\\n'); 285 | s = s.replaceAll('\t', '\\t'); 286 | return s.replaceAll('\r', '\\r'); 287 | } 288 | 289 | String getCharErrorDisplay(String c) => "'$c'"; 290 | } 291 | -------------------------------------------------------------------------------- /lib/src/recognizer.dart: -------------------------------------------------------------------------------- 1 | part of antlr4dart; 2 | 3 | /** 4 | * Some breaking changes in version 0.7 - changed interface from using 5 | * [ErrorListener]s to [Stream]s. Events can now be listened to with greater 6 | * specificity, and more appropriately named. The following [Stream]s can now 7 | * be listened to: 8 | * [Stream]<[SyntaxError]>[onSyntaxError], 9 | * [Stream]<[AmbiguityEvent]>[onAmbiguity], 10 | * [Stream]<[AttemptingFullContextEvent]>[onAttemptingFullContext], and 11 | * [Stream]<[ContextSensitivityEvent]>[onContextSensitivity]. 12 | * 13 | * 14 | * To fix broken code: 15 | * 16 | * Old broken code: 17 | * recognizer.addErrorListener(errorListener); 18 | * // do stuff; 19 | * recognizer.removeErrorListener(errorListener); 20 | * 21 | * Fix: 22 | * var subscription = recognizer.onSyntaxError.listen((e) 23 | * => doSomethingWith(e)); 24 | * // do stuff; 25 | * subscription.cancel(); 26 | * 27 | * Note if errorListener in the above code block has non-trivial implementations 28 | * of other functions e.g. reportContextSensitivity, these will have to be 29 | * subscribed to separately. 30 | * 31 | * Alternatively, you may extend Recognizer with [DeprecatedRecognizerMixin] 32 | * from antlr4dart._deprecation_fix.dart for an immediate fix. This isn't 33 | * advised, as it uses deprecated features that will be removed in future, but 34 | * may be a sufficient quick-fix for projects with a large code-base solution. 35 | * 36 | * Also note as of version 0.7, syntaxerrors are no longer automatically 37 | * [print]ed - you must subscribe if this is what you want: 38 | * recognizer.onSyntaxError.listen(print); 39 | */ 40 | abstract class Recognizer{ 41 | 42 | /** 43 | * [ErrorStrategy] determines how errors are handled - this is purely an 44 | * informative [Stream]. Some or all errors may be successfully handled by 45 | * the [ErrorStrategy] yet still be sent to this [Stream]. 46 | */ 47 | Stream get onSyntaxError; 48 | 49 | static final _tokenTypeMapCache = new HashMap(); 50 | static final _ruleIndexMapCache = new HashMap(); 51 | 52 | AtnInterpreter interpreter; 53 | 54 | /// Indicate that the recognizer has changed internal state that is 55 | /// consistent with the ATN state passed in. This way we always know 56 | /// where we are in the ATN as the parser goes along. The rule context 57 | /// objects form a stack that lets us see the stack of invoking rules. 58 | /// Combine this and we have complete ATN configuration information. 59 | int state = -1; 60 | 61 | Recognizer(); 62 | 63 | 64 | /// Used to print out token names like ID during debugging and 65 | /// error reporting. The generated parsers implement a method 66 | /// that overrides this to point to their List tokenNames. 67 | List get tokenNames; 68 | 69 | List get ruleNames; 70 | 71 | InputSource get inputSource; 72 | 73 | void set inputSource(InputSource input); 74 | 75 | TokenFactory get tokenFactory; 76 | 77 | void set tokenFactory(TokenFactory input); 78 | 79 | /// For debugging and other purposes, might want the grammar name. 80 | /// Have antlr4dart generate an implementation for this method. 81 | String get grammarFileName; 82 | 83 | Atn get atn; 84 | 85 | /// Get a map from token names to token types. 86 | /// 87 | /// Used for tree pattern compilation. 88 | Map get tokenTypeMap { 89 | if (tokenNames == null) { 90 | throw new UnsupportedError( 91 | "The current recognizer does not provide a list of token names."); 92 | } 93 | Map result = _tokenTypeMapCache[tokenNames]; 94 | if (result == null) { 95 | Map result = new HashMap(); 96 | for (int i = 0; i < tokenNames.length; i++) { 97 | result[tokenNames[i]] = i; 98 | } 99 | result["EOF"] = Token.EOF; 100 | result = new UnmodifiableMapView(result); 101 | _tokenTypeMapCache[tokenNames] = result; 102 | } 103 | return result; 104 | } 105 | 106 | /// Get a map from rule names to rule indexes. 107 | /// 108 | /// Used for tree pattern compilation. 109 | Map get ruleIndexMap { 110 | if (ruleNames == null) { 111 | throw new UnsupportedError( 112 | "The current recognizer does not provide a list of rule names."); 113 | } 114 | Map result = _ruleIndexMapCache[ruleNames]; 115 | if (result == null) { 116 | Map m = new HashMap(); 117 | for (int i = 0; i < ruleNames.length; i++) { 118 | m[ruleNames[i]] = i; 119 | } 120 | result = new UnmodifiableMapView(m); 121 | _ruleIndexMapCache[ruleNames] = result; 122 | } 123 | return result; 124 | } 125 | 126 | /// What is the error header, normally line/character position information? 127 | String getErrorHeader(RecognitionException exception) { 128 | int line = exception.offendingToken.line; 129 | int charPositionInLine = exception.offendingToken.charPositionInLine; 130 | return "line $line:$charPositionInLine"; 131 | } 132 | 133 | /// How should a token be displayed in an error message? 134 | /// 135 | /// The default is to display just the text, but during development you might 136 | /// want to have a lot of information spit out. Override in that case 137 | /// to use [token].toString() (which, for [CommonToken], dumps everything 138 | /// about the token). This is better than forcing you to override a method in 139 | /// your token objects because you don't have to go modify your lexer 140 | /// so that it creates a new Dart type. 141 | String getTokenErrorDisplay(Token token) { 142 | if (token == null) return ""; 143 | String s = token.text; 144 | if (s == null) { 145 | s = (token.type == Token.EOF) ? "":"<${token.type}>"; 146 | } 147 | s = s.replaceAll("\n","\\n"); 148 | s = s.replaceAll("\r","\\r"); 149 | s = s.replaceAll("\t","\\t"); 150 | return "'$s'"; 151 | } 152 | 153 | int getTokenType(String tokenName) { 154 | int ttype = tokenTypeMap[tokenName]; 155 | return ttype != null ? ttype : Token.INVALID_TYPE; 156 | } 157 | 158 | /// If this recognizer was generated, it will have a serialized ATN 159 | /// representation of the grammar. 160 | /// 161 | /// For interpreters, we don't know their serialized ATN despite having 162 | /// created the interpreter from it. 163 | String get serializedAtn { 164 | throw new UnsupportedError("there is no serialized ATN"); 165 | } 166 | 167 | /// Subclass needs to override these if there are sempreds or actions that 168 | /// the ATN interpreter needs to execute. 169 | bool semanticPredicate(RuleContext localContext, 170 | int ruleIndex, 171 | int actionIndex) => true; 172 | 173 | bool precedencePredicate(RuleContext localContext, int precedence) => true; 174 | 175 | void action(RuleContext localContext, int ruleIndex, int actionIndex) {} 176 | } 177 | -------------------------------------------------------------------------------- /lib/src/token.dart: -------------------------------------------------------------------------------- 1 | part of antlr4dart; 2 | 3 | /// The default mechanism for creating tokens. It's used by default in [Lexer] 4 | /// and the error handling strategy (to create missing tokens). Notifying the 5 | /// parser of a new factory means that it notifies it's token source and error 6 | /// strategy. 7 | /// 8 | /// This is the method used to create tokens in the lexer and in the error 9 | /// handling strategy. If `text != null`, than the `start` and `stop` positions 10 | /// are wiped to `-1` in the text override is set in the [CommonToken]. 11 | typedef T TokenFactory ( 12 | Pair source, 13 | int type, 14 | String text, 15 | int channel, 16 | int start, 17 | int stop, 18 | int line, 19 | int charPositionInLine); 20 | 21 | 22 | class CommonTokenFactory { 23 | 24 | static final TokenFactory DEFAULT = new CommonTokenFactory(); 25 | 26 | // Copy text for token out of input char source. Useful when input 27 | // source is unbuffered. 28 | final bool _copyText; 29 | 30 | /// Create factory and indicate whether or not the factory copy 31 | /// text out of the char source. 32 | CommonTokenFactory([this._copyText = false]); 33 | 34 | CommonToken call(Pair source, 35 | int type, 36 | String text, 37 | int channel, 38 | int start, 39 | int stop, 40 | int line, 41 | int charPositionInLine) { 42 | CommonToken t = new CommonToken(source, type, channel, start, stop); 43 | t.line = line; 44 | t.charPositionInLine = charPositionInLine; 45 | if (text != null) { 46 | t.text = text; 47 | } else if (_copyText && source.b != null) { 48 | t.text = source.b.getText(Interval.of(start, stop)); 49 | } 50 | return t; 51 | } 52 | 53 | } 54 | 55 | /// A token has properties: text, type, line, character position in the line 56 | /// (so we can ignore tabs), token channel, index, and source from which 57 | /// we obtained this token. 58 | abstract class Token { 59 | 60 | static const int INVALID_TYPE = 0; 61 | 62 | static const int MIN_USER_TOKEN_TYPE = 1; 63 | 64 | /// The value returned by [lookAhead] when the end of the source is 65 | /// reached. 66 | static const int EOF = -1; 67 | 68 | /// During lookahead operations, this "token" signifies we hit rule 69 | /// end ATN state and did not follow it despite needing to. 70 | static const int EPSILON = -2; 71 | 72 | /// All tokens go to the parser (unless skip() is called in that rule) 73 | /// on a particular "channel". The parser tunes to a particular channel 74 | /// so that whitespace etc... can go to the parser on a "hidden" channel. 75 | static const int DEFAULT_CHANNEL = 0; 76 | 77 | /// Anything on different channel than `DEFAULT_CHANNEL` is not parsed 78 | /// by parser. 79 | static const int HIDDEN_CHANNEL = 1; 80 | 81 | /// The text of the token. 82 | String get text; 83 | 84 | /// The token type of the token 85 | int get type; 86 | 87 | /// The line number on which the 1st character of this token 88 | /// was matched, line = 1..n 89 | int get line; 90 | 91 | /// The index of the first character of this token relative to the 92 | /// beginning of the line at which it occurs, 0..n-1 93 | int get charPositionInLine; 94 | 95 | /// Return the channel this token. Each token can arrive at the parser 96 | /// on a different channel, but the parser only "tunes" to a single channel. 97 | /// The parser ignores everything not on `DEFAULT_CHANNEL`. 98 | int get channel => 0; 99 | 100 | /// An index from 0..n-1 of the token object in the token source. 101 | /// This must be valid in order to print token source. 102 | /// 103 | /// Return -1 to indicate that this token was conjured up since 104 | /// it doesn't have a valid index. 105 | int get tokenIndex; 106 | 107 | /// The starting character index of the token 108 | /// This method is optional; return -1 if not implemented. 109 | int get startIndex; 110 | 111 | /// The last character index of the token. 112 | /// This method is optional; return -1 if not implemented. 113 | int get stopIndex; 114 | 115 | /// The [TokenProvider] which created this token. 116 | TokenProvider get tokenProvider; 117 | 118 | /// The [StringSource] from which this token was derived. 119 | StringSource get stringSource; 120 | } 121 | 122 | abstract class WritableToken extends Token { 123 | void set text(String text); 124 | 125 | void set type(int type); 126 | 127 | void set line(int line); 128 | 129 | void set charPositionInLine(int pos); 130 | 131 | void set channel(int channel); 132 | 133 | void set tokenIndex(int index); 134 | } 135 | 136 | class CommonToken implements WritableToken { 137 | static final _EMPTY_SOURCE = new Pair(null, null); 138 | 139 | Pair _source; 140 | 141 | String _text; 142 | 143 | int type; 144 | 145 | int line = 0; 146 | 147 | int charPositionInLine = -1; // set to invalid position 148 | 149 | int channel= Token.DEFAULT_CHANNEL; 150 | 151 | /// What token number is this from 0..n-1 tokens; < 0 implies invalid index. 152 | int tokenIndex = -1; 153 | 154 | /// The char position into the input buffer where this token starts. 155 | int startIndex = 0; 156 | 157 | /// The char position into the input buffer where this token stops. 158 | int stopIndex = 0; 159 | 160 | CommonToken(this._source, 161 | this.type, 162 | this.channel, 163 | this.startIndex, 164 | this.stopIndex) { 165 | if (_source.a != null) { 166 | line = _source.a.line; 167 | charPositionInLine = _source.a.charPositionInLine; 168 | } 169 | } 170 | 171 | CommonToken.ofType(this.type, [String text]) { 172 | if (text != null) { 173 | channel = Token.DEFAULT_CHANNEL; 174 | _text = text; 175 | } 176 | _source = _EMPTY_SOURCE; 177 | } 178 | 179 | CommonToken.from(Token oldToken) { 180 | text = oldToken.text; 181 | type = oldToken.type; 182 | line = oldToken.line; 183 | tokenIndex = oldToken.tokenIndex; 184 | charPositionInLine = oldToken.charPositionInLine; 185 | channel = oldToken.channel; 186 | startIndex = oldToken.startIndex; 187 | stopIndex = oldToken.stopIndex; 188 | if (oldToken is CommonToken) { 189 | _source = oldToken._source; 190 | } else { 191 | _source = new Pair(oldToken.tokenProvider, oldToken.stringSource); 193 | } 194 | } 195 | 196 | String get text { 197 | if (_text != null) return _text; 198 | StringSource input = stringSource; 199 | if (input == null) return null; 200 | int n = input.length; 201 | if (startIndex < n && stopIndex < n) { 202 | return input.getText(Interval.of(startIndex, stopIndex)); 203 | } else { 204 | return ""; 205 | } 206 | } 207 | 208 | /// Override the text for this token. 209 | /// 210 | /// `getText` will return this text rather than pulling from the buffer. 211 | /// 212 | /// Note that this does not mean that start/stop indexes are not valid. 213 | /// It means that that input was converted to a new string in the token 214 | /// object. 215 | void set text(String text) { 216 | _text = text; 217 | } 218 | 219 | TokenProvider get tokenProvider => _source.a; 220 | 221 | StringSource get stringSource => _source.b; 222 | 223 | String toString() { 224 | String channelStr = ""; 225 | if (channel > 0) channelStr = ",channel=$channel"; 226 | String txt = text; 227 | if (txt != null) { 228 | txt = txt.replaceAll("\n","\\n"); 229 | txt = txt.replaceAll("\r","\\r"); 230 | txt = txt.replaceAll("\t","\\t"); 231 | } else { 232 | txt = ""; 233 | } 234 | return "[@$tokenIndex,$startIndex:$stopIndex=" 235 | "'$txt',<$type>$channelStr,$line:$charPositionInLine]"; 236 | } 237 | } 238 | 239 | -------------------------------------------------------------------------------- /lib/src/token_provider.dart: -------------------------------------------------------------------------------- 1 | part of antlr4dart; 2 | 3 | /// A token provider must provide a sequence of tokens via [nextToken] 4 | /// and also must reveal it's source of characters; [CommonToken]'s text is 5 | /// computed from a [StringSource]; it only store indices into the character 6 | /// source. 7 | /// 8 | /// Errors from the lexer are never passed to the parser. Either you want 9 | /// to keep going or you do not upon token recognition error. If you do not 10 | /// want to continue lexing then you do not want to continue parsing. Just 11 | /// throw an exception not under [RecognitionException] and Dart will naturally 12 | /// toss you all the way out of the recognizers. If you want to continue 13 | /// lexing then you should not throw an exception to the parser -- it has 14 | /// already requested a token. Keep lexing until you get a valid one. 15 | /// Just report errors and keep going, looking for a valid token. 16 | abstract class TokenProvider { 17 | 18 | int get line; 19 | 20 | int get charPositionInLine; 21 | 22 | /// From what character source was this token created? You don't have to 23 | /// implement but it's nice to know where a Token comes from if you have 24 | /// include files etc... on the input. 25 | StringSource get inputSource; 26 | 27 | /// Where are you getting tokens from? normally the implication will simply 28 | /// ask lexers input source. 29 | String get sourceName; 30 | 31 | /// Optional method that lets users set factory in lexer or other source 32 | void set tokenFactory(TokenFactory factory); 33 | 34 | /// Gets the factory used for constructing tokens. 35 | TokenFactory get tokenFactory; 36 | 37 | /// Return a Token object from your input source (usually a [StringSource]). 38 | /// Do not fail/return upon lexing error; keep chewing on the characters 39 | /// until you get a good one; errors are not passed through to the parser. 40 | Token nextToken(); 41 | } 42 | -------------------------------------------------------------------------------- /lib/src/token_source.dart: -------------------------------------------------------------------------------- 1 | part of antlr4dart; 2 | 3 | /// An [InputSource] whose symbols are [Token] instances. 4 | abstract class TokenSource extends InputSource { 5 | 6 | /// The underlying [TokenProvider] which provides tokens for this source. 7 | TokenProvider get tokenProvider; 8 | 9 | /// The text of all tokens in the source. 10 | /// 11 | /// This getter behaves like the following code, including potential 12 | /// exceptions from the calls to [InputSource.length] and [getTextIn], 13 | /// but may be optimized by the specific implementation. 14 | /// 15 | /// TokenSource source = ...; 16 | /// String text = source.getTextIn(new Interval(0, source.length)); 17 | String get text; 18 | 19 | /// Get the [Token] instance associated with the value returned by 20 | /// `lookAhead(k)`. This method has the same pre- and post-conditions as 21 | /// [InputSource.lookAhead]. In addition, when the preconditions of this 22 | /// method are met, the return value is non-null and the value of 23 | /// `lookToken(k).type == lookAhead(k)`. 24 | Token lookToken(int k); 25 | 26 | /// Gets the [Token] at the specified `index` in the source. When 27 | /// the preconditions of this method are met, the return value is non-null. 28 | /// 29 | /// The preconditions for this method are the same as the preconditions of 30 | /// [InputSource.seek]. If the behavior of `seek(index)` is unspecified for 31 | /// the current state and given [index], then the behavior of this method 32 | /// is also unspecified. 33 | /// 34 | /// The symbol referred to by [index] differs from [seek] only in the case 35 | /// of filtering sources where [index] lies before the end of the source. 36 | /// Unlike [seek], this method does not adjust [index] to point to a 37 | /// non-ignored symbol. 38 | /// 39 | /// An [ArgumentError] occurs when if [index] is less than 0. 40 | /// An [UnsupportedError] occurs when the source does not support 41 | /// retrieving the token at the specified index 42 | Token get(int index); 43 | 44 | /// Return the text of all tokens within the specified [interval]. 45 | /// 46 | /// This method behaves like the following code (including potential 47 | /// exceptions for violating preconditions of [get], but may be optimized 48 | /// by the specific implementation. 49 | /// 50 | /// TokenSource source = ...; 51 | /// String text = ""; 52 | /// for (int i = interval.a; i <= interval.b; i++) { 53 | /// text += source.get(i).text; 54 | /// } 55 | /// 56 | /// [interval] is the interval of tokens within this source to get text 57 | /// for. 58 | /// 59 | /// Return The text of all tokens within the specified interval in this 60 | /// source. 61 | /// 62 | /// An [NullThrownError] occurs when [interval] is `null`. 63 | String getTextIn(Interval interval); 64 | 65 | /// Return the text of all tokens in this source between [start] and 66 | /// [stop] [Token] (inclusive). 67 | /// 68 | /// If the specified [start] or [stop] token was not provided by this source, 69 | /// or if the [stop] occurred before the [start] token, the behavior is 70 | /// unspecified. 71 | /// 72 | /// For sources which ensure that the [Token.tokenIndex] getter is 73 | /// accurate for all of its provided tokens, this method behaves like the 74 | /// following code. Other sources may implement this method in other ways 75 | /// provided the behavior is consistent with this at a high level. 76 | /// 77 | /// TokenSource source = ...; 78 | /// String text = ""; 79 | /// for (int i = start.tokenIndex; i <= stop.tokenIndex; i++) { 80 | /// text += source.get(i).text; 81 | /// } 82 | /// 83 | /// [start] is the first token in the interval to get text for. 84 | /// [stop] is the last token in the interval to get text for (inclusive). 85 | /// 86 | /// Return the text of all tokens lying between the specified [start] 87 | /// and [stop] tokens. 88 | /// 89 | /// An [UnsupportedError] occurs when this source does not support this 90 | /// method for the specified tokens. 91 | String getText(Token start, Token stop); 92 | } 93 | 94 | /// Buffer all input tokens but do on-demand fetching of new tokens from lexer. 95 | /// Useful when the parser or lexer has to set context/mode info before proper 96 | /// lexing of future tokens. The ST template parser needs this, for example, 97 | /// because it has to constantly flip back and forth between inside/output 98 | /// templates. E.g., `}>` has to parse names as part of an 99 | /// expression but `"hi, "` as a nested template. 100 | class BufferedTokenSource implements TokenSource { 101 | 102 | TokenProvider _tokenProvider; 103 | 104 | // Record every single token pulled from the source so we can reproduce 105 | // chunks of it later. This list captures everything so we can access 106 | // complete input text. 107 | List _tokens = new List(); 108 | 109 | // The index into tokens of the current token (next token to 110 | // consume). tokens[_index] should be lookToken(1). _index =- 1 indicates 111 | // need to initialize with first token. The constructor doesn't get 112 | // a token. First call to lookToken(1) or whatever gets the first token 113 | // and sets _index = 0;. 114 | int _index = -1; 115 | 116 | // Set to true when the EOF token is fetched. Do not continue fetching 117 | // tokens after that point, or multiple EOF tokens could end up in the 118 | // tokens list. 119 | bool _fetchedEof = false; 120 | 121 | BufferedTokenSource(TokenProvider tokenProvider) { 122 | if (tokenProvider == null) throw new NullThrownError(); 123 | _tokenProvider = tokenProvider; 124 | } 125 | 126 | TokenProvider get tokenProvider => _tokenProvider; 127 | 128 | int get index => _index; 129 | 130 | int get mark => 0; 131 | 132 | int get length => _tokens.length; 133 | 134 | List get tokens => _tokens; 135 | 136 | String get sourceName => _tokenProvider.sourceName; 137 | 138 | /// Get the text of all tokens in this buffer. 139 | String get text { 140 | _lazyInit(); 141 | fill(); 142 | return getTextIn(Interval.of(0, length - 1)); 143 | } 144 | 145 | /// Reset this token source by setting its token source. 146 | void set tokenProvider(TokenProvider tokenProvider) { 147 | _tokenProvider = tokenProvider; 148 | _tokens.clear(); 149 | _index = -1; 150 | } 151 | 152 | String getTextIn(Interval interval) { 153 | int start = interval._a; 154 | int stop = interval._b; 155 | if (start < 0 || stop < 0) return ""; 156 | _lazyInit(); 157 | if (stop >= _tokens.length) stop = _tokens.length - 1; 158 | StringBuffer sb = new StringBuffer(); 159 | for (int i = start; i <= stop; i++) { 160 | Token t = _tokens[i]; 161 | if (t.type == Token.EOF) break; 162 | sb.write(t.text); 163 | } 164 | return sb.toString(); 165 | } 166 | 167 | String getText(Token start, Token stop) { 168 | return (start != null && stop != null) 169 | ? getTextIn(Interval.of(start.tokenIndex, stop.tokenIndex)) : ""; 170 | } 171 | 172 | /// Get all tokens from lexer until EOF. 173 | void fill() { 174 | _lazyInit(); 175 | int blockSize = 1000; 176 | while (true) { 177 | int fetched = _fetch(blockSize); 178 | if (fetched < blockSize) return; 179 | } 180 | } 181 | 182 | /// Given a start and stop index, return a List of all tokens in the token 183 | /// type [BitSet]. Return null if no tokens were found. 184 | /// 185 | /// This method looks at both on and off channel tokens. 186 | List getTokens(int start, int stop, [Set types]) { 187 | _lazyInit(); 188 | if (start < 0 189 | || stop >= _tokens.length 190 | || stop < 0 191 | || start >= _tokens.length) { 192 | throw new RangeError( 193 | "start $start or stop $stop not in 0..${_tokens.length - 1}"); 194 | } 195 | if (start > stop) return null; 196 | List filteredTokens = new List(); 197 | for (int i = start; i <= stop; i++) { 198 | Token token = _tokens[i]; 199 | if (types == null || types.contains(token.type)) { 200 | filteredTokens.add(token); 201 | } 202 | } 203 | if (filteredTokens.isEmpty) filteredTokens = null; 204 | return filteredTokens; 205 | } 206 | 207 | void release(int marker) {} 208 | 209 | void reset() { 210 | seek(0); 211 | } 212 | 213 | void seek(int index) { 214 | _lazyInit(); 215 | _index = adjustSeekIndex(index); 216 | } 217 | 218 | void consume() { 219 | bool skipEofCheck; 220 | if (_index >= 0) { 221 | if (_fetchedEof) { 222 | // the last token in tokens is EOF. skip check if p indexes any 223 | // fetched token except the last. 224 | skipEofCheck = _index < tokens.length - 1; 225 | } else { 226 | // no EOF token in tokens. skip check if p indexes a fetched token. 227 | skipEofCheck = _index < tokens.length; 228 | } 229 | } else { 230 | // not yet initialized 231 | skipEofCheck = false; 232 | } 233 | if (!skipEofCheck && lookAhead(1) == Token.EOF) { 234 | throw new StateError("cannot consume EOF"); 235 | } 236 | if (_sync(_index + 1)) { 237 | _index = adjustSeekIndex(_index + 1); 238 | } 239 | } 240 | 241 | Token get(int i) { 242 | if (i < 0 || i >= _tokens.length) { 243 | throw new RangeError( 244 | "token index $i out of range 0..${_tokens.length - 1}"); 245 | } 246 | return _tokens[i]; 247 | } 248 | 249 | /// Get all tokens from start..stop inclusively 250 | List getRange(int start, int stop) { 251 | if (start < 0 || stop < 0) return null; 252 | _lazyInit(); 253 | List subset = new List(); 254 | if (stop >= _tokens.length) stop = _tokens.length - 1; 255 | for (int i = start; i <= stop; i++) { 256 | Token t = _tokens[i]; 257 | if (t.type == Token.EOF) break; 258 | subset.add(t); 259 | } 260 | return subset; 261 | } 262 | 263 | int lookAhead(int i) => lookToken(i).type; 264 | 265 | Token lookToken(int k) { 266 | _lazyInit(); 267 | if (k == 0) return null; 268 | if (k < 0) return _lookBack(-k); 269 | int i = _index + k - 1; 270 | _sync(i); 271 | // EOF must be last token 272 | if (i >= _tokens.length) return _tokens.last; 273 | return _tokens[i]; 274 | } 275 | 276 | /// Collect all tokens on specified channel to the right of the current token 277 | /// up until we see a token on `DEFAULT_TOKEN_CHANNEL` or `EOF`. If channel 278 | /// is `-1`, find any non default channel token. 279 | List getHiddenTokensToRight(int tokenIndex, [int channel = -1]) { 280 | _lazyInit(); 281 | if ( tokenIndex<0 || tokenIndex >= _tokens.length) { 282 | throw new RangeError("$tokenIndex not in 0..${_tokens.length - 1}"); 283 | } 284 | int nextOnChannel = _nextTokenOnChannel( 285 | tokenIndex + 1, Token.DEFAULT_CHANNEL); 286 | int to; 287 | int from = tokenIndex+1; 288 | // if none onchannel to right, nextOnChannel=-1 so set to = last token 289 | if (nextOnChannel == -1) { 290 | to = length - 1; 291 | } else { 292 | to = nextOnChannel; 293 | } 294 | return _filterForChannel(from, to, channel); 295 | } 296 | 297 | /// Collect all tokens on specified channel to the left of the current token 298 | /// up until we see a token on `DEFAULT_TOKEN_CHANNEL`. If channel is `-1`, 299 | /// find any non default channel token. 300 | List getHiddenTokensToLeft(int tokenIndex, [int channel = -1]) { 301 | _lazyInit(); 302 | if (tokenIndex < 0 || tokenIndex >= _tokens.length) { 303 | throw new RangeError("$tokenIndex not in 0..${_tokens.length - 1}"); 304 | } 305 | // obviously no tokens can appear before the first token 306 | if (tokenIndex == 0) return null; 307 | int prevOnChannel = _previousTokenOnChannel( 308 | tokenIndex - 1, Token.DEFAULT_CHANNEL); 309 | if (prevOnChannel == tokenIndex - 1) return null; 310 | // if none onchannel to left, prevOnChannel=-1 then from=0 311 | int from = prevOnChannel + 1; 312 | int to = tokenIndex-1; 313 | return _filterForChannel(from, to, channel); 314 | } 315 | 316 | /// Allowed derived classes to modify the behavior of operations which change 317 | /// the current source position by adjusting the target token index of a seek 318 | /// operation. The default implementation simply returns i. If an exception 319 | /// is thrown in this method, the current source index should not be changed. 320 | /// 321 | /// For example, [CommonTokenSource] overrides this method to ensure that 322 | /// the seek target is always an on-channel token. 323 | /// 324 | /// [i] is the target token index. 325 | /// 326 | /// Return the adjusted target token index. 327 | int adjustSeekIndex(int i) => i; 328 | 329 | // Make sure index i in _tokens has a token. 330 | // Return true if a token is located at index i, otherwise false. 331 | bool _sync(int i) { 332 | assert(i >= 0); 333 | int n = i - _tokens.length + 1; // how many more elements we need? 334 | if (n > 0) { 335 | int fetched = _fetch(n); 336 | return fetched >= n; 337 | } 338 | return true; 339 | } 340 | 341 | // Add n elements to buffer. 342 | // Return the actual number of elements added to the buffer. 343 | int _fetch(int n) { 344 | if (_fetchedEof) return 0; 345 | for (int i = 0; i < n; i++) { 346 | Token token = _tokenProvider.nextToken(); 347 | if (token is WritableToken) { 348 | token.tokenIndex = _tokens.length; 349 | } 350 | _tokens.add(token); 351 | if (token.type == Token.EOF) { 352 | _fetchedEof = true; 353 | return i + 1; 354 | } 355 | } 356 | return n; 357 | } 358 | 359 | Token _lookBack(int k) => (_index - k) < 0 ? null : _tokens[_index - k]; 360 | 361 | void _lazyInit() { 362 | if (_index == -1) { 363 | _sync(0); 364 | _index = adjustSeekIndex(0); 365 | } 366 | } 367 | 368 | // Given a starting index, return the index of the next token on channel. 369 | // Return i if _tokens[i] is on channel. Return -1 if there are no tokens 370 | // on channel between i and EOF. 371 | int _nextTokenOnChannel(int i, int channel) { 372 | _sync(i); 373 | Token token = _tokens[i]; 374 | if (i >= length) return -1; 375 | while (token.channel != channel) { 376 | if (token.type == Token.EOF) return -1; 377 | i++; 378 | _sync(i); 379 | token = _tokens[i]; 380 | } 381 | return i; 382 | } 383 | 384 | // Given a starting index, return the index of the previous token on 385 | // channel. Return i if tokens[i] is on channel. Return -1 if there are 386 | // no tokens on channel between i and 0. 387 | // 388 | // If i specifies an index at or after the EOF token, the EOF token 389 | // index is returned. This is due to the fact that the EOF token is treated 390 | // as though it were on every channel. 391 | int _previousTokenOnChannel(int i, int channel) { 392 | _sync(i); 393 | // the EOF token is on every channel 394 | if (i >= length) return length - 1; 395 | while (i >= 0) { 396 | Token token = tokens[i]; 397 | if (token.type == Token.EOF || token.channel == channel) return i; 398 | i--; 399 | } 400 | return i; 401 | } 402 | 403 | List _filterForChannel(int from, int to, int channel) { 404 | List hidden = new List(); 405 | for (int i = from; i <= to; i++) { 406 | Token t = _tokens[i]; 407 | if (channel == -1) { 408 | if (t.channel != Token.DEFAULT_CHANNEL) hidden.add(t); 409 | } else { 410 | if (t.channel == channel) hidden.add(t); 411 | } 412 | } 413 | if (hidden.length == 0) return null; 414 | return hidden; 415 | } 416 | } 417 | 418 | /// The most common source of tokens where every token is buffered up 419 | /// and tokens are filtered for a certain channel (the parser will only 420 | /// see these tokens). 421 | /// 422 | /// Even though it buffers all of the tokens, this token source pulls tokens 423 | /// from the tokens source on demand. In other words, until you ask for a 424 | /// token using [consume], [lookToken], etc. the source does not pull from 425 | /// the lexer. 426 | /// 427 | /// The only difference between this source and [BufferedTokenSource] superclass 428 | /// is that this source knows how to ignore off channel tokens. There may be 429 | /// a performance advantage to using the superclass if you don't pass 430 | /// whitespace and comments etc. to the parser on a hidden channel (i.e., 431 | /// you set `channel` instead of calling [Lexer.skip] in lexer rules.) 432 | class CommonTokenSource extends BufferedTokenSource { 433 | 434 | // Skip tokens on any channel but this one; this is how we skip whitespace... 435 | int _channel; 436 | 437 | CommonTokenSource(TokenProvider tokenProvider, 438 | [this._channel = Token.DEFAULT_CHANNEL]) 439 | : super(tokenProvider); 440 | 441 | /// Count EOF just once. 442 | int get numberOfOnChannelTokens { 443 | int n = 0; 444 | fill(); 445 | for (int i = 0; i < _tokens.length; i++) { 446 | Token t = _tokens[i]; 447 | if (t.channel == _channel) n++; 448 | if (t.type == Token.EOF) break; 449 | } 450 | return n; 451 | } 452 | 453 | int adjustSeekIndex(int i) { 454 | return _nextTokenOnChannel(i, _channel); 455 | } 456 | 457 | Token lookToken(int k) { 458 | _lazyInit(); 459 | if (k == 0) return null; 460 | if (k < 0) return _lookBack(-k); 461 | int i = _index; 462 | int n = 1; // we know tokens[p] is a good one 463 | // find k good tokens 464 | while (n < k) { 465 | // skip off-channel tokens, but make sure to not look past EOF 466 | if (_sync(i + 1)) { 467 | i = _nextTokenOnChannel(i + 1, _channel); 468 | } 469 | n++; 470 | } 471 | return _tokens[i]; 472 | } 473 | 474 | Token _lookBack(int k) { 475 | if (k == 0 || (_index - k) < 0) return null; 476 | int i = _index; 477 | int n = 1; 478 | // find k good tokens looking backwards 479 | while (n <= k) { 480 | // skip off-channel tokens 481 | i = _previousTokenOnChannel(i - 1, _channel); 482 | n++; 483 | } 484 | if (i < 0) return null; 485 | return _tokens[i]; 486 | } 487 | } 488 | 489 | -------------------------------------------------------------------------------- /lib/src/tree/parse_tree.dart: -------------------------------------------------------------------------------- 1 | part of antlr4dart; 2 | 3 | abstract class ParseTreeListener { 4 | void visitTerminal(TerminalNode terminalNode); 5 | void visitErrorNode(ErrorNode errorNode); 6 | void enterEveryRule(ParserRuleContext context); 7 | void exitEveryRule(ParserRuleContext context); 8 | } 9 | 10 | /// The basic notion of a tree has a parent, a payload, and a list of children. 11 | /// 12 | /// It is the most abstract class for all the trees used by antlr4dart. 13 | abstract class Tree { 14 | /// The parent of this node. 15 | /// 16 | /// If `null`, then this node is the root of the tree. 17 | Tree get parent; 18 | 19 | /// Whatever object represents the data at this note. For example, for parse 20 | /// trees, the payload can be a [Token] representing a leaf node or a 21 | /// [RuleContext] object representing a rule invocation. 22 | /// 23 | /// For abstract syntax trees (ASTs), this is a [Token] object. 24 | Object get payload; 25 | 26 | /// How many children are there? If there is none, then this node represents 27 | /// a leaf node. 28 | int get childCount; 29 | 30 | /// If there are children, get the `i`th value indexed from `0`. 31 | Tree getChild(int i); 32 | 33 | /// Print out a whole tree, not just a node, in LISP format 34 | /// `(root child1..childN)` or just a node when this is a leaf. 35 | String toString(); 36 | } 37 | 38 | /// A tree that knows about an interval in a token source is some kind of 39 | /// syntax tree. Subclasses distinguish between parse trees and other kinds 40 | /// of syntax trees we might want to create. 41 | abstract class SyntaxTree extends Tree { 42 | /// Return an [Interval] indicating the index in the [TokenSource] of the 43 | /// first and last token associated with this subtree. If this node is a 44 | /// leaf, then the interval represents a single token. 45 | /// 46 | /// If source interval is unknown, this returns [Interval.INVALID]. 47 | Interval get sourceInterval; 48 | } 49 | 50 | /// An abstract class to access the tree of [RuleContext] objects created 51 | /// during a parse that makes the data structure look like a simple parse 52 | /// tree. 53 | /// 54 | /// This node represents both internal nodes, rule invocations and leaf 55 | /// nodes token matches. 56 | /// 57 | /// The payload is either a [Token] or a [RuleContext] object. 58 | abstract class ParseTree extends SyntaxTree { 59 | 60 | /// The [ParseTreeVisitor] needs a double dispatch method. 61 | dynamic accept(ParseTreeVisitor visitor); 62 | 63 | /// Return the combined text of all leaf nodes. Does not get any 64 | /// off-channel tokens (if any) so won't return whitespace and 65 | /// comments if they are sent to parser on hidden channel. 66 | String get text; 67 | } 68 | 69 | abstract class RuleNode extends ParseTree { 70 | RuleContext get ruleContext; 71 | } 72 | 73 | class TerminalNode implements ParseTree { 74 | 75 | Token symbol; 76 | ParseTree parent; 77 | 78 | TerminalNode(this.symbol); 79 | 80 | Token get payload => symbol; 81 | 82 | int get childCount => 0; 83 | 84 | String get text => symbol.text; 85 | 86 | Interval get sourceInterval { 87 | if (symbol == null) return Interval.INVALID; 88 | int tokenIndex = symbol.tokenIndex; 89 | return new Interval(tokenIndex, tokenIndex); 90 | } 91 | 92 | ParseTree getChild(int i) => null; 93 | 94 | dynamic accept(ParseTreeVisitor visitor) => visitor.visitTerminal(this); 95 | 96 | String toString() => (symbol.type == Token.EOF) ? "" : symbol.text; 97 | } 98 | 99 | /// Represents a token that was consumed during resynchronization rather than 100 | /// during a valid match operation. For example, we will create this kind of a 101 | /// node during single token insertion and deletion as well as during "consume 102 | /// until error recovery set" upon no viable alternative exceptions. 103 | class ErrorNode extends TerminalNode { 104 | ErrorNode(Token token) : super(token); 105 | dynamic accept(ParseTreeVisitor visitor) => visitor.visitErrorNode(this); 106 | } 107 | 108 | -------------------------------------------------------------------------------- /lib/src/tree/parse_tree_visitor.dart: -------------------------------------------------------------------------------- 1 | part of antlr4dart; 2 | 3 | /// This abstract class defines the basic notion of a parse tree visitor. 4 | /// 5 | /// Generated visitors extend this class and the implements the `XVisitor` 6 | /// interface for grammar `X`. 7 | /// 8 | /// [T] is the return type of the visit operation. Use `void` for operations 9 | /// with no return type. 10 | abstract class ParseTreeVisitor { 11 | 12 | /// Gets the default value returned by visitor methods. 13 | /// 14 | /// This value is returned by the default implementations of [visitTerminal], 15 | /// [visitErrorNode]. 16 | /// 17 | /// The default implementation of [visitChildren] initializes its aggregate 18 | /// result to this value. 19 | /// 20 | /// The base implementation returns `null`. 21 | /// 22 | /// Return the default value returned by visitor methods. 23 | T get defaultResult => null; 24 | 25 | /// The default implementation calls [ParseTree.accept] on the 26 | /// specified tree. 27 | T visit(ParseTree tree) => tree.accept(this); 28 | 29 | /// The default implementation initializes the aggregate result to 30 | /// [defaultResult]. 31 | /// 32 | /// Before visiting each child, it calls [shouldVisitNextChild]; if the 33 | /// result is `false` no more children are visited and the current aggregate 34 | /// result is returned. 35 | /// 36 | /// After visiting a child, the aggregate result is updated by calling 37 | /// [aggregateResult] with the previous aggregate result and the result of 38 | /// visiting the child. 39 | T visitChildren(RuleNode ruleNode) { 40 | T result = defaultResult; 41 | int n = ruleNode.childCount; 42 | for (int i = 0; i < n; i++) { 43 | if (!shouldVisitNextChild(ruleNode, result)) break; 44 | ParseTree c = ruleNode.getChild(i); 45 | result = aggregateResult(result, c.accept(this)); 46 | } 47 | return result; 48 | } 49 | 50 | /// The default implementation returns [defaultResult]. 51 | T visitTerminal(TerminalNode terminalNode) => defaultResult; 52 | 53 | /// The default implementation returns [defaultResult]. 54 | T visitErrorNode(ErrorNode errorNode) => defaultResult; 55 | 56 | /// Aggregates the results of visiting multiple children of a node. 57 | /// 58 | /// After either all children are visited or [shouldVisitNextChild] returns 59 | /// `false`, the aggregate value is returned as the result of [visitChildren]. 60 | /// 61 | /// The default implementation returns [nextResult], meaning [visitChildren] 62 | /// will return the result of the last child visited (or return the initial 63 | /// value if the node has no children). 64 | /// 65 | /// [aggregate] is the previous aggregate value. In the default 66 | /// implementation, the aggregate value is initialized to [defaultResult], 67 | /// which is passed as the [aggregate] argument to this method after the 68 | /// first child node is visited. 69 | /// [nextResult] is the result of the immediately preceeding call to visit 70 | /// a child node. 71 | /// 72 | /// Return the updated aggregate result. 73 | T aggregateResult(T aggregate, T nextResult) => nextResult; 74 | 75 | /// This method is called after visiting each child in [visitChildren]. 76 | /// 77 | /// This method is first called before the first child is visited; at that 78 | /// point [currentResult] will be the initial value (in the default 79 | /// implementation, the initial value is [defaultResult]. 80 | /// 81 | /// This method is not called after the last child is visited. 82 | /// 83 | /// The default implementation always returns `true`, indicating that 84 | /// [visitChildren] should only return after all children are visited. 85 | /// 86 | /// One reason to override this method is to provide a "short circuit" 87 | /// evaluation option for situations where the result of visiting a single 88 | /// child has the potential to determine the result of the visit operation as 89 | /// a whole. 90 | /// 91 | /// [ruleNode] is the [RuleNode] whose children are currently being visited. 92 | /// [currentResult] is the current aggregate result of the children visited 93 | /// to the current point. 94 | /// 95 | /// Return `true` to continue visiting children. Otherwise return `false` to 96 | /// stop visiting children and immediately return the current aggregate 97 | /// result from [visitChildren]. 98 | bool shouldVisitNextChild(RuleNode ruleNode, T currentResult) => true; 99 | } 100 | -------------------------------------------------------------------------------- /lib/src/tree/parse_tree_walker.dart: -------------------------------------------------------------------------------- 1 | part of antlr4dart; 2 | 3 | class ParseTreeWalker { 4 | 5 | static final ParseTreeWalker DEFAULT = new ParseTreeWalker(); 6 | 7 | void walk(ParseTreeListener listener, ParseTree tree) { 8 | if (tree is ErrorNode) { 9 | listener.visitErrorNode(tree); 10 | return; 11 | } else if (tree is TerminalNode) { 12 | listener.visitTerminal(tree); 13 | return; 14 | } 15 | enterRule(listener, tree); 16 | int n = tree.childCount; 17 | for (int i = 0; i < n; i++) { 18 | walk(listener, tree.getChild(i)); 19 | } 20 | exitRule(listener, tree); 21 | } 22 | 23 | void enterRule(ParseTreeListener listener, RuleNode ruleNode) { 24 | ParserRuleContext context = (ruleNode as ParserRuleContext).ruleContext; 25 | listener.enterEveryRule(context); 26 | context.enterRule(listener); 27 | } 28 | 29 | void exitRule(ParseTreeListener listener, RuleNode ruleNode) { 30 | ParserRuleContext context = (ruleNode as ParserRuleContext).ruleContext; 31 | context.exitRule(listener); 32 | listener.exitEveryRule(context); 33 | } 34 | } 35 | -------------------------------------------------------------------------------- /lib/src/tree/trees.dart: -------------------------------------------------------------------------------- 1 | part of antlr4dart; 2 | 3 | /// A set of utility routines useful for all kinds of antlr4dart trees. 4 | class Trees { 5 | 6 | /// Print out a whole tree in LISP form. [getNodeText] is used on the 7 | /// node payloads to get the text for the nodes. Detect parse trees and 8 | /// extract data appropriately. 9 | static String toStringTree(Tree tree, [dynamic rules]) { 10 | if (rules is Parser) rules = rules.ruleNames; 11 | String s = getNodeText(tree, rules).replaceAll("\t", "\\t"); 12 | s = s.replaceAll("\n", "\\n"); 13 | s = s.replaceAll("\r", "\\r"); 14 | if (tree.childCount == 0) return s; 15 | StringBuffer sb = new StringBuffer("(") 16 | ..write(s) 17 | ..write(' '); 18 | for (int i = 0; i < tree.childCount; i++) { 19 | if (i > 0) sb.write(' '); 20 | sb.write(toStringTree(tree.getChild(i), rules)); 21 | } 22 | sb.write(")"); 23 | return sb.toString(); 24 | } 25 | 26 | static String getNodeText(Tree tree, dynamic rules) { 27 | if (rules is Parser) rules = rules.ruleNames; 28 | if (rules != null) { 29 | if (tree is RuleNode) { 30 | return rules[tree.ruleContext.ruleIndex]; 31 | } else if (tree is ErrorNode) { 32 | return tree.toString(); 33 | } else if (tree is TerminalNode) { 34 | Token symbol = tree.symbol; 35 | if (symbol != null) return symbol.text; 36 | } 37 | } 38 | // no recog for rule names 39 | Object payload = tree.payload; 40 | if (payload is Token) return payload.text; 41 | return payload.toString(); 42 | } 43 | 44 | /// Return a list of all ancestors of this node. The first node of 45 | /// list is the root and the last is the parent of this node. 46 | static List getAncestors(Tree tree) { 47 | if (tree.parent == null) return []; 48 | List ancestors = new List(); 49 | tree = tree.parent; 50 | while (tree != null) { 51 | ancestors.insert(0, tree); // insert at start 52 | tree = tree.parent; 53 | } 54 | return ancestors; 55 | } 56 | 57 | Trees._internal() {} 58 | } 59 | -------------------------------------------------------------------------------- /lib/src/util/bit_set.dart: -------------------------------------------------------------------------------- 1 | part of antlr4dart; 2 | 3 | /// This class implements a vector of bits that grows as needed. Each 4 | /// component of the bit set has a `bool` value. The 5 | /// bits of a `BitSet` are indexed by nonnegative ints. 6 | /// Individual indexed bits can be examined, set, or cleared. One 7 | /// `BitSet` may be used to modify the contents of another 8 | /// `BitSet` through logical AND, logical inclusive OR, and 9 | /// logical exclusive OR operations. 10 | /// 11 | /// By default, all bits in the set initially have the value `false`. 12 | /// 13 | /// Every bit set has a current size, which is the number of bits 14 | /// of space currently in use by the bit set. Note that the size is 15 | /// related to the implementation of a bit set, so it may change with 16 | /// implementation. The length of a bit set relates to logical length 17 | /// of a bit set and is defined independently of implementation. 18 | /// 19 | /// Unless otherwise noted, passing a null parameter to any of the 20 | /// methods in a `BitSet` will result in a [NullThrownError]. 21 | /// 22 | class BitSet { 23 | 24 | // The bits in this BitSet. 25 | BigInteger _word; 26 | 27 | /// Creates an empty bit set. 28 | BitSet() { 29 | _word = BigInteger.ZERO; 30 | } 31 | 32 | /// Returns a hash code value for this bit set. The hash code 33 | /// depends only on which bits have been set within this 34 | /// `BitSet`. The algorithm used to compute it may be described 35 | /// as follows. 36 | /// 37 | /// int get hashCode { 38 | /// var h = new BigInteger(1234); 39 | /// h ^= _word; 40 | /// return ((h >> 32) ^ h).intValue(); 41 | /// } 42 | /// 43 | /// Note that the hash code values change if the set of bits is altered. 44 | /// 45 | /// Return a hash code value for this bit set. 46 | int get hashCode { 47 | var h = new BigInteger(1234); 48 | h ^= _word; 49 | return ((h >> 32) ^ h).intValue(); 50 | } 51 | 52 | /// Returns the "logical size" of this [BitSet]: the index of the highest 53 | /// set bit in the [BitSet] plus one. Returns zero if the [BitSet] contains 54 | /// no set bits. 55 | /// 56 | /// Return the logical size of this [BitSet]. 57 | int get length => _word.bitLength(); 58 | 59 | /// Returns true if this [BitSet] contains no bits that are set to `true`. 60 | bool get isEmpty => _word == BigInteger.ZERO; 61 | 62 | /// Returns the number of bits set to `true` in this [BitSet]. 63 | int get cardinality => _word.bitCount(); 64 | 65 | /// Sets the bit at the specified index to the specified [value]. 66 | /// 67 | /// [bitIndex] is a bit index. 68 | /// 69 | /// A [RangeError] occurs when the specified index is negative. 70 | void set(int bitIndex, [bool value = false]) { 71 | if (value) { 72 | if (bitIndex < 0) throw new RangeError("bitIndex < 0: $bitIndex"); 73 | _word = _word.setBit(bitIndex); 74 | } else { 75 | clear(bitIndex); 76 | } 77 | } 78 | 79 | /// Sets the bit specified by the index to `false`. 80 | /// 81 | /// [bitIndex] is the index of the bit to be cleared. 82 | /// 83 | /// A [RangeError] occurs when the specified index is negative. 84 | void clear(int bitIndex) { 85 | if (bitIndex < 0) throw new RangeError("bitIndex < 0: $bitIndex"); 86 | _word = _word.clearBit(bitIndex); 87 | } 88 | 89 | /// Returns the value of the bit with the specified index. The value is 90 | /// `true` if the bit with the index [bitIndex] is currently set in this 91 | /// [BitSet]; otherwise, the result is `false`. 92 | /// 93 | /// [bitIndex] is the bit index 94 | /// 95 | /// A [RangeError] occurs when the specified index is negative. 96 | bool get(int bitIndex) { 97 | if (bitIndex < 0) throw new RangeError("bitIndex < 0: $bitIndex"); 98 | return _word.testBit(bitIndex); 99 | } 100 | 101 | /// Returns the index of the first bit that is set to `true` that occurs on 102 | /// or after the specified starting index. If no such bit exists then 103 | /// `code -1` is returned. 104 | /// 105 | /// To iterate over the `true` bits in a [BitSet], 106 | /// use the following loop: 107 | /// 108 | /// for (int i = bs.nextSetBit(0); i >= 0; i = bs.nextSetBit(i+1)) { 109 | /// // operate on index i here 110 | /// } 111 | /// 112 | /// [fromIndex] is the index to start checking from (inclusive). 113 | /// 114 | /// A [RangeError] occurs when the specified index is negative. 115 | int nextSetBit(int fromIndex) { 116 | if (fromIndex < 0) throw new RangeError("fromIndex < 0: $fromIndex"); 117 | var size = _word.bitLength(); 118 | for (int i = fromIndex; i < size;i++) { 119 | if (_word.testBit(i)) return i; 120 | } 121 | return -1; 122 | } 123 | 124 | /// Returns the index of the first bit that is set to `false` that occurs on 125 | /// or after the specified starting index. 126 | /// 127 | /// [fromIndex] is the index to start checking from (inclusive) 128 | /// 129 | /// A [RangeError] occurs when the specified index is negative. 130 | int nextClearBit(int fromIndex) { 131 | if (fromIndex < 0) throw new RangeError("fromIndex < 0: $fromIndex"); 132 | var size = _word.bitLength(); 133 | for (int i = fromIndex; i < size;i++) { 134 | if (!_word.testBit(i)) return i; 135 | } 136 | return -1; 137 | } 138 | 139 | /// Performs a logical **OR** of this bit set with the bit set argument. 140 | /// 141 | /// This [BitSet] is modified so that a bit in it has the value `true` if 142 | /// and only if it either already had the value `true` or the corresponding 143 | /// bit in the bit set argument has the value `true`. 144 | void or(BitSet bitSet) { 145 | if (this != bitSet) _word |= bitSet._word; 146 | } 147 | 148 | /// Compares this object against the specified object. 149 | /// 150 | /// The result is `true` if and only if the argument is not `null` and is 151 | /// a [Bitset] object that has exactly the same set of bits set to `true` 152 | /// as this bit set. That is, for every nonnegative `int` index `k`, 153 | /// `(other as BitSet).get(k) == this.get(k)` must be true. The current 154 | /// sizes of the two bit sets are not compared. 155 | /// 156 | /// [other] is the the object to compare with. 157 | /// 158 | /// Return `true` if the objects are the same;`false` otherwise. 159 | bool operator==(Object other) { 160 | return other is BitSet ? _word == other._word : false; 161 | } 162 | 163 | /// Returns a string representation of this bit set. 164 | /// 165 | /// For every index for which this [BitSet] contains a bit in the set state, 166 | /// the decimal representation of that index is included in the result. 167 | /// Such indices are listed in order from lowest to highest, separated 168 | /// by ", " (a comma and a space) and surrounded by braces, 169 | /// resulting in the usual mathematical notation for a set of integers. 170 | /// 171 | /// Example: 172 | /// 173 | /// BitSet drPepper = new BitSet(); 174 | /// 175 | /// Now `drPepper.toString()` returns `"{}"`. 176 | /// 177 | /// drPepper.set(2, true); 178 | /// 179 | /// Now `drPepper.toString()` returns `"{2}"`. 180 | /// 181 | /// drPepper.set(4, true); 182 | /// drPepper.set(10, true); 183 | /// 184 | /// Now `drPepper.toString()` returns `"{2, 4, 10}"`. 185 | /// 186 | /// Return a string representation of this bit set. 187 | String toString() { 188 | StringBuffer sb = new StringBuffer('{'); 189 | int i = nextSetBit(0); 190 | if (i != -1) { 191 | sb.write(i); 192 | for (i = nextSetBit(i+1); i >= 0; i = nextSetBit(i+1)) { 193 | int endOfRun = nextClearBit(i); 194 | do { 195 | sb 196 | ..write(", ") 197 | ..write(i); 198 | } while (++i < endOfRun); 199 | } 200 | } 201 | sb.write('}'); 202 | return sb.toString(); 203 | } 204 | } 205 | -------------------------------------------------------------------------------- /lib/src/util/double_key_map.dart: -------------------------------------------------------------------------------- 1 | part of antlr4dart; 2 | 3 | /// Sometimes we need to map a key to a value but key is two pieces of data. 4 | class DoubleKeyMap { 5 | 6 | Map> data; 7 | 8 | DoubleKeyMap() { 9 | data = new LinkedHashMap>(); 10 | } 11 | 12 | Iterable get values { 13 | Set set = new HashSet(); 14 | for (Map key in data.values) 15 | for (Value value in key.values) set.add(value); 16 | return set; 17 | } 18 | 19 | Set get keys => data.keys; 20 | 21 | Value put(Key1 key1, Key2 key2, Value value) { 22 | Map data = this.data[key1]; 23 | Value prev = null; 24 | if (data == null) { 25 | data = new LinkedHashMap(); 26 | this.data[key1] = data; 27 | } else { 28 | prev = data[key2]; 29 | } 30 | data[key2] = value; 31 | return prev; 32 | } 33 | 34 | Value get(Key1 key1, Key2 key2) { 35 | Map data = this.data[key1]; 36 | if (data == null) return null; 37 | return data[key2]; 38 | } 39 | 40 | Map operator [](Key1 key) => data[key]; 41 | 42 | /// Get all values associated with primary [key]. 43 | Iterable valuesForKey(Key1 key) { 44 | Map data = this.data[key]; 45 | if (data == null) return null; 46 | return data.values; 47 | } 48 | 49 | Set keySetForKey(Key1 key) { 50 | Map data = this.data[key]; 51 | if (data == null) return null; 52 | return data.keys; 53 | } 54 | } 55 | 56 | -------------------------------------------------------------------------------- /lib/src/util/interval.dart: -------------------------------------------------------------------------------- 1 | part of antlr4dart; 2 | 3 | /// An immutable inclusive interval `a..b`. 4 | class Interval { 5 | 6 | static const int INTERVAL_POOL_MAX_VALUE = 1000; 7 | 8 | static final Interval INVALID = new Interval(-1,-2); 9 | 10 | static List cache = new List(INTERVAL_POOL_MAX_VALUE + 1); 11 | 12 | static int creates = 0; 13 | static int misses = 0; 14 | static int hits = 0; 15 | static int outOfRange = 0; 16 | 17 | int _a; 18 | int _b; 19 | 20 | Interval(this._a, this._b); 21 | 22 | /// Return number of elements between a and b inclusively. `x..x` is length 1. 23 | /// If `b < a`, then length is 0. For example 9..10 has length 2. 24 | int get length => (_b < _a) ? 0 : _b - _a + 1; 25 | 26 | /// [Interval] objects are used readonly so share all with the same single 27 | /// value a == b up to some max size. Use a list as a perfect hash. 28 | /// 29 | /// [a] and [b] could be both [int]s or single character [String]s. 30 | /// 31 | /// Return a shared object for `0..INTERVAL_POOL_MAX_VALUE` or a new [Interval] 32 | /// object with `a..a` in it. 33 | static Interval of(dynamic a, dynamic b) { 34 | if (a is String) a = a.codeUnitAt(0); 35 | if (b is String) b = b.codeUnitAt(0); 36 | // cache just a..a 37 | if (a != b || a < 0 || a > INTERVAL_POOL_MAX_VALUE) { 38 | return new Interval(a, b); 39 | } 40 | if (cache[a] == null) cache[a] = new Interval(a, a); 41 | return cache[a]; 42 | } 43 | 44 | bool operator==(Interval o) => _a == o._a && _b == o._b; 45 | 46 | /// Does this start completely before other? Disjoint. 47 | bool startsBeforeDisjoint(Interval other) => _a < other._a && _b < other._a; 48 | 49 | /// Does this start at or before other? Nondisjoint. 50 | bool startsBeforeNonDisjoint(Interval other) 51 | => _a <= other._a && _b >= other._a; 52 | 53 | /// Does this.a start after other.b? May or may not be disjoint. 54 | bool startsAfter(Interval other) => _a > other._a; 55 | 56 | /// Does this start completely after other? Disjoint. 57 | bool startsAfterDisjoint(Interval other) => _a > other._b; 58 | 59 | /// Does this start after other? NonDisjoint. 60 | bool startsAfterNonDisjoint(Interval other) 61 | => _a > other._a && _a <= other._b; 62 | 63 | /// Are both ranges disjoint? I.e., no overlap? 64 | bool disjoint(Interval other) 65 | => startsBeforeDisjoint(other) || startsAfterDisjoint(other); 66 | 67 | /// Are two intervals adjacent such as 0..41 and 42..42? 68 | bool adjacent(Interval other) => _a == other._b + 1 || _b == other._a - 1; 69 | 70 | bool properlyContains(Interval other) => other._a >= _a && other._b <= _b; 71 | 72 | /// Return the interval computed from combining this and [other]. 73 | Interval union(Interval other) 74 | => Interval.of(min(_a, other._a), max(_b, other._b)); 75 | 76 | /// Return the interval in common between this and [other]. 77 | Interval intersection(Interval other) 78 | => Interval.of(max(_a, other._a), min(_b, other._b)); 79 | 80 | /// Return the interval with elements from this not in [other]. 81 | /// 82 | /// [other] must not be totally enclosed (properly contained) within this, 83 | /// which would result in two disjoint intervals instead of the single one 84 | /// returned by this method. 85 | Interval differenceNotProperlyContained(Interval other) { 86 | Interval diff = null; 87 | if (other.startsBeforeNonDisjoint(this)) { 88 | // other.a to left of this.a (or same) 89 | diff = Interval.of(max(_a, other._b + 1), _b); 90 | } else if (other.startsAfterNonDisjoint(this)) { 91 | // other.a to right of this.a 92 | diff = Interval.of(_a, other._a - 1); 93 | } 94 | return diff; 95 | } 96 | 97 | String toString() => "$_a..$_b"; 98 | } 99 | -------------------------------------------------------------------------------- /lib/src/util/interval_set.dart: -------------------------------------------------------------------------------- 1 | part of antlr4dart; 2 | 3 | /// A set of [int]s that relies on ranges being common to do 4 | /// `run-length-encoded` like compression (if you view an [IntervalSet] like 5 | /// a [BitSet] with runs of 0s and 1s). Only ranges are recorded so that 6 | /// a few ints up near value 1000 don't cause massive bitsets, just two 7 | /// int intervals. 8 | /// 9 | /// Element values may be negative. Useful for sets of `EPSILON` and `EOF`. 10 | /// 11 | /// `0..9` char range is index pair `['\u0030','\u0039']`. 12 | /// Multiple ranges are encoded with multiple index pairs. Isolated elements 13 | /// are encoded with an index pair where both intervals are the same. 14 | /// 15 | /// The ranges are ordered and disjoint so that `2..6` appears 16 | /// before `101..103`. 17 | class IntervalSet { 18 | 19 | static final COMPLETE_CHAR_SET = IntervalSet.of(0, Lexer.MAX_CHAR_VALUE); 20 | static final EMPTY_SET = new IntervalSet(); 21 | 22 | // The list of sorted, disjoint intervals. 23 | List _intervals; 24 | 25 | bool isReadonly = false; 26 | 27 | IntervalSet([dynamic els]) { 28 | if (els == null) { 29 | _intervals = new List(); 30 | } else if (els is List) { 31 | _intervals = new List(); 32 | els.forEach((e) => addSingle(e)); 33 | } else { 34 | _intervals = els; 35 | } 36 | } 37 | 38 | IntervalSet.from(IntervalSet set) { 39 | _intervals = new List(); 40 | addAll(set); 41 | } 42 | 43 | /// Create a set with a single element [a]. 44 | /// 45 | /// [a] could be an [int] or a single character [String]. 46 | static IntervalSet ofSingle(dynamic a) => new IntervalSet([a]); 47 | 48 | /// Create a set with all ints within range `[a..b]` (inclusive). 49 | /// 50 | /// [a] and [b] could be [int]s or sigle character [String]s. 51 | static IntervalSet of(dynamic a, dynamic b) { 52 | if (a is String) a = a.codeUnitAt(0); 53 | if (b is String) b = b.codeUnitAt(0); 54 | IntervalSet intervalSet = new IntervalSet(); 55 | intervalSet.add(a,b); 56 | return intervalSet; 57 | } 58 | 59 | /// Combine all sets in the list [sets]. 60 | static IntervalSet combine(List sets) { 61 | IntervalSet intervalSet = new IntervalSet(); 62 | sets.forEach((s) => intervalSet.addAll(s)); 63 | return intervalSet; 64 | } 65 | 66 | int get length { 67 | int n = 0; 68 | int numIntervals = _intervals.length; 69 | if (numIntervals == 1) { 70 | Interval firstInterval = _intervals.first; 71 | return firstInterval._b - firstInterval._a + 1; 72 | } 73 | for (int i = 0; i < numIntervals; i++) { 74 | Interval _i = _intervals[i]; 75 | n += (_i._b - _i._a + 1); 76 | } 77 | return n; 78 | } 79 | 80 | /// Return true if this set has no members. 81 | bool get isNil => _intervals.isEmpty; 82 | 83 | /// If this set is a single [int]? Return it, otherwise [Token.INVALID_TYPE]. 84 | int get singleElement { 85 | if (_intervals.length == 1) { 86 | Interval i = _intervals.first; 87 | if (i._a == i._b ) return i._a; 88 | } 89 | return Token.INVALID_TYPE; 90 | } 91 | 92 | int get maxElement => (isNil) ? Token.INVALID_TYPE : _intervals.last._b; 93 | 94 | /// Return minimum element >= 0. 95 | int get minElement { 96 | if (isNil) return Token.INVALID_TYPE; 97 | int n = _intervals.length; 98 | for (Interval i in _intervals) { 99 | int a = i._a; 100 | int b = i._b; 101 | for (int v = a; v <= b; v++) { 102 | if (v >= 0) return v; 103 | } 104 | } 105 | return Token.INVALID_TYPE; 106 | } 107 | 108 | /// Return a list of Interval objects. 109 | List get intervals => _intervals; 110 | 111 | int get hashCode { 112 | int hash = MurmurHash.initialize(); 113 | for (Interval i in _intervals) { 114 | hash = MurmurHash.update(hash, i._a); 115 | hash = MurmurHash.update(hash, i._b); 116 | } 117 | return MurmurHash.finish(hash, _intervals.length * 2); 118 | } 119 | 120 | /// Are two [IntervalSet]s equal? 121 | bool operator==(Object other) { 122 | if (other is IntervalSet) { 123 | if (_intervals.length == other._intervals.length) { 124 | for (int i = 0; i < _intervals.length; i++) 125 | if (_intervals[i] != other._intervals[i]) return false; 126 | return true; 127 | } 128 | } 129 | return false; 130 | } 131 | 132 | void clear() { 133 | if (isReadonly) throw new StateError("can't alter readonly IntervalSet"); 134 | _intervals.clear(); 135 | } 136 | 137 | /// Add a single element to the set. An isolated element is stored 138 | /// as a range `a..a`. 139 | /// 140 | /// [a] could be an [int] or a single character [String]. 141 | void addSingle(dynamic a) { 142 | if (isReadonly) throw new StateError("can't alter readonly IntervalSet"); 143 | add(a, a); 144 | } 145 | 146 | /// Add interval; i.e., add all integers from [a] to ][b] to set. 147 | /// 148 | /// If [b] < [a], do nothing. 149 | /// 150 | /// Keep list in sorted order (by left range value). 151 | /// 152 | /// If overlap, combine ranges. For example, if this is `{1..5, 10..20}`, 153 | /// adding `6..7` yields `{1..5, 6..7, 10..20}`. Adding `4..8` yields 154 | /// `{1..8, 10..20}`. 155 | /// 156 | /// [a] and [b] could be [int]s or single character [String]s. 157 | void add(dynamic a, dynamic b) { 158 | _add(Interval.of(a, b)); 159 | } 160 | 161 | IntervalSet addAll(IntervalSet set) { 162 | if (set == null) return this; 163 | if (set is! IntervalSet) { 164 | throw new ArgumentError( 165 | "can't add non IntSet (${set.runtimeType}) to IntervalSet"); 166 | } 167 | set._intervals.forEach((i) => add(i._a, i._b)); 168 | return this; 169 | } 170 | 171 | /// Given the set of possible values, return a new set containing all 172 | /// elements in [vocabulary], but not in `this`. 173 | /// 174 | /// The computation is ([vocabulary] - `this`). 175 | /// 176 | /// 'this' is assumed to be either a subset or equal to [vocabulary]. 177 | IntervalSet complement(IntervalSet vocabulary) { 178 | if (vocabulary == null) return null; // nothing in common with null set 179 | if (vocabulary is! IntervalSet) { 180 | throw new ArgumentError( 181 | "can't complement with non IntervalSet (${vocabulary.runtimeType})"); 182 | } 183 | IntervalSet vocabularyCopy = new IntervalSet.from(vocabulary); 184 | int maxElement = vocabularyCopy.maxElement; 185 | IntervalSet compl = new IntervalSet(); 186 | int n = _intervals.length; 187 | if (n == 0) return compl; 188 | Interval first = _intervals.first; 189 | // add a range from 0 to first.a constrained to vocab 190 | if (first._a > 0) { 191 | IntervalSet s = IntervalSet.of(0, first._a - 1); 192 | IntervalSet a = s.and(vocabularyCopy); 193 | compl.addAll(a); 194 | } 195 | for (int i = 1; i < n; i++) { // from 2nd interval .. nth 196 | Interval previous = _intervals[i - 1]; 197 | Interval current = _intervals[i]; 198 | IntervalSet s = IntervalSet.of(previous._b + 1, current._a - 1); 199 | IntervalSet a = s.and(vocabularyCopy); 200 | compl.addAll(a); 201 | } 202 | Interval last = intervals[n - 1]; 203 | // add a range from last.b to maxElement constrained to vocab 204 | if (last._b < maxElement) { 205 | IntervalSet s = IntervalSet.of(last._b + 1, maxElement); 206 | IntervalSet a = s.and(vocabularyCopy); 207 | compl.addAll(a); 208 | } 209 | return compl; 210 | } 211 | 212 | /// Compute `this` - [other] via `this` & ~[other]. 213 | /// 214 | /// Return a new set containing all elements in this but not in other. 215 | /// 216 | /// [other] is assumed to be a subset of `this`; anything that is in [other] 217 | /// but not in this will be ignored. 218 | IntervalSet subtract(IntervalSet other) { 219 | // assume the whole unicode range here for the complement 220 | // because it doesn't matter. Anything beyond the max of this' set 221 | // will be ignored since we are doing this & ~other. The intersection 222 | // will be empty. The only problem would be when this' set max value 223 | // goes beyond Lexer.MAX_CHAR_VALUE, but hopefully the constant 224 | // Lexer.MAX_CHAR_VALUE will prevent this. 225 | return and(other.complement(COMPLETE_CHAR_SET)); 226 | } 227 | 228 | IntervalSet or(IntervalSet a) { 229 | return new IntervalSet() 230 | ..addAll(this) 231 | ..addAll(a); 232 | } 233 | 234 | /// Return a new set with the intersection of `this` set with [other]. 235 | /// 236 | /// Because the intervals are sorted, we can use an iterator for each list 237 | /// and just walk them together. This is roughly `O(min(n,m))` for interval 238 | /// set lengths `n` and `m`. 239 | IntervalSet and(IntervalSet other) { 240 | if (other == null) return null; // nothing in common with null set 241 | List theirIntervals = other._intervals; 242 | IntervalSet intersection = null; 243 | int mySize = _intervals.length; 244 | int theirSize = theirIntervals.length; 245 | int i = 0; 246 | int j = 0; 247 | // iterate down both interval lists looking for nondisjoint intervals 248 | while (i < mySize && j < theirSize) { 249 | Interval mine = _intervals[i]; 250 | Interval theirs = theirIntervals[j]; 251 | if (mine.startsBeforeDisjoint(theirs)) { 252 | // move this iterator looking for interval that might overlap 253 | i++; 254 | } else if (theirs.startsBeforeDisjoint(mine)) { 255 | // move other iterator looking for interval that might overlap 256 | j++; 257 | } else if (mine.properlyContains(theirs)) { 258 | // overlap, add intersection, get next theirs 259 | if (intersection == null) intersection = new IntervalSet(); 260 | intersection._add(mine.intersection(theirs)); 261 | j++; 262 | } else if (theirs.properlyContains(mine)) { 263 | // overlap, add intersection, get next mine 264 | if (intersection == null) intersection = new IntervalSet(); 265 | intersection._add(mine.intersection(theirs)); 266 | i++; 267 | } else if (!mine.disjoint(theirs)) { 268 | // overlap, add intersection 269 | if (intersection == null) intersection = new IntervalSet(); 270 | intersection._add(mine.intersection(theirs)); 271 | // Move the iterator of lower range [a..b], but not 272 | // the upper range as it may contain elements that will collide 273 | // with the next iterator. So, if mine=[0..115] and 274 | // theirs=[115..200], then intersection is 115 and move mine 275 | // but not theirs as theirs may collide with the next range 276 | // in thisIter. 277 | // move both iterators to next ranges 278 | if (mine.startsAfterNonDisjoint(theirs)) { 279 | j++; 280 | } else if (theirs.startsAfterNonDisjoint(mine)) { 281 | i++; 282 | } 283 | } 284 | } 285 | if (intersection == null) return new IntervalSet(); 286 | return intersection; 287 | } 288 | 289 | /// Is [a] in any range of this set? 290 | /// 291 | /// [a] could be a [int] or a single character [String]. 292 | bool contains(dynamic a) { 293 | if (a is String) a = a.codeUnitAt(0); 294 | for (Interval i in _intervals) { 295 | // list is sorted and el is before this interval; not here 296 | if (a < i._a) break; 297 | if (a >= i._a && a <= i._b) return true; // found in this interval 298 | } 299 | return false; 300 | } 301 | 302 | String toString([bool elemAreChar = false]) { 303 | StringBuffer sb = new StringBuffer(); 304 | if (_intervals == null || _intervals.isEmpty) return "{}"; 305 | if (length > 1) sb.write("{"); 306 | Iterator iter = _intervals.iterator; 307 | bool first = true; 308 | while (iter.moveNext()) { 309 | if (!first) sb.write(", "); 310 | else first = false; 311 | Interval i = iter.current; 312 | int a = i._a; 313 | int b = i._b; 314 | if (a == b) { 315 | if (a == -1) { 316 | sb.write(""); 317 | } else if (elemAreChar) { 318 | sb.write("'$a'"); 319 | } else { 320 | sb.write(a); 321 | } 322 | } else { 323 | if (elemAreChar) { 324 | sb.write("'$a'..'$b'"); 325 | } else { 326 | sb.write("$a..$b"); 327 | } 328 | } 329 | } 330 | if (length > 1) sb.write("}"); 331 | return sb.toString(); 332 | } 333 | 334 | String toTokenString(List tokenNames) { 335 | StringBuffer sb = new StringBuffer(); 336 | if (_intervals == null || _intervals.isEmpty) return "{}"; 337 | if (length > 1) sb.write("{"); 338 | Iterator iter = _intervals.iterator; 339 | bool first = true; 340 | while (iter.moveNext()) { 341 | if (!first) { 342 | sb.write(", "); 343 | } else { 344 | first = false; 345 | } 346 | Interval I = iter.current; 347 | int a = I._a; 348 | int b = I._b; 349 | if (a == b) { 350 | sb.write(_elementName(tokenNames, a)); 351 | } else { 352 | for (int i=a; i<=b; i++) { 353 | if (i > a) sb.write(", "); 354 | sb.write(_elementName(tokenNames, i)); 355 | } 356 | } 357 | } 358 | if (length > 1) { 359 | sb.write("}"); 360 | } 361 | return sb.toString(); 362 | } 363 | 364 | List toList() { 365 | List values = new List(); 366 | int n = _intervals.length; 367 | for (Interval i in _intervals) { 368 | for (int v = i._a; v <= i._b; v++) { 369 | values.add(v); 370 | } 371 | } 372 | return values; 373 | } 374 | 375 | Set toSet() { 376 | Set set = new HashSet(); 377 | for (Interval i in _intervals) { 378 | for (int v = i._a; v <= i._b; v++) { 379 | set.add(v); 380 | } 381 | } 382 | return set; 383 | } 384 | 385 | void remove(dynamic a) { 386 | if (isReadonly) throw new StateError("can't alter readonly IntervalSet"); 387 | if (a is String) a = a.codeUnitAt(0); 388 | int n = _intervals.length; 389 | for (int i = 0; i < n; i++) { 390 | Interval interval = _intervals[i]; 391 | // list is sorted and el is before this interval; not here 392 | if (a < interval._a) break; 393 | // if whole interval x..x, rm 394 | if (a == interval._a && a == interval._b) { 395 | _intervals.removeAt(i); 396 | break; 397 | } 398 | // if on left edge x..b, adjust left 399 | if (a == interval._a) { 400 | interval._a++; 401 | break; 402 | } 403 | // if on right edge a..x, adjust right 404 | if (a == interval._b) { 405 | interval._b--; 406 | break; 407 | } 408 | // if in middle a..x..b, split interval 409 | if (a > interval._a && a < interval._b) { 410 | // found in this interval 411 | int oldb = interval._b; 412 | interval._b = a - 1; // [a..x-1] 413 | add(a + 1, oldb); // add [x+1..b] 414 | } 415 | } 416 | } 417 | 418 | // copy on write so we can cache a..a intervals and sets of that 419 | void _add(Interval addition) { 420 | if (isReadonly) throw new StateError("can't alter readonly IntervalSet"); 421 | if (addition._b < addition._a) return; 422 | // find position in list 423 | // Use iterators as we modify list in place 424 | for (int i = 0; i < _intervals.length; i++) { 425 | Interval r = _intervals[i]; 426 | if (addition == r) return; 427 | if (addition.adjacent(r) || !addition.disjoint(r)) { 428 | // next to each other, make a single larger interval 429 | Interval bigger = addition.union(r); 430 | _intervals[i] = bigger; 431 | // make sure we didn't just create an interval that 432 | // should be merged with next interval in list 433 | while (i < _intervals.length - 1) { 434 | Interval next = _intervals[++i]; 435 | if (!bigger.adjacent(next) && bigger.disjoint(next)) break; 436 | _intervals.remove(next); // remove this one 437 | _intervals[--i] = bigger.union(next); // set to 3 merged ones 438 | } 439 | return; 440 | } 441 | if (addition.startsBeforeDisjoint(r)) { 442 | // insert before r 443 | _intervals.insert(i, addition); 444 | return; 445 | } 446 | // if disjoint and after r, a future iteration will handle it 447 | } 448 | // ok, must be after last interval (and disjoint from last interval) 449 | // just add it 450 | _intervals.add(addition); 451 | } 452 | 453 | String _elementName(List tokenNames, int a) { 454 | if (a == Token.EOF) { 455 | return ""; 456 | } else if (a == Token.EPSILON) { 457 | return ""; 458 | } 459 | return tokenNames[a]; 460 | } 461 | } 462 | -------------------------------------------------------------------------------- /lib/src/util/murmur_hash.dart: -------------------------------------------------------------------------------- 1 | part of antlr4dart; 2 | 3 | class MurmurHash { 4 | 5 | static const int DEFAULT_SEED = 0; 6 | 7 | const MurmurHash._internal(); 8 | 9 | /// Initialize the hash using the specified [seed]. 10 | /// 11 | /// Return the intermediate hash value. 12 | static int initialize([int seed = DEFAULT_SEED]) => seed; 13 | 14 | /// Update the intermediate hash value for the next input [value]. 15 | /// 16 | /// [hash] is the intermediate hash value. 17 | /// [value] the value to add to the current hash. 18 | /// 19 | /// Return the updated intermediate hash value. 20 | static int update(int hash, [int value]) { 21 | value = value != null ? value.hashCode : 0; 22 | final int c1 = 0xCC9E2D51; 23 | final int c2 = 0x1B873593; 24 | final int r1 = 15; 25 | final int r2 = 13; 26 | final int m = 5; 27 | final int n = 0xE6546B64; 28 | int k = value; 29 | k = k * c1; 30 | k = (k << r1) | ((k & 0xFFFFFFFF) >> (32 - r1)); 31 | k = k * c2; 32 | hash = hash ^ k; 33 | hash = (hash << r2) | ((hash & 0xFFFFFFFF) >> (32 - r2)); 34 | hash = hash * m + n; 35 | return hash & 0xFFFFFFFFFFFFFF; 36 | } 37 | 38 | /// Apply the final computation steps to the intermediate value [hash] 39 | /// to form the final result of the MurmurHash 3 hash function. 40 | /// 41 | /// [hash] is the intermediate hash value. 42 | /// [numberOfWords] is the number of integer values added to the hash. 43 | /// 44 | /// Return the final hash result. 45 | static int finish(int hash, int numberOfWords) { 46 | hash = hash ^ (numberOfWords * 4); 47 | hash = hash ^ ((hash & 0xFFFFFFFF) >> 16); 48 | hash = hash * 0x85EBCA6B; 49 | hash = hash ^ ((hash & 0xFFFFFFFF) >> 13); 50 | hash = hash * 0xC2B2AE35; 51 | hash = hash ^ ((hash & 0xFFFFFFFF) >> 16); 52 | return hash & 0xFFFFFFFF; 53 | } 54 | 55 | /// Utility function to compute the hash code of an array using the 56 | /// MurmurHash algorithm. 57 | /// 58 | /// [data] is the iterable data. 59 | /// [seed] is the seed for the MurmurHash algorithm. 60 | /// 61 | /// Return the hash code of the data. 62 | static int calcHashCode(Iterable data, int seed) { 63 | int hash = initialize(seed); 64 | for (var value in data) { 65 | hash = update(hash, value.hashCode); 66 | } 67 | return finish(hash, data.length) & 0xFFFFFFFFF; 68 | } 69 | } 70 | -------------------------------------------------------------------------------- /lib/src/util/pair.dart: -------------------------------------------------------------------------------- 1 | part of antlr4dart; 2 | 3 | class Pair { 4 | final A a; 5 | final B b; 6 | const Pair(this.a, this.b); 7 | } 8 | -------------------------------------------------------------------------------- /pubspec.yaml: -------------------------------------------------------------------------------- 1 | name: antlr4dart 2 | version: 0.7.0 3 | author: Tiago Mazzutti 4 | description: ANTLR4 runtime library for Dart. 5 | homepage: https://github.com/tiagomazzutti/antlr4dart-runtime.git 6 | dependencies: 7 | bignum: any 8 | collection: any --------------------------------------------------------------------------------