├── README.md
├── LICENSE
├── LL1 Syntax Analyser.cpp
└── Lexical Analyser.cpp


/README.md:
--------------------------------------------------------------------------------
  1 | # Compiler-Design
  2 | 
  3 | Compiler has broadly 6 phases : Lexical Analyser, Syntax Analyser, Semantic Analyser, Intermediate Code Generator, Code Optimizer, Code Generator.
  4 | 
  5 | 
  6 | ![Compiler Flow Chart](https://user-images.githubusercontent.com/75173703/115982029-b50b1c80-a5b5-11eb-876d-10fc330a0fba.PNG)
  7 | 
  8 | 
  9 | # Lexical Analyser
 10 | 
 11 | Lexical Analyzer : Responsible for generating a stream of tokens <name,value> issued for each lexeme ( chunk of text ). It is also responsible for constructing the symbol Table.
 12 | 
 13 | Tokens ( NFA ) : <name,value> ;
 14 | 
 15 | eg. <RELOP,EQ>, <num,80>, <if>
 16 |   
 17 | For our purpose, we have considered 8 Lexical Categories. Actual Number of lexical categories, may vary and can be quite large.
 18 | 
 19 | ![Lexical Categories](https://user-images.githubusercontent.com/75173703/115982032-b63c4980-a5b5-11eb-8696-f38bf054082c.PNG)
 20 | 
 21 | List Order of Lexical Categories with the types of token's issued by them :
 22 | 
 23 | ![Order and Type of Tokens](https://user-images.githubusercontent.com/75173703/115982139-572b0480-a5b6-11eb-9f26-22096649f835.PNG)
 24 | 
 25 | Here I have implemented NFA's using Switch-case statements. However, the NFA for Keywords is implemented using Trie data-structure.
 26 | 
 27 | Also I have implemenented a Symbol Table using Class and a Hash-Map. Symbol table stores the variables and issues token number for variables depending on the row number.
 28 | 
 29 | Also, for generating tokens and navigating through the source text file, we have two techniques : Serial and Parallel.
 30 | 
 31 | If for a lexemme, no token is issues a Lexical Error is declared !
 32 | 
 33 | ## NFA's Example
 34 | 
 35 | - Relational Operators category : RELOP
 36 | 
 37 | ![RELOP NFA](https://user-images.githubusercontent.com/75173703/115982180-af620680-a5b6-11eb-92df-af9dd976eb20.PNG)
 38 | 
 39 | - Variable
 40 | 
 41 | ![VAR NFA](https://user-images.githubusercontent.com/75173703/115982198-d1f41f80-a5b6-11eb-8488-e2fa4a285a05.PNG)
 42 | 
 43 | - Keywords
 44 | 
 45 | ![KEYWORD NFA](https://user-images.githubusercontent.com/75173703/115982263-55157580-a5b7-11eb-9c4b-d0fc934e26a5.PNG)
 46 | 
 47 | - Numbers
 48 | 
 49 | ![NUMBERS NFA](https://user-images.githubusercontent.com/75173703/115982305-a58cd300-a5b7-11eb-8c09-7122bf4826ad.PNG)
 50 | 
 51 | - Whitespaces
 52 | 
 53 | ![WS NFA](https://user-images.githubusercontent.com/75173703/115982315-c3f2ce80-a5b7-11eb-890a-72430bb8989a.PNG)
 54 | 
 55 | 
 56 | ## Psuedo code for NFA's
 57 | 
 58 | ![NFA Psedo Code](https://user-images.githubusercontent.com/75173703/115982242-2d261200-a5b7-11eb-84b5-d6351f92a708.PNG)
 59 | 
 60 | ## Serial Navigation
 61 | 
 62 | NFA which says yes first wins. ( NFA's are applied in decreasing order ).
 63 | 
 64 | ![NFA Order](https://user-images.githubusercontent.com/75173703/115982389-3368be00-a5b8-11eb-92bb-578724b0483e.PNG)
 65 | 
 66 | 
 67 | ## Parallel Navigation
 68 | 
 69 | Combine All NFA's in one common diagram.
 70 | 
 71 | In case of multiple Yes :
 72 | - Go for the longest lexeme in case of Multiple Yes
 73 | - Go for Order, if the length is also same
 74 | 
 75 | 
 76 | 
 77 | # Syntax Analyser
 78 | 
 79 | Parses the string to check if the rule of grammar were followed, and if followed - creates a parse tree for the input string.
 80 | 
 81 | Parsing : Constructing a tree for an input string using the CFG ( Context Free Grammar ) Rules. There are two types of parsing, Top-down parsing and bottom-up parsing.
 82 | 
 83 | 
 84 | ## First
 85 | 
 86 | To Compute FIRST(X) for all grammar symbols X, apply the following rules until no more terminals or **ε** symbols can be added to any FIRST set.
 87 | 
 88 | ![First](https://user-images.githubusercontent.com/75173703/115982585-09180000-a5ba-11eb-8d9f-2abc6785fa7e.PNG)
 89 | 
 90 | ## Follow
 91 | 
 92 | To Compute FOLLOW(A) for all nonterminals A, apply the following rules until nothing can be added to any FOLLOW set.
 93 | 
 94 | ![Follow](https://user-images.githubusercontent.com/75173703/115982615-3d8bbc00-a5ba-11eb-83e3-fa0a47ef3630.PNG)
 95 | 
 96 | 
 97 | ## LL(1) Parser or Predictive Parser.
 98 | 
 99 | LL(1) is top-down parser, short for _**Left to Right Scanning**_, _**Left most derivation**_ with _**one Look ahead**_.
100 | 
101 | It starts from the root or the Start Variable S and constructs the tree. Before applyinh make sure that there is no Left Recursion in the Grammar.
102 | 
103 | It Constructs a parsing table, and the performs a sequence of moves on the input string. Eg.
104 | 
105 | - **Grammar**
106 | 
107 | ![Grammar](https://user-images.githubusercontent.com/75173703/115982721-d4587880-a5ba-11eb-93c8-7f8c6bce1acb.PNG)
108 | 
109 | - **Grammar After Removing Left-Recursion**
110 | 
111 | Left recursion is such that _A --> Aα_
112 | 
113 | Replace the left recursive productions by non-left recusrsive Productions.
114 | 
115 | ![Left Recursion Removal Rule](https://user-images.githubusercontent.com/75173703/115982745-08339e00-a5bb-11eb-85b1-e4e8b2d0714a.PNG)
116 | 
117 | For our Grammar :
118 | 
119 | ![Grammar without LR](https://user-images.githubusercontent.com/75173703/115982767-20a3b880-a5bb-11eb-9eeb-17143c837fe4.PNG)
120 | 
121 | - **Constructing Parsing Table from the Given Grammar**
122 | 
123 | Then, we construct a Predictive Parsing Table :
124 | 
125 | ![Parsing Table](https://user-images.githubusercontent.com/75173703/115982791-4cbf3980-a5bb-11eb-8373-c379e7c28682.PNG)
126 | 
127 | 
128 | - **Sequence of Moves** for input _id + id * id_
129 | 
130 | Moves made by the predictive parser on our input :
131 | 
132 | ![Parse](https://user-images.githubusercontent.com/75173703/115982816-7a0be780-a5bb-11eb-98f4-f07724720838.PNG)
133 | 
134 | Outcome : _**( $,$ )**_ : Yes or Accept ( No Syntax Error ).
135 | 
136 | 
137 | # References
138 | 
139 | [Compilers Priciples, Techniques & Tools - Second Edition : Alfred V. Ahno, Monica S. Lam, Ravi Sethi, Jeffrey D. Ullman](https://en.wikipedia.org/wiki/Compilers:_Principles,_Techniques,_and_Tools#:~:text=Compilers%3A%20Principles%2C%20Techniques%2C%20and%20Tools%20is%20a%20computer%20science,compiler%20construction%20for%20programming%20languages.)
140 | 


--------------------------------------------------------------------------------
/LICENSE:
--------------------------------------------------------------------------------
  1 |                                  Apache License
  2 |                            Version 2.0, January 2004
  3 |                         http://www.apache.org/licenses/
  4 | 
  5 |    TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
  6 | 
  7 |    1. Definitions.
  8 | 
  9 |       "License" shall mean the terms and conditions for use, reproduction,
 10 |       and distribution as defined by Sections 1 through 9 of this document.
 11 | 
 12 |       "Licensor" shall mean the copyright owner or entity authorized by
 13 |       the copyright owner that is granting the License.
 14 | 
 15 |       "Legal Entity" shall mean the union of the acting entity and all
 16 |       other entities that control, are controlled by, or are under common
 17 |       control with that entity. For the purposes of this definition,
 18 |       "control" means (i) the power, direct or indirect, to cause the
 19 |       direction or management of such entity, whether by contract or
 20 |       otherwise, or (ii) ownership of fifty percent (50%) or more of the
 21 |       outstanding shares, or (iii) beneficial ownership of such entity.
 22 | 
 23 |       "You" (or "Your") shall mean an individual or Legal Entity
 24 |       exercising permissions granted by this License.
 25 | 
 26 |       "Source" form shall mean the preferred form for making modifications,
 27 |       including but not limited to software source code, documentation
 28 |       source, and configuration files.
 29 | 
 30 |       "Object" form shall mean any form resulting from mechanical
 31 |       transformation or translation of a Source form, including but
 32 |       not limited to compiled object code, generated documentation,
 33 |       and conversions to other media types.
 34 | 
 35 |       "Work" shall mean the work of authorship, whether in Source or
 36 |       Object form, made available under the License, as indicated by a
 37 |       copyright notice that is included in or attached to the work
 38 |       (an example is provided in the Appendix below).
 39 | 
 40 |       "Derivative Works" shall mean any work, whether in Source or Object
 41 |       form, that is based on (or derived from) the Work and for which the
 42 |       editorial revisions, annotations, elaborations, or other modifications
 43 |       represent, as a whole, an original work of authorship. For the purposes
 44 |       of this License, Derivative Works shall not include works that remain
 45 |       separable from, or merely link (or bind by name) to the interfaces of,
 46 |       the Work and Derivative Works thereof.
 47 | 
 48 |       "Contribution" shall mean any work of authorship, including
 49 |       the original version of the Work and any modifications or additions
 50 |       to that Work or Derivative Works thereof, that is intentionally
 51 |       submitted to Licensor for inclusion in the Work by the copyright owner
 52 |       or by an individual or Legal Entity authorized to submit on behalf of
 53 |       the copyright owner. For the purposes of this definition, "submitted"
 54 |       means any form of electronic, verbal, or written communication sent
 55 |       to the Licensor or its representatives, including but not limited to
 56 |       communication on electronic mailing lists, source code control systems,
 57 |       and issue tracking systems that are managed by, or on behalf of, the
 58 |       Licensor for the purpose of discussing and improving the Work, but
 59 |       excluding communication that is conspicuously marked or otherwise
 60 |       designated in writing by the copyright owner as "Not a Contribution."
 61 | 
 62 |       "Contributor" shall mean Licensor and any individual or Legal Entity
 63 |       on behalf of whom a Contribution has been received by Licensor and
 64 |       subsequently incorporated within the Work.
 65 | 
 66 |    2. Grant of Copyright License. Subject to the terms and conditions of
 67 |       this License, each Contributor hereby grants to You a perpetual,
 68 |       worldwide, non-exclusive, no-charge, royalty-free, irrevocable
 69 |       copyright license to reproduce, prepare Derivative Works of,
 70 |       publicly display, publicly perform, sublicense, and distribute the
 71 |       Work and such Derivative Works in Source or Object form.
 72 | 
 73 |    3. Grant of Patent License. Subject to the terms and conditions of
 74 |       this License, each Contributor hereby grants to You a perpetual,
 75 |       worldwide, non-exclusive, no-charge, royalty-free, irrevocable
 76 |       (except as stated in this section) patent license to make, have made,
 77 |       use, offer to sell, sell, import, and otherwise transfer the Work,
 78 |       where such license applies only to those patent claims licensable
 79 |       by such Contributor that are necessarily infringed by their
 80 |       Contribution(s) alone or by combination of their Contribution(s)
 81 |       with the Work to which such Contribution(s) was submitted. If You
 82 |       institute patent litigation against any entity (including a
 83 |       cross-claim or counterclaim in a lawsuit) alleging that the Work
 84 |       or a Contribution incorporated within the Work constitutes direct
 85 |       or contributory patent infringement, then any patent licenses
 86 |       granted to You under this License for that Work shall terminate
 87 |       as of the date such litigation is filed.
 88 | 
 89 |    4. Redistribution. You may reproduce and distribute copies of the
 90 |       Work or Derivative Works thereof in any medium, with or without
 91 |       modifications, and in Source or Object form, provided that You
 92 |       meet the following conditions:
 93 | 
 94 |       (a) You must give any other recipients of the Work or
 95 |           Derivative Works a copy of this License; and
 96 | 
 97 |       (b) You must cause any modified files to carry prominent notices
 98 |           stating that You changed the files; and
 99 | 
100 |       (c) You must retain, in the Source form of any Derivative Works
101 |           that You distribute, all copyright, patent, trademark, and
102 |           attribution notices from the Source form of the Work,
103 |           excluding those notices that do not pertain to any part of
104 |           the Derivative Works; and
105 | 
106 |       (d) If the Work includes a "NOTICE" text file as part of its
107 |           distribution, then any Derivative Works that You distribute must
108 |           include a readable copy of the attribution notices contained
109 |           within such NOTICE file, excluding those notices that do not
110 |           pertain to any part of the Derivative Works, in at least one
111 |           of the following places: within a NOTICE text file distributed
112 |           as part of the Derivative Works; within the Source form or
113 |           documentation, if provided along with the Derivative Works; or,
114 |           within a display generated by the Derivative Works, if and
115 |           wherever such third-party notices normally appear. The contents
116 |           of the NOTICE file are for informational purposes only and
117 |           do not modify the License. You may add Your own attribution
118 |           notices within Derivative Works that You distribute, alongside
119 |           or as an addendum to the NOTICE text from the Work, provided
120 |           that such additional attribution notices cannot be construed
121 |           as modifying the License.
122 | 
123 |       You may add Your own copyright statement to Your modifications and
124 |       may provide additional or different license terms and conditions
125 |       for use, reproduction, or distribution of Your modifications, or
126 |       for any such Derivative Works as a whole, provided Your use,
127 |       reproduction, and distribution of the Work otherwise complies with
128 |       the conditions stated in this License.
129 | 
130 |    5. Submission of Contributions. Unless You explicitly state otherwise,
131 |       any Contribution intentionally submitted for inclusion in the Work
132 |       by You to the Licensor shall be under the terms and conditions of
133 |       this License, without any additional terms or conditions.
134 |       Notwithstanding the above, nothing herein shall supersede or modify
135 |       the terms of any separate license agreement you may have executed
136 |       with Licensor regarding such Contributions.
137 | 
138 |    6. Trademarks. This License does not grant permission to use the trade
139 |       names, trademarks, service marks, or product names of the Licensor,
140 |       except as required for reasonable and customary use in describing the
141 |       origin of the Work and reproducing the content of the NOTICE file.
142 | 
143 |    7. Disclaimer of Warranty. Unless required by applicable law or
144 |       agreed to in writing, Licensor provides the Work (and each
145 |       Contributor provides its Contributions) on an "AS IS" BASIS,
146 |       WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
147 |       implied, including, without limitation, any warranties or conditions
148 |       of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
149 |       PARTICULAR PURPOSE. You are solely responsible for determining the
150 |       appropriateness of using or redistributing the Work and assume any
151 |       risks associated with Your exercise of permissions under this License.
152 | 
153 |    8. Limitation of Liability. In no event and under no legal theory,
154 |       whether in tort (including negligence), contract, or otherwise,
155 |       unless required by applicable law (such as deliberate and grossly
156 |       negligent acts) or agreed to in writing, shall any Contributor be
157 |       liable to You for damages, including any direct, indirect, special,
158 |       incidental, or consequential damages of any character arising as a
159 |       result of this License or out of the use or inability to use the
160 |       Work (including but not limited to damages for loss of goodwill,
161 |       work stoppage, computer failure or malfunction, or any and all
162 |       other commercial damages or losses), even if such Contributor
163 |       has been advised of the possibility of such damages.
164 | 
165 |    9. Accepting Warranty or Additional Liability. While redistributing
166 |       the Work or Derivative Works thereof, You may choose to offer,
167 |       and charge a fee for, acceptance of support, warranty, indemnity,
168 |       or other liability obligations and/or rights consistent with this
169 |       License. However, in accepting such obligations, You may act only
170 |       on Your own behalf and on Your sole responsibility, not on behalf
171 |       of any other Contributor, and only if You agree to indemnify,
172 |       defend, and hold each Contributor harmless for any liability
173 |       incurred by, or claims asserted against, such Contributor by reason
174 |       of your accepting any such warranty or additional liability.
175 | 
176 |    END OF TERMS AND CONDITIONS
177 | 
178 |    APPENDIX: How to apply the Apache License to your work.
179 | 
180 |       To apply the Apache License to your work, attach the following
181 |       boilerplate notice, with the fields enclosed by brackets "[]"
182 |       replaced with your own identifying information. (Don't include
183 |       the brackets!)  The text should be enclosed in the appropriate
184 |       comment syntax for the file format. We also recommend that a
185 |       file or class name and description of purpose be included on the
186 |       same "printed page" as the copyright notice for easier
187 |       identification within third-party archives.
188 | 
189 |    Copyright [yyyy] [name of copyright owner]
190 | 
191 |    Licensed under the Apache License, Version 2.0 (the "License");
192 |    you may not use this file except in compliance with the License.
193 |    You may obtain a copy of the License at
194 | 
195 |        http://www.apache.org/licenses/LICENSE-2.0
196 | 
197 |    Unless required by applicable law or agreed to in writing, software
198 |    distributed under the License is distributed on an "AS IS" BASIS,
199 |    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
200 |    See the License for the specific language governing permissions and
201 |    limitations under the License.
202 | 


--------------------------------------------------------------------------------
/LL1 Syntax Analyser.cpp:
--------------------------------------------------------------------------------
  1 | #include<bits/stdc++.h>
  2 | using namespace std;
  3 | 
  4 | class Cfg_rule // a CFG Grammer rule object
  5 | {
  6 |     public :
  7 | 
  8 |         char root; // Eg. S
  9 |         vector<char> leaf; // Eg. A, B, C
 10 | 
 11 |         // makes S -> A B C ( a grammar rule )
 12 | 
 13 |         Cfg_rule(char root = 'e')
 14 |         {
 15 |             this->root = root; // initalizes the root as Epsilon
 16 |         }
 17 | 
 18 |         int is_present(char l) // to search an element in the leaf of a particular CFG Rule
 19 |         {
 20 |             for(int i=0;i<leaf.size();i++)
 21 |             {
 22 |                 if(l==leaf[i])
 23 |                 return i; // returns the index if its present in the Cfg_rule
 24 |             }
 25 | 
 26 |             return -1; // if it does not exist
 27 |         }
 28 | 
 29 |         void display()
 30 |         {
 31 |             cout<<root<<" --> ";
 32 | 
 33 |             for(int i=0;i<leaf.size();i++)
 34 |             {
 35 |                 cout<<leaf[i];
 36 |             }
 37 |         }
 38 | 
 39 | };
 40 | 
 41 | 
 42 | set<char> First_single( char f,map< char , list<Cfg_rule> > &rules, set<char> &terminals, set<char> &non_terminals) // returns a character vector in the first set
 43 | {
 44 |     set<char> first;
 45 | 
 46 |     if(terminals.count(f))
 47 |     first.insert(f);
 48 | 
 49 |     else
 50 |     {
 51 |         queue<char> BFS;
 52 |         BFS.push(f);
 53 | 
 54 |         while(!BFS.empty())
 55 |         {
 56 | 
 57 |             char d = BFS.front();
 58 |             BFS.pop();
 59 |             
 60 |             if( (d == 'e' || terminals.count(d)!=0) && first.count(d)==0 ) // if the character is an epsilon, or a terminal and not present already in the set
 61 |             first.insert(d);
 62 | 
 63 |             else
 64 |             {
 65 |             for(auto node_1 : rules[d]) // iterating the list on Cfg_rule object
 66 |             {
 67 |                 // node_1 is a Cfg_rule type object
 68 | 
 69 |                 BFS.push(node_1.leaf[0]);
 70 |     
 71 |             }
 72 |             }
 73 | 
 74 |         }
 75 | 
 76 |     }
 77 | 
 78 |     return first;
 79 | 
 80 | }
 81 | 
 82 | 
 83 | map<char, set<char>> First_all( map< char , list<Cfg_rule> > &rules, set<char> &terminals, set<char> &non_terminals)
 84 | {
 85 | 
 86 |     map<char, set<char>> first;
 87 | 
 88 |     // for terminals , first(terminal) = terminal
 89 | 
 90 |     for(auto node : terminals)
 91 |         // cout<<"FIRST ( "<<node<<" ) : { " <<node<<" }"<<endl; ## Make use only if printing is necessary
 92 |         first[node].insert(node);
 93 | 
 94 |     
 95 |     // for non-terminals we need to find the first
 96 | 
 97 |     for( auto node : non_terminals)
 98 |     {   
 99 |         /*
100 | 
101 |         cout << "FIRST ( " << node << " ) : { ";
102 | 
103 |         set<char> first; // to store the first characters
104 | 
105 |         */
106 | 
107 |         queue<char> BFS; // applying a BFS-Search on the graph
108 | 
109 | 
110 |         BFS.push(node);
111 | 
112 |         while(!BFS.empty())
113 |         {
114 | 
115 |             char d = BFS.front();
116 |             BFS.pop();
117 |             
118 |             if( (d == 'e' || terminals.count(d)!=0) && first[node].count(d)==0 ) // if the character is an epsilon, or a terminal and not present already in the set
119 |             first[node].insert(d);
120 | 
121 |             else
122 |             {
123 |                 for(auto node_1 : rules[d]) // iterating the list on Cfg_rule object
124 |                 {
125 |                     // node_1 is a Cfg_rule type object
126 | 
127 |                     BFS.push(node_1.leaf[0]);
128 |         
129 |                 }
130 |             }
131 | 
132 |         }
133 |         
134 |         /*
135 | 
136 |         for(auto f : first)
137 |         cout<<f<<", ";
138 | 
139 |         cout<<" }"<<endl;
140 | 
141 |         */
142 |     }
143 | 
144 |     return first;
145 | }
146 | 
147 | 
148 | void Follow_single(char f, map< char, set<char> > &Follow, map< char , list<Cfg_rule> > &rules, set<char> &terminals, set<char> &non_terminals )
149 | {
150 | 
151 |     for( auto node : rules ) // node is a map iterator with 'key : non-terminals' and 'value : list of Cfg_rule objects'
152 |     {
153 |         for( auto node1 : node.second ) // iterating the list of Cfg_rule objects
154 |         {
155 |             if(node1.is_present(f)!=-1) // if the non-terminal is present
156 |             {
157 |                 int pos = node1.is_present(f); // acquired the position in the cfg_rule
158 | 
159 |                 if( (pos+1) >= node1.leaf.size() )
160 |                 {
161 |                     // of the form A --> αB
162 | 
163 |                     if(node1.root != f)
164 |                     {
165 |                         if(Follow.count(node1.root)==0)
166 |                             Follow_single(node1.root, Follow, rules, terminals, non_terminals);
167 | 
168 |                         for(auto node2 : Follow[node1.root])
169 |                             Follow[f].insert(node2);
170 |                     }
171 | 
172 |                 }
173 |                 else 
174 |                 {
175 |                    // of the form A --> αBβ ; Now calculating first of β
176 | 
177 |                     bool only_epsilon = true;
178 | 
179 |                     for(auto node2 : First_single(node1.leaf[pos+1],rules,terminals,non_terminals))
180 |                        if(node2 != 'e')
181 | 
182 |                     only_epsilon = false;
183 | 
184 |                     
185 |                     if(only_epsilon == true)
186 |                     {
187 | 
188 |                         // if only_epsilon == True --> means, the first of (β) contains only ε, i.e the grammar is of the form A --> αBβ where first of 'β' is 'e' or 'ε' or Epsilon
189 | 
190 |                         if(node1.root != f)
191 |                         {
192 |                             if(Follow.count(node1.root)==0)
193 |                                 Follow_single(node1.root, Follow, rules, terminals, non_terminals);
194 | 
195 |                             for(auto node2 : Follow[node1.root])
196 |                                 Follow[f].insert(node2);
197 | 
198 |                         }
199 | 
200 |                     }
201 | 
202 |                     else
203 |                     {
204 |                         // if only_epsilon == False --> means, the first of (β) contains more than only ε, i.e the grammar is of the form of the form A --> αBβ
205 | 
206 |                         for(auto node2 : First_single(node1.leaf[pos+1],rules,terminals,non_terminals))
207 |                         {
208 |                             if(node2 != 'e')
209 |                             Follow[f].insert(node2);
210 |                             
211 |                         }
212 |                     }
213 | 
214 |                 }
215 | 
216 |             }
217 |         }
218 |     }
219 | 
220 | 
221 | }
222 | 
223 | 
224 | map<char, set<char>> Follow_all(char start_node,map< char , list<Cfg_rule> > &rules, set<char> &terminals, set<char> &non_terminals )
225 | {
226 |     map< char, set<char> > Follow; // the Follow map
227 | 
228 |     Follow[start_node].insert('$'); // Inserting '$' for the start node
229 | 
230 |     Follow_single(start_node,Follow,rules,terminals,non_terminals);
231 | 
232 |     for(auto node : non_terminals)
233 |     {
234 |         if(Follow.count(node)==0)
235 |             Follow_single(node,Follow,rules,terminals,non_terminals);
236 |     }
237 | 
238 |     /*
239 | 
240 |     ## Make the return of the fuction void and remove all the lines below this comment to see previous version of Applicaton
241 | 
242 |     for(auto node : Follow) // Printing all the Follow's
243 |     {
244 |         cout<<"FOLLOW ( " << node.first <<" ) : { ";
245 | 
246 |         for(auto node1 : node.second)
247 |         {
248 |             cout<<node1<<", ";
249 |         }
250 | 
251 |         cout<<"}"<<endl;
252 |     }
253 | 
254 |     */
255 | 
256 |    return Follow;
257 | }
258 | 
259 | 
260 | map<char, map<char,Cfg_rule>> LL1_Parsing_Table_Construct(map< char , list<Cfg_rule> > &rules, map<char, set<char>> &First, map<char, set<char>> &Follow, set<char> &non_terminals, set<char> &terminals)
261 | {
262 |     // Columns : Consists of all terminals and '$'
263 | 
264 |     // Rows : Consists of all non-terminals
265 | 
266 |     // No element will be present for error parser
267 | 
268 |     map<char, map<char, Cfg_rule>> LL1_Parser_Table;
269 | 
270 |     for( auto rule : rules )
271 |     {
272 |         for( auto rule1 : rule.second) // traversing all the rules
273 |         {
274 |             // Consider Rule/Production of the form A --> α
275 | 
276 |             char alpha = rule1.leaf[0];
277 | 
278 |             if(alpha != 'e') // alpha is not equal to 'Epsilon' or 'e'
279 |             {
280 | 
281 |                 for(auto node : First[alpha])
282 |                 {
283 |                     if(node != 'e')
284 |                     {
285 |                         // STEP 1 : Add A --> α to M[A,a] where a belongs to FIRST(α)
286 | 
287 |                         LL1_Parser_Table[rule1.root][node] = rule1;
288 |                     }
289 |                     else
290 |                     {
291 |                         for(auto node1 : Follow[rule1.root] )
292 |                         {
293 |                             // STEP 2 : Add A --> α to M[A,b] where b belongs to FOLLOW(A)
294 | 
295 |                             LL1_Parser_Table[rule1.root][node1] = rule1;
296 |                         }
297 |                     }
298 |                 }
299 | 
300 |             }
301 | 
302 |             else
303 |             {
304 |                 for(auto node1 : Follow[rule1.root] )
305 |                 {
306 |                     // STEP 2 : Add A --> α to M[A,b] where b belongs to FOLLOW(A)
307 | 
308 |                     LL1_Parser_Table[rule1.root][node1] = rule1;
309 |                 }
310 |             }
311 |         }
312 | 
313 |     }
314 | 
315 |     return LL1_Parser_Table;
316 | }
317 | 
318 | 
319 | void Display_Parsing_Table_Construct(map<char, map<char,Cfg_rule>> &LL1_Parsing_Table)
320 | {
321 |     for(auto node : LL1_Parsing_Table)
322 |     {
323 |         for( auto node1 : node.second)
324 |         {
325 |             cout<<"M [ "<<node.first<<" , "<<node1.first<<" ] : ";
326 |             node1.second.display();
327 |             cout<<"\t\t\t,\t\t\t";
328 |         }
329 | 
330 |         cout<<endl;
331 |     }
332 | }
333 | 
334 | 
335 | bool LL1_Parse(string test, char start_node, map< char, map< char, Cfg_rule > > LL1_Parser)
336 | {
337 |     stack<char> S;
338 |     S.push('$');S.push(start_node);
339 | 
340 |     int pos = 0; // pointer for test string
341 | 
342 |     test[test.size()] = '$';
343 | 
344 |     while(S.top()!='$')
345 |     {
346 |         if(S.top()==test[pos]) // if a Match is found
347 |         {
348 |             cout<<"Match for "<< test[pos];
349 |             pos++;
350 |             S.pop();
351 |         }
352 | 
353 |         else
354 |         {
355 |             if(LL1_Parser.count(S.top()))
356 |             {
357 |                 if(LL1_Parser[S.top()].count(test[pos]))
358 |                 {
359 |                     // Check M [ S.top() , test[pos] ] and and enter the value into the stack
360 |                     cout<<"Check cell M [ " << S.top() <<" , "<<test[pos]<<" ] and replace ";
361 |                     Cfg_rule rule = LL1_Parser[S.top()][test[pos]];
362 |                     S.pop();
363 | 
364 |                     
365 |                     rule.display();
366 | 
367 | 
368 |                     for(int i = rule.leaf.size()-1;i>=0;--i)
369 |                     {
370 |                         if(rule.leaf[i]!='e')
371 |                             S.push(rule.leaf[i]);                     
372 |                     }
373 |                 }
374 |                 else
375 |                 {
376 |                     cout<<"No cell found for M [ "<<S.top()<<" , "<<test[pos]<<" ] and Hence returning Erorr !!";
377 |                     break; // No match found
378 |                 }
379 |             }
380 |             else
381 |             {
382 |                 cout<<"No cell found for M [ "<<S.top()<<" , "<<test[pos]<<" ] and Hence returning Erorr !!";
383 |                 break; // No match found
384 |             }
385 |         }
386 | 
387 |         cout<<endl;
388 |     }
389 | 
390 |     // if the Stack has '$' and the test string has ended with '\0' the string is accepted
391 |     if(S.top()=='$' && test[pos]=='$')
392 |         return true;
393 |     
394 |     return false;
395 |         
396 | }
397 | 
398 | 
399 | int main()
400 | {
401 |     cout<<"Enter number of test cases : ";
402 |     int t; // test cases
403 |     cin>>t;
404 |     
405 |     while(t--) // starts from t-1 to 0 ( t times )
406 |     {
407 | 
408 |         cout << endl << "Enter the number of grammar rules : ";
409 |         int nr ; // nr : number of grammer rules
410 | 
411 |         cin>>nr;
412 | 
413 |         map< char , list<Cfg_rule> > rules; // a map to store all rules ; key : a non-terminal character ; Value : a list of Cfg_rule objects
414 | 
415 |         set<char> terminals ; // a vector to store terminals
416 | 
417 |         set<char> non_terminals ; // a vector to store non-terminals
418 | 
419 |         cout<< endl << "\"Note : '$' is reserved; 'e' is reserved as Epsilon; 'i' is reserved for 'id'; 'n' is reserved for 'num'; 'x' is reserved for '*'\"\n\nStart Entering the Grammar Rules one-by-one in fromat \"S ABC\" which stands for \"S -> ABC\" :" << endl ;
420 | 
421 |         for(int i=0;i<nr;i++)
422 |         {
423 | 
424 |             Cfg_rule node ; // Creating the rule object
425 | 
426 |             cin>>node.root; // Input the root Non-Terminal Node
427 | 
428 |             if(non_terminals.count(node.root)==0)
429 |             non_terminals.insert(node.root); // assuming 'S' stands for stard and 'ϵ' stands for epsilon
430 | 
431 |             string leafs; // Input the leaf nodes
432 |             cin>>leafs;
433 | 
434 |             for(int i=0;leafs[i]!='\0';i++)
435 |             {
436 |                 if(leafs[i]=='e')
437 |                 {
438 |                     // do nothing
439 |                 }
440 |                 
441 |                 else if(isupper(leafs[i]))
442 |                 {
443 |                     if(non_terminals.count(leafs[i])==0)
444 |                     non_terminals.insert(leafs[i]);
445 |                 }
446 | 
447 |                 else if(islower(leafs[i]))
448 |                 {
449 |                     if(terminals.count(leafs[i])==0)
450 |                     terminals.insert(leafs[i]);
451 |                 }
452 | 
453 |                 node.leaf.push_back(leafs[i]);
454 |             }
455 | 
456 |             // made the node object
457 | 
458 |             // now enter the object into the map
459 | 
460 |             rules[node.root].push_back(node);
461 | 
462 |         }
463 | 
464 |         cout<<endl<<"Enter the Starting Node : ";
465 |         char start_node;cin>>start_node; // start_node is the start for the grammar
466 |         cout<<endl;
467 | 
468 | 
469 |         map< char, set<char> > First = First_all( rules, terminals, non_terminals); // Calculates all the First's
470 | 
471 |         map< char, set<char> > Follow = Follow_all(start_node, rules, terminals,non_terminals ); // Calculates all the Follow's
472 | 
473 |         map< char, map< char, Cfg_rule > > LL1_Parsing_Table = LL1_Parsing_Table_Construct(rules,First,Follow,non_terminals,terminals); // A Double map to store the parsing table
474 | 
475 |         Display_Parsing_Table_Construct(LL1_Parsing_Table);
476 | 
477 |         fflush(stdin);
478 |         string to_parse;
479 |         cout<<endl<<"Enter the Test statement to pass through the LL1 Parser : ";
480 |         cin>>to_parse;
481 |         
482 |         cout<<endl;
483 |         if(LL1_Parse(to_parse, start_node, LL1_Parsing_Table))
484 |             cout<<endl<<endl<<"String Accepted : No Syntax Error"<<endl;
485 |         else
486 |             cout<<endl<<endl<<"String Un-accepted : Syntax error is Present"<<endl;
487 | 
488 |     }
489 | 
490 |     cout<<endl<<endl;
491 |     return 0;
492 | }


--------------------------------------------------------------------------------
/Lexical Analyser.cpp:
--------------------------------------------------------------------------------
   1 | #include<bits/stdc++.h>
   2 | using namespace std;
   3 | 
   4 | /*
   5 | 
   6 | Global Variables
   7 | 
   8 |     - keywordTrie : a instance of keyword_Trie
   9 |     - symbolTable : a instance of class Symbol_Table and object type data-structure
  10 | 
  11 | 
  12 | Utils data-structures
  13 | 
  14 |     - keyword_node : a class data-type for keyword_Trie data-structure
  15 |     - keyword_Trie : a Trie data-structure to keywords
  16 |     - Symbol_Table : a class type data-base to store real-time declared variables and provide them id
  17 |     - Token : a class to store string-type tokens
  18 | 
  19 | 
  20 | Global Functions
  21 | 
  22 |     - lexical_analysis() : Lexically analyses a input-stream and returns a vectors of tokens 
  23 | 
  24 | 
  25 | Utils
  26 | 
  27 |     - vector_to_string() : Functions to convert vector of character's to a string
  28 | 
  29 | 
  30 | Serial lexical Analysis utils
  31 | 
  32 |     - getKeyword() : Looks if the Lexemme is a Keyword
  33 |     - getVariable() : Looks if the Lexemme is a Variable
  34 |     - getParenthesis() : Looks if the lexemme is a Paranthesis
  35 |     - getPunctuation() : Looks if the lexemme is a Punctuation
  36 |     - getWhitespace() : Looks if the lexemme is a WhiteSpace
  37 |     - getRelop() : Looks if the lexemme is a Relop
  38 |     - getNumber() : Looks if the lexemme is a Number
  39 | 
  40 | 
  41 | */
  42 | 
  43 | 
  44 | class identifier
  45 | {
  46 |     public :
  47 |     int id; // to store the id of the variable
  48 |     string name; // to store the name of the variable
  49 | 
  50 |     identifier() // a null identifier
  51 |     {
  52 |         id=-1;
  53 |         name="\0";
  54 |     }
  55 | 
  56 |     identifier(int id, string name)
  57 |     {
  58 |         this->id = id;
  59 |         this->name = name;
  60 |     }
  61 | };
  62 | 
  63 | class Symbol_Table
  64 | {
  65 |     public :
  66 | 
  67 |     int current_size;
  68 |     map<int,identifier> identifiers; // map with 'key : id' and 'value : identifier type object'
  69 | 
  70 |     Symbol_Table()
  71 |     {
  72 |         current_size = 0; // no id's present at initialization
  73 |     }
  74 | 
  75 |     int insert(string var) // returns valid 'integer' if insertions is successful, -1 if not successful
  76 |     {
  77 |         bool allocate = true;
  78 | 
  79 |         // check if the variable is already declared of not, if not allocate position in map, if yes return -1
  80 | 
  81 |         for(auto node : identifiers)
  82 |         {
  83 | 
  84 |             if(var.compare(node.second.name)==0) // using string::str.compare()
  85 |             {allocate=false;break;}
  86 |             
  87 |         }
  88 | 
  89 |         if(allocate==false)
  90 |         return -1;
  91 | 
  92 |         int id = ++current_size;
  93 | 
  94 |         identifiers[id] = identifier(id,var);
  95 | 
  96 |         return id;
  97 |     }
  98 | 
  99 |     void del(int id)
 100 |     {
 101 |         identifiers.erase(id);
 102 |     }
 103 | 
 104 |     void pop()
 105 |     {
 106 |         identifiers.erase(current_size);
 107 |         current_size--;
 108 |     }
 109 | 
 110 | }SymbolTable;
 111 | 
 112 | string vector_to_string(vector<char> str)
 113 | {
 114 |     string to_string(str.begin(),str.end());
 115 | 
 116 |     return to_string;
 117 | }
 118 | 
 119 | class Token // class token to issue tokens - in this case RELOP tokens
 120 | {
 121 |     public :
 122 | 
 123 |     string t; // represents the token string for this simple operations
 124 | 
 125 |     Token()
 126 |     {
 127 |        t = "Not an identifiable Token";
 128 |     }
 129 | 
 130 |     Token(string token_type)
 131 |     {
 132 |         t = "< " + token_type + " >" ;
 133 |     }
 134 | 
 135 |     Token(string token_type ,string token_value) // if the type of token and the name of the operator is provided
 136 |     {
 137 |         t = "< " + token_type + " , " + token_value + " >";
 138 |     }
 139 | 
 140 |     Token(string token_type, int token_value)
 141 |     {
 142 |         t = "< " + token_type + " , " + to_string(token_value) + " >";
 143 |     }
 144 | 
 145 |     Token(string token_type ,char token_value) // if the type of token and the name of the operator is provided
 146 |     {
 147 |         t = "< " + token_type + " , " + token_value + " >";
 148 |     }
 149 | 
 150 | };
 151 | 
 152 | class keyword_node
 153 | {
 154 |     public :
 155 | 
 156 |         char a;
 157 |         map<char,keyword_node*> dict;
 158 |         bool isTerminal;
 159 | 
 160 |         keyword_node( char a ='\0' )
 161 |         {
 162 |             this->a = a;
 163 |             isTerminal = false; // default the node is not a terminat
 164 |         }
 165 | 
 166 | };
 167 | 
 168 | class keyword_Trie
 169 | {
 170 |     public :
 171 | 
 172 |         keyword_node *root;
 173 | 
 174 |         keyword_Trie()
 175 |         {
 176 |             root = new keyword_node(); // the root is pointing to a null object
 177 |         }
 178 | 
 179 |         void insert(string str) // inserts a string into the Trie data-structure
 180 |         {
 181 |             keyword_node *temp = root;
 182 |             int i = 0;
 183 | 
 184 |             while(str[i]!='\0')
 185 |             {
 186 |                 char data = str[i];
 187 | 
 188 |                 if(temp->dict.count(data) == 0)
 189 |                     temp->dict[data] = new keyword_node(data);
 190 | 
 191 |                 temp = temp->dict[data];
 192 |                 i++;
 193 |             }
 194 | 
 195 |             temp->isTerminal = true;
 196 |         }
 197 |         
 198 |         bool nfa(string file_input, vector<Token> &tokens,int &i)
 199 |         {
 200 |             int pos = i;
 201 |             bool valid = false;
 202 |             bool loop = true;
 203 |             char data;
 204 | 
 205 |             vector<char> word ;
 206 | 
 207 |             keyword_node *temp = root;
 208 | 
 209 |             while(loop)
 210 |             {
 211 | 
 212 |                 if(pos<=file_input.length())
 213 |                 data = file_input[pos];
 214 |                 else
 215 |                 {loop = false;data='\0';}
 216 | 
 217 | 
 218 |                 if(temp->isTerminal && !isalnum(data))
 219 |                 {
 220 |                     valid=true;
 221 |                     Token T ;
 222 |                     T =  Token(vector_to_string(word));
 223 |                     tokens.push_back(T);
 224 |                     loop = false;
 225 |                 }
 226 | 
 227 |                 if(temp->dict.count(data)) // exist in Trie
 228 |                 {
 229 |                     temp = temp->dict[data];
 230 |                     pos++;
 231 |                     word.push_back(data);
 232 |                 }
 233 |                 else
 234 |                 {
 235 |                     loop = false;
 236 |                 }
 237 |             }
 238 | 
 239 |             if(valid==true)
 240 |             {
 241 |                 i=pos;// update i;
 242 |             }
 243 | 
 244 |             return valid;
 245 |         }
 246 | 
 247 | }keywordTrie; // keywords is a global Trie type data-structure
 248 | 
 249 | void initialize_keywords()
 250 | {
 251 |     // initializing all the keywords
 252 | 
 253 |     int number_of_keywords = 2;
 254 | 
 255 |     string keyword[number_of_keywords] = {"if",
 256 |                         "else"
 257 |                         };
 258 | 
 259 |     for(int i=0;i<number_of_keywords;i++)
 260 |     {
 261 |         keywordTrie.insert(keyword[i]);
 262 |     }
 263 | }
 264 | 
 265 | bool getKeyword(string file_input, vector<Token> &tokens,int &i)
 266 | {
 267 |     int pos = i;
 268 | 
 269 |     bool valid = keywordTrie.nfa(file_input,tokens,i);
 270 | 
 271 |     return valid;
 272 | }
 273 | 
 274 | bool getVariable(string file_input, vector<Token> &tokens,int &i)
 275 | {
 276 | 
 277 |     int pos = i;
 278 | 
 279 |     int state = 0;
 280 | 
 281 |     bool loop = true;
 282 |     bool valid = false;
 283 | 
 284 |     vector<char> var;
 285 | 
 286 |     char c;
 287 | 
 288 |     while(loop)
 289 |     {
 290 | 
 291 |         if(pos<=file_input.length())
 292 |         c = file_input[pos];
 293 |         else
 294 |         {loop = false;c='\0';}
 295 | 
 296 |         // we will use c++ std::'bool isalpha(char c)' , 'bool isalnum(char c)'
 297 | 
 298 |         switch(state)
 299 |         {
 300 | 
 301 |             case 0 : if(isalpha(c))
 302 |                     {
 303 |                         state=1;
 304 |                         var.push_back(c);
 305 |                         pos++;
 306 |                     }
 307 | 
 308 |                     else
 309 |                     {
 310 |                         loop=false;
 311 |                     }
 312 | 
 313 |                     break;
 314 | 
 315 |             case 1 : if(isalnum(c))
 316 |                     {
 317 |                         state=1;
 318 |                         var.push_back(c);
 319 |                         pos++;
 320 |                     }
 321 | 
 322 |                     else
 323 |                     {
 324 |                         state=2;
 325 |                         pos++;
 326 |                     }
 327 | 
 328 |                     break;
 329 | 
 330 |             case 2 : valid = true;loop=false;pos--;
 331 |                     
 332 |                     /*
 333 |                     Steps : 
 334 |                     - Issue the token
 335 |                     - Enter this int Symbol Table
 336 |                     */
 337 | 
 338 |                    int id = SymbolTable.insert(vector_to_string(var));
 339 | 
 340 | 
 341 |                    if(id!=-1)
 342 |                    {
 343 |                        Token T("ID",id);
 344 |                        tokens.push_back(T);
 345 |                    }
 346 |                    else
 347 |                    {
 348 |                        Token T;
 349 |                        tokens.push_back(T); // it will return an un-identifiable Token found mean-while because token is already in symbol-table
 350 |                    }
 351 | 
 352 |                     break;
 353 |         }
 354 | 
 355 |     }
 356 | 
 357 |     if(valid==true)
 358 |     i=pos;
 359 | 
 360 |     return valid;
 361 | }
 362 | 
 363 | bool getArithmetic(string file_input, vector<Token> &tokens,int &i)
 364 | {
 365 |     bool valid = false;
 366 | 
 367 |     char c = file_input[i];
 368 | 
 369 |     Token T;
 370 | 
 371 |     switch(c)
 372 |     {
 373 |         case '+' : valid=true;
 374 |                     T = Token("ARITH",c);
 375 |                     tokens.push_back(T);
 376 | 
 377 |                     break;
 378 | 
 379 |         case '-' : valid=true;
 380 |                     T = Token("ARITH",c);
 381 |                     tokens.push_back(T);
 382 | 
 383 |                     break;
 384 | 
 385 |         case '*' : valid=true;
 386 |                     T = Token("ARITH",c);
 387 |                     tokens.push_back(T);
 388 | 
 389 |                     break;
 390 | 
 391 |         case '/' : valid=true;
 392 |                     T = Token("ARITH",c);
 393 |                     tokens.push_back(T);
 394 | 
 395 |                     break;
 396 |     }
 397 | 
 398 |     if(valid==true)
 399 |     i++;
 400 | 
 401 |     return valid;
 402 | }
 403 | 
 404 | bool getParenthesis(string file_input, vector<Token> &tokens,int &i)
 405 | {
 406 |     bool valid = false;
 407 | 
 408 |     char c = file_input[i];
 409 | 
 410 |     Token T;
 411 | 
 412 |     switch(c)
 413 |     {
 414 |         case '(' : valid=true;
 415 |                     T = Token("PAREN_OPEN",c);
 416 |                     tokens.push_back(T);
 417 | 
 418 |                     break;
 419 | 
 420 |         case ')' : valid=true;
 421 |                     T = Token("PAREN_CLOSE",c);
 422 |                     tokens.push_back(T);
 423 | 
 424 |                     break;
 425 | 
 426 |         case '{' : valid=true;
 427 |                     T = Token("PAREN_OPEN",c);
 428 |                     tokens.push_back(T);
 429 | 
 430 |                     break;
 431 | 
 432 |         case '}' : valid=true;
 433 |                     T = Token("PAREN_CLOSE",c);
 434 |                     tokens.push_back(T);
 435 | 
 436 |                     break;
 437 | 
 438 |         case '[' : valid=true;
 439 |                     T = Token("PAREN_OPEN",c);
 440 |                     tokens.push_back(T);
 441 | 
 442 |                     break;
 443 | 
 444 |         case ']' : valid=true;
 445 |                     T = Token("PAREN_CLOSE",c);
 446 |                     tokens.push_back(T);
 447 | 
 448 |                     break;
 449 |     }
 450 | 
 451 |     if(valid==true)
 452 |     i++;
 453 | 
 454 |     return valid;
 455 | }
 456 | 
 457 | bool getPunctuation(string file_input, vector<Token> &tokens,int &i)
 458 | {
 459 |     bool valid = false;
 460 | 
 461 |     char c = file_input[i];
 462 | 
 463 |     Token T;
 464 | 
 465 |     switch(c)
 466 |     {
 467 |         case ';' : valid=true;
 468 |                     T = Token("PUNC",c);
 469 |                     tokens.push_back(T);
 470 | 
 471 |                     break;
 472 | 
 473 |         case '"' : valid=true;
 474 |                     T = Token("PUNC",c);
 475 |                     tokens.push_back(T);
 476 | 
 477 |                     break;
 478 | 
 479 |         case '\'' : valid=true;
 480 |                     T = Token("PUNC",c);
 481 |                     tokens.push_back(T);
 482 | 
 483 |                     break;
 484 | 
 485 |     }
 486 | 
 487 |     if(valid==true)
 488 |     i++;
 489 | 
 490 |     return valid;
 491 | }
 492 | 
 493 | bool getWhitespace(string file_input, vector<Token> &tokens,int &i)
 494 | {
 495 |     int pos = i;
 496 | 
 497 |     int state = 0;
 498 | 
 499 |     bool loop = true;
 500 |     bool valid = false;
 501 | 
 502 |     char c;
 503 | 
 504 |     while(loop)
 505 |     {
 506 |         if(pos<=file_input.length())
 507 |         c = file_input[pos];
 508 |         else
 509 |         {loop = false;c='\0';}
 510 | 
 511 |         // we will call the c++ std::isspace(char ch) to check if its a white-space or not
 512 |         
 513 |         switch(state)
 514 |         {
 515 |         
 516 |         case 0 : if(isspace(c))
 517 |                     {
 518 |                         state = 1;
 519 |                         pos++;
 520 |                     }
 521 | 
 522 |                     else
 523 |                     {
 524 |                         loop = valid = false;
 525 |                     }
 526 | 
 527 |                     break;
 528 | 
 529 |         case 1 : if(isspace(c))
 530 |                     {
 531 |                         pos++;
 532 |                     }
 533 | 
 534 |                     else
 535 |                     {
 536 |                         state=2;
 537 |                         pos++;
 538 |                     }
 539 | 
 540 |                     break;
 541 | 
 542 |         case 2 : valid = true; loop=false; pos--;
 543 |                     break;
 544 | 
 545 |         }
 546 | 
 547 | 
 548 |     }
 549 | 
 550 |     if(valid==true)
 551 |     i=pos;
 552 | 
 553 |     return valid;
 554 | }
 555 | 
 556 | bool getRelop(string file_input, vector<Token> &tokens,int &i)
 557 | {
 558 |     int pos = i ; // saving the current position of the 'i' input stream pointer
 559 | 
 560 |     int state = 0; // state is initialized to 0
 561 | 
 562 |     bool loop = true; // while we have to loop
 563 |     bool valid = false; // valid = True when something is found
 564 | 
 565 |     char c;
 566 | 
 567 |     while(loop)
 568 |     {
 569 |         
 570 |         if(pos<=file_input.length())
 571 |         c = file_input[pos];
 572 |         else
 573 |         {loop = false;c='\0';}
 574 | 
 575 |         switch(state)
 576 |         {
 577 | 
 578 |         case 0 : if(c=='<')
 579 |                 {
 580 |                     pos++;
 581 |                     state = 1;
 582 |                 }
 583 | 
 584 |                 else if(c=='=')
 585 |                 {
 586 |                     pos++;
 587 |                     state = 5;
 588 |                 }
 589 | 
 590 |                 else if(c=='>')
 591 |                 {
 592 |                     pos++;
 593 |                     state = 6;
 594 |                 }
 595 | 
 596 |                 else
 597 |                 {
 598 |                     valid = false; // the valid turns to False
 599 |                     loop = false;
 600 |                 } 
 601 | 
 602 |                 break;
 603 | 
 604 | 
 605 |         case 1 :if(c=='=')
 606 |                 {
 607 |                     pos++;
 608 |                     state = 2;
 609 |                 }
 610 | 
 611 |                 else if(c=='>')
 612 |                 {
 613 |                     pos++;
 614 |                     state = 3;
 615 |                 }
 616 | 
 617 |                 else
 618 |                 {
 619 |                     pos++;
 620 |                     state = 4;
 621 |                 } 
 622 | 
 623 |                 break;
 624 | 
 625 | 
 626 |     
 627 |         case 2 :{Token T("RELOP","LE");
 628 |                 tokens.push_back(T);
 629 | 
 630 |                 valid = true;
 631 |                 loop = false;
 632 | 
 633 |                 break;}
 634 | 
 635 | 
 636 |         case 3 : {Token T("RELOP","NE");
 637 |                 tokens.push_back(T);
 638 | 
 639 |                 valid = true;
 640 |                 loop = false;
 641 | 
 642 |                 break;
 643 |                 }
 644 | 
 645 | 
 646 |         case 4 : {Token T("RELOP","LT");
 647 |                 tokens.push_back(T);
 648 | 
 649 |                 valid = true;
 650 |                 loop = false;
 651 | 
 652 |                 break;}
 653 | 
 654 | 
 655 |         case 5 : {Token T("RELOP","EQ");
 656 |                 tokens.push_back(T);
 657 |  
 658 |                 valid = true;
 659 |                 loop = false;
 660 | 
 661 |                 break;}
 662 | 
 663 | 
 664 |         case 6 : if(c=='=')
 665 |                 {
 666 |                     pos++;
 667 |                     state = 7;
 668 |                 }
 669 | 
 670 |                 else
 671 |                 {
 672 |                     pos++;
 673 |                     state = 8;
 674 |                 } 
 675 | 
 676 |                 break;
 677 | 
 678 | 
 679 |         case 7 : {Token T("RELOP","GE");
 680 |                 tokens.push_back(T);
 681 | 
 682 |                 valid = true;
 683 |                 loop = false;
 684 | 
 685 |                 break;}
 686 | 
 687 | 
 688 |         case 8 :{ Token T("RELOP","GT");
 689 |                 tokens.push_back(T);
 690 | 
 691 |                 pos--;
 692 |                 valid = true;
 693 |                 loop = false;
 694 | 
 695 |                 break;}
 696 | 
 697 |         // End of the loop and the switch conditional statements
 698 |     }
 699 |     }
 700 | 
 701 |     if(valid==true)
 702 |     i = pos; // update pointer to the input in stream
 703 | 
 704 |     return valid; // either true or false
 705 | }
 706 | 
 707 | bool getNumber(string file_input, vector<Token> &tokens,int &i)
 708 | {
 709 |     int pos = i ; // saving the current position of the 'i' input stream pointer
 710 | 
 711 |     int state = 0; // state is initialized to 0
 712 | 
 713 |     bool loop = true; // while we have to loop
 714 |     bool valid = false; // valid = True when something is found
 715 | 
 716 |     vector<char> number_string;
 717 | 
 718 |     char c ;
 719 | 
 720 |     while(loop)
 721 |     {
 722 |         if(pos<=file_input.length())
 723 |         c = file_input[pos];
 724 |         else
 725 |         {loop = false;c='\0';}
 726 | 
 727 |         switch(state)
 728 |         {
 729 | 
 730 |         case 0 : if(isdigit(c))
 731 |                 {
 732 |                     pos++;
 733 |                     state = 1;
 734 |                     number_string.push_back(c);
 735 |                 }
 736 | 
 737 |                 else
 738 |                 {
 739 |                     valid = false; // the valid turns to False
 740 |                     loop = false;
 741 |                 } 
 742 | 
 743 |                 break;
 744 | 
 745 | 
 746 |         case 1 :if(isdigit(c))
 747 |                 {
 748 |                     pos++;
 749 |                     state = 1;
 750 |                     number_string.push_back(c);
 751 |                 }
 752 | 
 753 |                 else if(c=='.')
 754 |                 {
 755 |                     pos++;
 756 |                     state = 3;
 757 |                     number_string.push_back(c);
 758 |                 }
 759 | 
 760 |                 else if(c=='e')
 761 |                 {
 762 |                     pos++;
 763 |                     state = 6;
 764 |                     number_string.push_back(c);
 765 |                 }
 766 | 
 767 |                 else
 768 |                 {
 769 |                     pos++;
 770 |                     state = 2;
 771 |                 } 
 772 | 
 773 |                 break;
 774 | 
 775 | 
 776 |         case 2 :{ Token T; // put the number into the token
 777 | 
 778 |                 T = Token("NUM",vector_to_string(number_string));
 779 | 
 780 |                 tokens.push_back(T);
 781 | 
 782 |                 pos--;
 783 |                 valid = true;
 784 |                 loop = false;
 785 | 
 786 |                 break;}
 787 | 
 788 | 
 789 |          case 3 :if(isdigit(c))
 790 |                 {
 791 |                     pos++;
 792 |                     state = 4;
 793 |                     number_string.push_back(c);
 794 |                 }
 795 | 
 796 |                 else
 797 |                 {
 798 |                     valid = false; // the valid turns to False
 799 |                     loop = false;
 800 |                 } 
 801 | 
 802 |                 break;
 803 | 
 804 | 
 805 |         case 4 :if(isdigit(c))
 806 |                 {
 807 |                     pos++;
 808 |                     state = 4;
 809 |                     number_string.push_back(c);
 810 |                 }
 811 | 
 812 |                 else if(c=='e')
 813 |                 {
 814 |                     pos++;
 815 |                     state = 6;
 816 |                     number_string.push_back(c);
 817 |                 }
 818 | 
 819 |                 else
 820 |                 {
 821 |                     pos++;
 822 |                     state = 5;
 823 |                 } 
 824 | 
 825 |                 break;
 826 | 
 827 | 
 828 |         case 5 : { Token T; // put the number into the token
 829 | 
 830 |                 T = Token("NUM",vector_to_string(number_string));
 831 | 
 832 |                 tokens.push_back(T);
 833 | 
 834 |                 pos--;
 835 |                 valid = true;
 836 |                 loop = false;
 837 | 
 838 |                 break;}
 839 | 
 840 | 
 841 |         case 6 :if(isdigit(c))
 842 |                 {
 843 |                     pos++;
 844 |                     state = 8;
 845 |                     number_string.push_back(c);
 846 |                 }
 847 | 
 848 |                 else if(c=='+' || c=='-')
 849 |                 {
 850 |                     pos++;
 851 |                     state = 7;
 852 |                     number_string.push_back(c);
 853 |                 }
 854 | 
 855 |                 else
 856 |                 {
 857 |                     valid = false; // the valid turns to False
 858 |                     loop = false;
 859 |                 } 
 860 | 
 861 |                 break;
 862 | 
 863 | 
 864 |         case 7 :if(isdigit(c))
 865 |                 {
 866 |                     pos++;
 867 |                     state = 8;
 868 |                     number_string.push_back(c);
 869 |                 }
 870 | 
 871 |                 else
 872 |                 {
 873 |                     valid = false; // the valid turns to False
 874 |                     loop = false;
 875 |                 } 
 876 | 
 877 |                 break;
 878 | 
 879 | 
 880 |         case 8 :if(isdigit(c))
 881 |                 {
 882 |                     pos++;
 883 |                     state = 8;
 884 |                     number_string.push_back(c);
 885 |                 }
 886 | 
 887 |                 else
 888 |                 {
 889 |                     pos++;
 890 |                     state = 9;
 891 |                 } 
 892 | 
 893 |                 break;
 894 | 
 895 | 
 896 |         case 9 : { Token T; // put the number into the token
 897 | 
 898 |                 T = Token("NUM",vector_to_string(number_string));
 899 | 
 900 |                 tokens.push_back(T);
 901 | 
 902 |                 pos--;
 903 |                 valid = true;
 904 |                 loop = false;
 905 | 
 906 |                 break;}
 907 | 
 908 |         // End of the loop and the switch conditional statements
 909 |     }
 910 |     }
 911 | 
 912 |     if(valid==true)
 913 |     i = pos; // update pointer to the input in stream
 914 | 
 915 |     return valid; // either true or false
 916 | }
 917 | 
 918 | void serial_lexical_analysis(string file_input, vector<Token> &tokens)
 919 | {
 920 |     int i = 0; // index where the file pointer is at the start - acts like string pointer
 921 | 
 922 |     while(file_input[i]!='\0' && i<=file_input.length()) // till the input stream does not end
 923 |     {
 924 | 
 925 |         if(getKeyword(file_input,tokens,i));
 926 | 
 927 |         else if(getVariable(file_input,tokens,i));
 928 | 
 929 |         else if(getNumber(file_input,tokens,i)); // i is the stream pointer - which will be updated if return is True
 930 | 
 931 |         else if(getRelop(file_input,tokens,i)); // i is the stream pointer - which will be updated if return is True
 932 | 
 933 |         else if(getArithmetic(file_input,tokens,i));
 934 | 
 935 |         else if(getParenthesis(file_input,tokens,i));
 936 |         
 937 |         else if(getPunctuation(file_input,tokens,i));
 938 | 
 939 |         else if(getWhitespace(file_input,tokens,i));
 940 |         
 941 |         else
 942 |         {
 943 |             Token T; // not an identifiabel token
 944 |             tokens.push_back(T);
 945 |             i++; // increment i
 946 |         }
 947 |     }
 948 | 
 949 | }
 950 | 
 951 | void parallel_lexical_analysis(string file_input, vector<Token> &tokens)
 952 | {
 953 |     int i = 0; // index where the file pointer is at the start - acts like string pointer
 954 | 
 955 |     while(file_input[i]!='\0' && i<=file_input.length()) // till the input stream does not end
 956 |     {
 957 |         int final_pos[8] = {0,0,0,0,0,0,0,0}; // Order : keyword, Variable, Number, Relop, Arithmetic, Parenthesis, Punctuation, Whitespace
 958 | 
 959 |         vector<Token> temp_tokens; // will be deleted as the loop iterates
 960 | 
 961 |         for(int k=0;k<8;k++)
 962 |         {
 963 |             final_pos[k] = i;
 964 | 
 965 |             switch(k)
 966 |             {
 967 |                 case 0 : getKeyword(file_input,temp_tokens,final_pos[0]); break;
 968 | 
 969 |                 case 1 : getVariable(file_input,temp_tokens,final_pos[1]); break;
 970 | 
 971 |                 case 2 : getNumber(file_input,temp_tokens,final_pos[2]); break;
 972 | 
 973 |                 case 3 : getRelop(file_input,temp_tokens,final_pos[3]); break;
 974 | 
 975 |                 case 4 : getArithmetic(file_input,temp_tokens,final_pos[4]); break;
 976 | 
 977 |                 case 5 : getParenthesis(file_input,temp_tokens,final_pos[5]); break;
 978 | 
 979 |                 case 6 : getPunctuation(file_input,temp_tokens,final_pos[6]); break;
 980 | 
 981 |                 case 7 : getWhitespace(file_input,temp_tokens,final_pos[7]); break;
 982 |             }
 983 |         }
 984 | 
 985 | 
 986 |         // see what to call finally
 987 | 
 988 |         int max_index = -1;
 989 |         int max_pos = i;
 990 | 
 991 | 
 992 |         for(int k=0;k<8;k++)
 993 |         {
 994 |             if(final_pos[k]>max_pos)
 995 |             {
 996 |                 max_index = k;
 997 |                 max_pos = final_pos[k];
 998 |             }
 999 |         }
1000 | 
1001 | 
1002 |         if(max_index == -1) // i.e no token is issued
1003 |         {
1004 |             Token T; // not an identifiabel token
1005 |             tokens.push_back(T);
1006 |             i++; // increment i
1007 |         }
1008 |         else
1009 |         {
1010 | 
1011 |             // A Token will surely be issued
1012 | 
1013 |             // correcting the Symbol Table
1014 | 
1015 |             if(final_pos[1]>i)
1016 |             SymbolTable.pop(); // popping the identifier if issued from the symbol table
1017 | 
1018 |             // Now issuing the original token by using the max_index
1019 | 
1020 |             switch(max_index)
1021 |             {
1022 |                 case 0 : getKeyword(file_input,tokens,i); break;
1023 | 
1024 |                 case 1 : getVariable(file_input,tokens,i);break;
1025 | 
1026 |                 case 2 : getNumber(file_input,tokens,i); break;
1027 | 
1028 |                 case 3 : getRelop(file_input,tokens,i); break;
1029 | 
1030 |                 case 4 : getArithmetic(file_input,tokens,i); break;
1031 | 
1032 |                 case 5 : getParenthesis(file_input,tokens,i); break;
1033 | 
1034 |                 case 6 : getPunctuation(file_input,tokens,i); break;
1035 | 
1036 |                 case 7 : getWhitespace(file_input,tokens,i); break;
1037 |             }
1038 | 
1039 |         }
1040 |         
1041 |     }
1042 | 
1043 | }
1044 | 
1045 | int main()
1046 | {
1047 | 
1048 |     // Initializing start
1049 | 
1050 |     initialize_keywords();
1051 | 
1052 |     // Initialization done
1053 | 
1054 |     cout<<"Enter number of test cases : ";
1055 |     int t; // test cases
1056 |     cin>>t;
1057 |     while(t--) // starts from t-1 to 0 ( t times )
1058 |     {
1059 |         cout<<"Enter the Input Stream : ";
1060 |         string input_stream;
1061 |         fflush(stdin);
1062 |         getline(cin,input_stream);
1063 |         cout<<"\n"<<"Lexical Analysis for Input Stream : "<<input_stream;
1064 | 
1065 |         vector<Token> tokens;
1066 | 
1067 |         // for serial navigation
1068 |         // serial_lexical_analysis(input_stream,tokens);
1069 | 
1070 |         // for paralle navigation
1071 |         parallel_lexical_analysis(input_stream,tokens);
1072 | 
1073 |         cout<<endl;
1074 |         for(int i=0;i<tokens.size();i++)
1075 |         cout<<tokens[i].t<<endl;
1076 | 
1077 |         cout<<"\n";
1078 |         cout<<endl;
1079 |     }
1080 |     return 0;
1081 | }


--------------------------------------------------------------------------------