├── .gitignore ├── Denada201.png ├── Denada202.png ├── README.md ├── check.go ├── context.go ├── denada.go ├── denada ├── .gitignore ├── check.go ├── cmdline.go ├── format.go └── parse.go ├── denada_parser.go ├── element.go ├── elist.go ├── grammar_test.go ├── import_test.go ├── marshal.go ├── marshal_test.go ├── parser_test.go ├── rules.go ├── rules_test.go ├── testsuite ├── case1.dnd ├── case2.dnd ├── case3.dnd ├── case4.dnd ├── case5.dnd ├── case6.dnd ├── case7.dnd ├── case8.dnd ├── config.grm ├── ecase1.dnd ├── ecase2.dnd ├── ecase3.dnd ├── ecase4.dnd ├── ecase5.dnd ├── ecase6.dnd ├── ecase7.dnd ├── ecase8.dnd ├── multi.grm ├── reference.grm └── schema.grm ├── testsuite_test.go ├── token.go ├── transforms.go ├── unparse.go └── utils.go /.gitignore: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/xogeny/denada-go/2e1e49d48ffacd12666801b129c1982f110fb8db/.gitignore -------------------------------------------------------------------------------- /Denada201.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/xogeny/denada-go/2e1e49d48ffacd12666801b129c1982f110fb8db/Denada201.png -------------------------------------------------------------------------------- /Denada202.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/xogeny/denada-go/2e1e49d48ffacd12666801b129c1982f110fb8db/Denada202.png -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Denada - A declarative language for creating simple DSLs 2 | 3 | This is an implementation of Denada in Go (golang). This is an 4 | attempt to recreate the functionality of my previous 5 | [Javascript implementation of Denada](https://github.com/xogeny/denada-js) 6 | in the Go language. 7 | 8 | ## TL;DR 9 | 10 | Denada allows you to very quickly create a DSL for expressing specific 11 | data and/or structure. It has the advantage (in my opinion) over XML 12 | and/or JSON that it allows you to formulate a DSL that is human 13 | readable and provide better diagnostic error messages. Defining a 14 | grammar for your DSL is super easy. 15 | 16 | ## Background 17 | 18 | Denada is based on a project I once worked on where we needed to build 19 | and quickly evolve a simple domain-specific language (DSL). But an 20 | important aspect of the project was that there were several 21 | non-developers involved. We developed a language very similar to this 22 | one that had several interesting properties (which I'll come to 23 | shortly). Recently, I was faced with a situation where I needed to 24 | develop a DSL for a project and decided to follow the same approach. 25 | 26 | I can already imagine people rolling their eyes at the premise. But 27 | give me five more minutes and I'll explain why I did it. 28 | 29 | There are lots of different ways to build DSLs. Let's take a quick 30 | walk across the spectrum of possibilities. One approach to DSL design 31 | is to create an "internal DSL" where you simply use clever syntactic 32 | tricks in some host language to create what looks like a domain 33 | specific language but is really just a set of domain specific 34 | primitives layered on top of an existing language. Scala is 35 | particularly good for things like this (using `implicit` constructs) 36 | but you can do it in a number of languages (yes, Lisp works well for 37 | this too...but I don't care for the aesthetics of homoiconic 38 | representations). The problem here is that you expose the user of the 39 | language to the (potentiall complicated) semantics of the host 40 | language. Depending on the use case, leveraging the host language's 41 | semantics could be a win (you need to implement similar semantics in 42 | your language) or a loss (you add a bunch of complexity and sharp 43 | edges to a language for non-experts). 44 | 45 | Another approach is to create a so-called "external DSL". For this, 46 | you might using a parser generator (e.g. ANTLR) to create a parser for 47 | your language. This allows you to completely define your semantics 48 | (without exposing people to the host language semantics). This allows 49 | you to control the complexity of the language. But you've still got 50 | to create the parser, debug parsing issues, generate a tree, and then 51 | write code to walk the tree. So this can be a significant investment 52 | of time. Sure, parser generators can really speed things up. But 53 | there are cases where some of this work can be skipped. 54 | 55 | Another route you can go is to just use various markup languages to 56 | try and represent your data. Representations like XML, YAML, JSON or 57 | even INI files can be used in this way. But for some cases this is 58 | either overly verbose, too "technical" or unreadable. 59 | 60 | ## So Why Denada? 61 | 62 | The general philosophy of Denada is to define a syntax *a priori*. As 63 | a result, you don't need to write a parser for it. You don't get a 64 | choice about how the language looks. Sure, it's fun to "design" 65 | languages. But there are a wide range of simple DSLs that can be 66 | implemented naturally within the limited syntax of Denada. 67 | 68 | So, I can already hear people saying "But if you've decided on the 69 | syntax, you've already designed **a** language, what's all this talk 70 | about designing DSLs". Although the syntax of Denada is defined, 71 | **the grammar isn't**. Denada allows us to impose a grammar on top of 72 | the syntax in the same way that "XML Schemas" allow us to impose a 73 | structure on top of XML. And, like "XML Schemas" we use the same 74 | syntax for the grammar as for the language. But unlike anything XML 75 | related, Denada looks (kind of) like a computer language designed for 76 | humans to read. It also includes a richer data model. 77 | 78 | ## An Example 79 | 80 | To demonstrate how Denada works, let's work through a simple example. 81 | Imagine I'm a system administator and I need a file format to list all 82 | the assets in the company. 83 | 84 | The generic syntax of Denada is simple. There are two things you can 85 | express in Denada. One looks like a variable declaration and the 86 | other expresses nested structure (which can contain instances of these 87 | same two things). For example, this is valid Denada code: 88 | 89 | ``` 90 | printer ABC { 91 | set location = "By my desk"; 92 | set model = "HP 8860"; 93 | } 94 | ``` 95 | 96 | This doesn't really *mean* anything, but it conforms to the required 97 | syntax. Although this might be useful as is, the real use case for 98 | Denada is defining a grammar that restricts what is permitted. That's 99 | because this is also completely legal: 100 | 101 | ``` 102 | aardvark ABC { 103 | set location = "By my desk"; 104 | set order = "Tubulidentata"; 105 | } 106 | ``` 107 | 108 | So in this case, we want to restrict ourselves (initially) to 109 | cataloging printers. To do this, we specify a grammar for our assets 110 | file. Initially, our grammar could look like this: 111 | 112 | ``` 113 | printer _ "printer*" { 114 | set location = "$string" "location"; 115 | set model = "$string" "model"; 116 | set networkName = "$string" "name?"; 117 | } 118 | ``` 119 | 120 | Note how this looks almost exactly like our original input text? That 121 | is because **grammars in Denada are Denada files**. They just have 122 | some special annotations (not syntax!). In this case, the "name" of 123 | the printer is given as just `_`. This is a wildcard in Denada means 124 | "any identifier". Also note the "descriptive string" following the 125 | printer definition, `"printer*"`. That means that this defines the 126 | `printer` rule and the star indicates we can have zero or more of them 127 | in our file. 128 | 129 | Furthermore, this grammar defines the contents of a `printer` 130 | specification (*i.e.*, what information we associated with a printer). 131 | It shows that there can be three lines inside a printer definition. 132 | The first is the `location` of the printer. This is mandatory because 133 | the rule name, `"location"` has no cardinality specified. Similarly, 134 | we also have a mandatory `model` property. Finally, we have an 135 | optional `networkName` property. We know it is optional because the 136 | rule name `"name?"` ends with a `?`. 137 | 138 | By defining the grammar in this way, we specify precisely what can be 139 | included in the Denada file. But let's not limit ourselves to 140 | printers. Assume we want to list the computers in the company too. 141 | We could simply create a new rule for computers, *e.g.,* 142 | 143 | ``` 144 | printer _ "printer*" { 145 | set location = "$string" "location"; 146 | set model = "$string" "model"; 147 | set networkName = "$string" "name?"; 148 | } 149 | 150 | computer _ "computer*" { 151 | set location = "$string" "location"; 152 | set model = "$string" "model"; 153 | set networkName = "$string" "name?"; 154 | } 155 | ``` 156 | 157 | In this case, the contents of these definitions are the same, so we 158 | could even do this: 159 | 160 | ``` 161 | 'printer|computer' _ "asset*" { 162 | set location = "$string" "location"; 163 | set model = "$string" "model"; 164 | set networkName = "$string" "name?"; 165 | } 166 | ``` 167 | 168 | With just this simple grammar, we've created a parser for a DSL that 169 | can parse our sample asset list above and flag errors. 170 | 171 | ### Named Rules and Recursion 172 | 173 | To created recursively nested grammars, it is necessary to somehow 174 | "break" the potentially infinite structure. Since Denada grammars are 175 | (at least up until now) isomorphic with the input structures, handling 176 | recursion is a challenge. In fact, any time there are repeated 177 | patterns you'll have an issue with repeating yourself (potentially an 178 | infinite number of times). 179 | 180 | For this reason, the "description" field of a **definition** rule can 181 | also include a specification of what **rules** should be matched for 182 | the children. The plural in "rules" is important. The specification 183 | for child rules appears in the description after a `>`. What follows 184 | the `>` can be one of the following: 185 | 186 | * `$root` - Use the rules that appear at the root of the document. 187 | 188 | * `$this` - Use the children of the current definition. This 189 | is the default so you never have to specify it explicitly (although it 190 | will work). 191 | 192 | * `` - Where `` is the name of a **fully 193 | qualified** definition rule. The set of possible rule matches 194 | for the children will correspond to the children of all rules 195 | that match the specified rulename. Since multiple rules can 196 | have the same name, the search for a match is done across 197 | **all** children from all rules. 198 | 199 | A simple and convenient shorthand here is to prefix the rule 200 | description with a `^`. This indicates that the children of the 201 | associated definition should match the siblings of that definition 202 | (*i.e.,* this is a simple way to describe a simple recursive 203 | relationship where the same entities can appear at every level from 204 | this point down). 205 | 206 | Note that the Denada cardinality syntax allows you to specify a `-` 207 | for the cardinality. This means exactly zero occurences of that 208 | entity. This is useful for creating collections of rules that are 209 | never matched by themselves, but can be referred to in multiple 210 | locations within the grammar for specifying the rules for children. 211 | 212 | ## Denada Syntax 213 | 214 | Here is EBNF for the Denada language: 215 | 216 | ``` 217 | File = { Definition | Declaration } . 218 | 219 | Definition = Preface [ string ] "{" File "}" . 220 | 221 | Declaration = Preface [ "=" expr ] [ string ] ";" . 222 | 223 | QualifiersAndId = { identifier } identifier . 224 | 225 | Modification = identifier "=" expr . 226 | 227 | Modifiers = "(" [ Modification { "," Modification } ] ")" . 228 | 229 | Preface = QualifiersAndId [ Modifiers ] . 230 | ``` 231 | 232 | Lexically, we have only a few types of tokens. Before getting into 233 | what matters, let's point out that whitespace and C++ style comments 234 | are ignored in the grammar (*i.e.*, they can be removed during lexical 235 | analysis). 236 | 237 | It is important to point out that because the grammar for a given 238 | Denada DSL is written in Denada, we need a lot of latitude in our 239 | identifiers (the ``identifier`` token in the EBNF grammar). This is 240 | because they will be used as regular expressions when they are used 241 | within a grammar. As such, an identifier in Denada is any sequence of 242 | characters that doesn't contain whitespace, a comment or the reserved 243 | characters ``{``, ``}``, ``(``, ``)``, ``/``, ``"``, ``=``, ``;`` or 244 | ``,``. 245 | 246 | There are really only two other token types in Denada. The first is 247 | quoted strings (the ``string`` token in the EBNF grammar). This is 248 | just a sequence of characters that start with a ``"`` and end with an 249 | unescaped ``"``. 250 | 251 | Finally, we have the ``expr`` token. This is a JSON value (*i.e*, not 252 | necessarily an object, but a value). 253 | -------------------------------------------------------------------------------- /check.go: -------------------------------------------------------------------------------- 1 | package denada 2 | 3 | import "fmt" 4 | import "regexp" 5 | import "log" 6 | 7 | import "github.com/bitly/go-simplejson" 8 | import "github.com/xeipuuv/gojsonschema" 9 | 10 | func Check(input ElementList, grammar ElementList, diag bool) error { 11 | context := RootContext(grammar) 12 | return CheckContents(input, grammar, diag, "", "", context) 13 | } 14 | 15 | type matchInfo struct { 16 | count int 17 | rule RuleInfo 18 | desc string 19 | } 20 | 21 | func CheckContents(input ElementList, grammar ElementList, diag bool, 22 | prefix string, parentRule string, context RuleContext) error { 23 | 24 | if len(grammar) == 0 && len(input) != 0 { 25 | return fmt.Errorf("Failure: No rules to match these elements %v (in context %v)", 26 | input, context) 27 | } 28 | 29 | // Initialize data associated with rule matching 30 | counts := map[string]*matchInfo{} 31 | 32 | // Loop over grammar rules and record counts information 33 | for _, g := range grammar { 34 | // Make sure grammar element has a (rule) description 35 | if g.Description == "" { 36 | return fmt.Errorf("Grammar element %s has no description", g.String()) 37 | } 38 | 39 | // Parse the rule information from the description 40 | rule, err := ParseRule(g.Description, ChildContext(g.Contents, &context)) 41 | 42 | // If there is an error in the rule description, add an error and 43 | // skip this grammar element 44 | if err != nil { 45 | return fmt.Errorf("Error in rule description: %v", err) 46 | } 47 | 48 | mi, exists := counts[rule.Name] 49 | if exists { 50 | if rule.Name != mi.rule.Name || rule.Cardinality != mi.rule.Cardinality { 51 | return fmt.Errorf("Unmatching rules with same name: %s vs %s", 52 | g.Description, mi.desc) 53 | } 54 | } else { 55 | counts[rule.Name] = &matchInfo{count: 0, rule: rule, desc: g.Description} 56 | } 57 | 58 | /* 59 | // Also initialize the named contexts if this is a definition 60 | if g.isDefinition() { 61 | // First, construct the fully qualified name for this definition's rule 62 | path := parentRule + "." + rule.Name 63 | if parentRule == "" { 64 | path = rule.Name 65 | } 66 | // Check to see if another rule has this same name (possible because of 67 | // the idiomatic use of multiple rules with the same name indicating an 68 | // or relationship) 69 | ctxt, exists := context[path] 70 | if exists { 71 | context[path] = append(ctxt, grammar...) 72 | } else { 73 | context[path] = grammar 74 | } 75 | } 76 | */ 77 | } 78 | 79 | // Now, loop over all the actual input elements and see if they match 80 | // any of the rules 81 | for _, in := range input { 82 | var likely error = nil 83 | ierrs := []error{} 84 | for _, g := range grammar { 85 | // Parse the rule information from the description (ignore error 86 | // because we already checked that) 87 | 88 | rule, _ := ParseRule(g.Description, ChildContext(g.Contents, &context)) 89 | 90 | path := parentRule + "." + rule.Name 91 | if parentRule == "" { 92 | path = rule.Name 93 | } 94 | 95 | ematch := matchElement(in, g, rule.Context.This, diag, prefix, 96 | path, rule.Context) 97 | if ematch == nil { 98 | // A match was found, so increment the count for this particular 99 | // grammar rule 100 | counts[rule.Name].count++ 101 | 102 | // Then check to see if this input has matched any previous rules 103 | // If not, then choose this match. This implies that the first 104 | // rule to match is the one that is chosen 105 | if in.rule == "" { 106 | in.rulepath = path 107 | in.rule = rule.Name 108 | // If not, indicate what rule this input matched 109 | if diag { 110 | log.Printf("%sInput %s matched %s (path: %s)", 111 | prefix, in.String(), rule.Name, in.rulepath) 112 | } 113 | } 114 | } else { 115 | if diag { 116 | log.Printf("%sInput %s did not match %s because\n%s", prefix, in.String(), 117 | rule.Name, ematch.Error()) 118 | } 119 | if len(grammar) == 1 { 120 | return ematch 121 | } 122 | if len(in.Qualifiers) == 1 && len(g.Qualifiers) == 1 && 123 | in.Qualifiers[0] == g.Qualifiers[0] { 124 | likely = ematch 125 | } 126 | if in.Name == g.Name { 127 | likely = ematch 128 | } 129 | ierrs = append(ierrs, ematch) 130 | } 131 | } 132 | if in.rule == "" { 133 | if likely == nil { 134 | if len(ierrs) == 0 { 135 | return fmt.Errorf("No match for element %v (empty rules?!?)", in) 136 | } else { 137 | return fmt.Errorf("No match for element %v because %v", 138 | in, listToError(ierrs)) 139 | } 140 | } else { 141 | return likely 142 | } 143 | } 144 | } 145 | 146 | // Check to make sure that all rules were matched the correct number 147 | // of times. 148 | for _, mi := range counts { 149 | rerrs := []error{} 150 | err := mi.rule.checkCount(mi.count) 151 | if err != nil { 152 | rerrs = append(rerrs, err) 153 | } 154 | if len(rerrs) > 0 { 155 | return listToError(rerrs) 156 | } 157 | } 158 | 159 | return nil 160 | } 161 | 162 | func matchString(input string, grammar string) bool { 163 | if grammar == "_" { 164 | return true 165 | } 166 | matched, err := regexp.MatchString(grammar, input) 167 | if err == nil && matched { 168 | return true 169 | } 170 | return false 171 | } 172 | 173 | func matchQualifiers(input *Element, grammar *Element) bool { 174 | imatch := make([]bool, len(input.Qualifiers)) 175 | for _, g := range grammar.Qualifiers { 176 | count := 0 177 | 178 | rule, err := ParseRuleName(g) 179 | 180 | if err != nil { 181 | log.Printf("Error parsing rule information in qualifier '%s': %v", g, err) 182 | return false 183 | } 184 | 185 | for i, in := range input.Qualifiers { 186 | matched := matchString(in, rule.Name) 187 | if matched { 188 | imatch[i] = true 189 | count++ 190 | } 191 | } 192 | 193 | // Check to see if the correct number of matches were found for this qualifier 194 | err = rule.checkCount(count) 195 | if err != nil { 196 | // If not, this is not a match 197 | return false 198 | } 199 | } 200 | 201 | // Now check to make sure every qualifier on the input element had a match 202 | for i, _ := range input.Qualifiers { 203 | if !imatch[i] { 204 | // This qualifier on the input element was never matched 205 | return false 206 | } 207 | } 208 | 209 | return true 210 | } 211 | 212 | func matchModifications(input *Element, grammar *Element, diag bool) bool { 213 | // Create a map to keep track of which modification keys on the input 214 | // element find a match 215 | imatch := map[string]bool{} 216 | for k, _ := range input.Modifications { 217 | imatch[k] = false 218 | } 219 | 220 | // Now loop over all keys and expresions in the grammar 221 | for r, ge := range grammar.Modifications { 222 | count := 0 223 | 224 | // Parse the rule 225 | rule, err := ParseRuleName(r) 226 | 227 | if err != nil { 228 | // If the rule is not valid, assume no match 229 | log.Printf("Error parsing rule information in key '%s': %v", r, err) 230 | return false 231 | } 232 | 233 | // Loop over all actual modification keys and values 234 | for i, ie := range input.Modifications { 235 | // Check to see if the keys match 236 | matched := matchString(i, rule.Name) 237 | if matched { 238 | // If so, check if the expressions match 239 | if matchExpr(ie, ge, diag) { 240 | // If so, this input is matched and so is the grammar rule 241 | imatch[i] = true 242 | count++ 243 | } 244 | } 245 | } 246 | 247 | // Now check to make sure this grammar rule has been matched an appropriate 248 | // number of times 249 | err = rule.checkCount(count) 250 | if err != nil { 251 | // If not, no match 252 | return false 253 | } 254 | } 255 | 256 | // Now check to make sure every key on the input element had a match 257 | for k, _ := range input.Modifications { 258 | if !imatch[k] { 259 | // This key on the input element was never matched 260 | return false 261 | } 262 | } 263 | 264 | return true 265 | } 266 | 267 | // Validation rules: 268 | // Grammar expr is: 269 | // String that starts with $ -> Look for type match 270 | // String (without $) -> Exact match 271 | // Object -> Treat object as a JSON schema and validate input with it 272 | // Otherwise -> No match 273 | func matchExpr(input *simplejson.Json, grammar *simplejson.Json, diag bool) bool { 274 | if grammar == nil && input == nil { 275 | return true 276 | } 277 | if grammar == nil || input == nil { 278 | if diag { 279 | log.Printf("Grammar was %v while input was %v", grammar, nil) 280 | } 281 | return false 282 | } 283 | stype, err := grammar.String() 284 | if err == nil { 285 | switch stype { 286 | case "$_": 287 | return true 288 | case "$string": 289 | _, terr := input.String() 290 | if terr != nil && diag { 291 | log.Printf("Input wasn't a string") 292 | } 293 | return terr == nil 294 | case "$bool": 295 | _, terr := input.Bool() 296 | if terr != nil && diag { 297 | log.Printf("Input wasn't a bool") 298 | } 299 | return terr == nil 300 | case "$int": 301 | _, terr := input.Int64() 302 | if terr != nil && diag { 303 | log.Printf("Input wasn't an int") 304 | } 305 | return terr == nil 306 | case "$number": 307 | _, terr := input.Float64() 308 | if terr != nil && diag { 309 | log.Printf("Input wasn't a number") 310 | } 311 | return terr == nil 312 | default: 313 | is, terr := input.String() 314 | log.Printf("treated as literal") 315 | return terr == nil && is == stype 316 | } 317 | } 318 | mtype, err := grammar.Map() 319 | if err == nil { 320 | schemaLoader := gojsonschema.NewGoLoader(mtype) 321 | documentLoader := gojsonschema.NewGoLoader(input) 322 | 323 | result, err := gojsonschema.Validate(schemaLoader, documentLoader) 324 | if err != nil { 325 | log.Printf("Validation error: %v", err) 326 | 327 | return false 328 | } 329 | 330 | for _, e := range result.Errors() { 331 | log.Printf(" JSON Schema validation failed because: %s", e) 332 | } 333 | return result.Valid() 334 | } 335 | return false 336 | } 337 | 338 | func matchElement(input *Element, grammar *Element, children ElementList, 339 | diag bool, prefix string, parentRule string, context RuleContext) error { 340 | // Check if the names match 341 | matched := matchString(input.Name, grammar.Name) 342 | 343 | // If the names don't match, no match 344 | if !matched { 345 | return fmt.Errorf("Name mismatch (%s doesn't match pattern %s)", 346 | input.Name, grammar.Name) 347 | } 348 | 349 | // Check whether the input is a definition or declaration 350 | if input.IsDefinition() { 351 | if grammar.IsDeclaration() { 352 | // If the input is a definition but the grammar is a declaration, no match 353 | return fmt.Errorf("Element type mismatch between %v and %v", input, grammar) 354 | } 355 | cerr := CheckContents(input.Contents, children, diag, prefix+" ", parentRule, context) 356 | if cerr != nil { 357 | // If the contents of input don't match the contents of grammar, no match 358 | return cerr 359 | } 360 | } else { 361 | if grammar.IsDefinition() { 362 | // If the input is a declaration but the grammar is a definition, no match 363 | return fmt.Errorf("Element type mismatch between %v and %v", input, grammar) 364 | } 365 | if !matchExpr(input.Value, grammar.Value, diag) { 366 | if input.Value == nil && grammar.Value != nil { 367 | return fmt.Errorf("Value pattern mismatch: vs %s", 368 | unparseValue(grammar.Value, "")) 369 | } else if input.Value != nil && grammar.Value == nil { 370 | return fmt.Errorf("Value pattern mismatch: %s vs ", 371 | unparseValue(input.Value, "")) 372 | } else { 373 | return fmt.Errorf("Value pattern mismatch: %s vs %s", 374 | unparseValue(input.Value, ""), unparseValue(grammar.Value, "")) 375 | } 376 | } 377 | } 378 | 379 | // TODO: Move these up, since they are quicker to establish 380 | if !matchQualifiers(input, grammar) { 381 | return fmt.Errorf("Qualifier mismatch (%v vs %v)", input.Qualifiers, 382 | grammar.Qualifiers) 383 | } 384 | 385 | if !matchModifications(input, grammar, diag) { 386 | return fmt.Errorf("Modification mismatch (%v vs %v)", input.Modifications, 387 | grammar.Modifications) 388 | } 389 | 390 | return nil 391 | } 392 | -------------------------------------------------------------------------------- /context.go: -------------------------------------------------------------------------------- 1 | package denada 2 | 3 | import "fmt" 4 | 5 | type RuleContext struct { 6 | This ElementList 7 | parent *RuleContext 8 | } 9 | 10 | func NullContext() RuleContext { 11 | return RuleContext{ 12 | This: ElementList{}, 13 | parent: nil, 14 | } 15 | } 16 | 17 | func RootContext(elems ElementList) RuleContext { 18 | return RuleContext{ 19 | This: elems, 20 | parent: nil, 21 | } 22 | } 23 | 24 | func ChildContext(elems ElementList, parent *RuleContext) RuleContext { 25 | return RuleContext{ 26 | This: elems, 27 | parent: parent, 28 | } 29 | } 30 | 31 | func (c RuleContext) String() string { 32 | names := []string{} 33 | for _, e := range c.This { 34 | rule, err := ParseRuleName(e.Description) 35 | if err != nil { 36 | names = append(names, "") 37 | } else { 38 | names = append(names, rule.Name) 39 | } 40 | } 41 | if c.parent == nil { 42 | return fmt.Sprintf("[%s]", names) 43 | } else { 44 | return fmt.Sprintf("[%s (%v)]", names, c.parent) 45 | } 46 | } 47 | 48 | func (c RuleContext) Find(path ...string) (ret RuleContext, err error) { 49 | /* If no path is provided, they must me this context */ 50 | if len(path) == 0 { 51 | ret = c 52 | return 53 | } 54 | 55 | /* Take the first element of the path and check against some "reserved" names */ 56 | head := path[0] 57 | 58 | /* Are they looking in the current context? */ 59 | if head == "." { 60 | return c.Find(path[1:]...) 61 | } 62 | 63 | /* Are they looking for the parent context? */ 64 | if head == ".." { 65 | if c.parent == nil { 66 | /* If we are at the root, we have no parent */ 67 | err = fmt.Errorf("Requested rule context at root level") 68 | return 69 | } else { 70 | /* Otherwise, resume the search in our parent context */ 71 | return c.parent.Find(path[1:]...) 72 | } 73 | } 74 | 75 | /* Are they looking for the root context? */ 76 | if head == "$root" { 77 | if c.parent == nil { 78 | /* If we are at the root, search this context */ 79 | return c.Find(path[1:]...) 80 | } else { 81 | /* If not, ask our parent for $root */ 82 | return c.parent.Find(path...) 83 | } 84 | } 85 | 86 | /* Check to see if head matches any uniquely named child definitions */ 87 | result := ElementList{} 88 | for _, d := range c.This { 89 | if !d.IsDefinition() { 90 | continue 91 | } 92 | rule, err := ParseRuleName(d.Description) 93 | if err != nil { 94 | continue 95 | } 96 | if rule.Name == head { 97 | result = append(result, d.Contents...) 98 | } 99 | } 100 | if result != nil { 101 | return ChildContext(result, &c), nil 102 | } 103 | 104 | err = fmt.Errorf("Unable to find context %s", head) 105 | return 106 | } 107 | -------------------------------------------------------------------------------- /denada.go: -------------------------------------------------------------------------------- 1 | package denada 2 | 3 | import "os" 4 | import "io" 5 | import "fmt" 6 | import "strings" 7 | 8 | // This file contains the API for the denada parser 9 | 10 | var errorList []error 11 | 12 | func listToError(l []error) error { 13 | msg := "Parsing errors:" 14 | for _, e := range l { 15 | msg += fmt.Sprintf("\n %v", e) 16 | } 17 | return fmt.Errorf("%s", msg) 18 | } 19 | 20 | func ParseString(s string) (ElementList, error) { 21 | r := strings.NewReader(s) 22 | return Parse(r) 23 | } 24 | 25 | func ParseFile(filename string) (ElementList, error) { 26 | r, err := os.Open(filename) 27 | if err != nil { 28 | return nil, err 29 | } 30 | defer r.Close() 31 | 32 | p, err := NewParser(r, filename) 33 | if err != nil { 34 | return nil, err 35 | } 36 | return p.ParseFile() 37 | } 38 | 39 | func Parse(r io.Reader) (ElementList, error) { 40 | p, err := NewParser(r, "") 41 | if err != nil { 42 | return nil, err 43 | } 44 | return p.ParseFile() 45 | } 46 | -------------------------------------------------------------------------------- /denada/.gitignore: -------------------------------------------------------------------------------- 1 | denada 2 | -------------------------------------------------------------------------------- /denada/check.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import "fmt" 4 | import "github.com/xogeny/denada-go" 5 | 6 | type CheckCommand struct { 7 | Positional struct { 8 | Input string `description:"Input file"` 9 | Grammar string `description:"Grammar file"` 10 | } `positional-args:"true" required:"true"` 11 | } 12 | 13 | func (f CheckCommand) Execute(args []string) error { 14 | if len(args) > 0 { 15 | return fmt.Errorf("Too many arguments") 16 | } 17 | 18 | ifile := f.Positional.Input 19 | 20 | elems, err := denada.ParseFile(ifile) 21 | if err != nil { 22 | return fmt.Errorf("Error parsing input file %s: %v", ifile, err) 23 | } 24 | 25 | gfile := f.Positional.Grammar 26 | grammar, err := denada.ParseFile(gfile) 27 | if err != nil { 28 | return fmt.Errorf("Error parsing grammar file %s: %v", gfile, err) 29 | } 30 | 31 | err = denada.Check(elems, grammar, false) 32 | if err != nil { 33 | denada.Check(elems, grammar, true) 34 | return fmt.Errorf("File %s was not a valid instance of the grammar in %s", 35 | ifile, gfile) 36 | } 37 | 38 | return nil 39 | } 40 | -------------------------------------------------------------------------------- /denada/cmdline.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import "os" 4 | 5 | import "github.com/jessevdk/go-flags" 6 | 7 | func main() { 8 | var options struct{} 9 | 10 | parser := flags.NewParser(&options, flags.Default) 11 | 12 | parser.AddCommand("format", 13 | "Rewrite a Denada file in canonical form", 14 | "Rewrite a Denada file in canonical form", 15 | &FormatCommand{}) 16 | 17 | parser.AddCommand("parse", 18 | "Parse a Denada file", 19 | "Parse a Denada file", 20 | &ParseCommand{}) 21 | 22 | parser.AddCommand("check", 23 | "Parse a Denada file and check it against a grammar file", 24 | "Parse a Denada file and check it against a grammar file", 25 | &CheckCommand{}) 26 | 27 | if _, err := parser.Parse(); err != nil { 28 | os.Exit(1) 29 | } 30 | } 31 | -------------------------------------------------------------------------------- /denada/format.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import "os" 4 | import "fmt" 5 | import "github.com/xogeny/denada-go" 6 | 7 | type FormatCommand struct { 8 | Positional struct { 9 | Term string `description:"Input file"` 10 | } `positional-args:"true" required:"true"` 11 | } 12 | 13 | func (f FormatCommand) Execute(args []string) error { 14 | if len(args) > 0 { 15 | return fmt.Errorf("Too many arguments") 16 | } 17 | 18 | file := f.Positional.Term 19 | elems, err := denada.ParseFile(file) 20 | if err != nil { 21 | return fmt.Errorf("Error parsing input file %s: %v", file, err) 22 | } 23 | 24 | fp, err := os.Create(file) 25 | if err != nil { 26 | return fmt.Errorf("Error rewriting %s: %v", file, err) 27 | } 28 | defer fp.Close() 29 | denada.UnparseTo(elems, fp) 30 | return nil 31 | } 32 | -------------------------------------------------------------------------------- /denada/parse.go: -------------------------------------------------------------------------------- 1 | package main 2 | 3 | import "os" 4 | import "fmt" 5 | import "path" 6 | import "github.com/xogeny/denada-go" 7 | 8 | type ParseCommand struct { 9 | Positional struct { 10 | Input string `description:"Input file"` 11 | } `positional-args:"true" required:"true"` 12 | Import bool `short:"i" long:"import" description:"Expand imports"` 13 | Echo bool `short:"e" long:"echo" description:"Echo parsed data"` 14 | } 15 | 16 | func (f ParseCommand) Execute(args []string) error { 17 | if len(args) > 0 { 18 | return fmt.Errorf("Too many arguments") 19 | } 20 | 21 | file := f.Positional.Input 22 | 23 | err := os.Chdir(path.Dir(file)) 24 | if err != nil { 25 | return fmt.Errorf("Error changing directory to %s: %v", path.Dir(file), err) 26 | } 27 | 28 | elems, err := denada.ParseFile(file) 29 | if err != nil { 30 | return fmt.Errorf("Error parsing input file %s: %v", file, err) 31 | } 32 | 33 | if f.Import { 34 | elems, err = denada.ImportTransform(elems) 35 | if err != nil { 36 | return fmt.Errorf("Error doing imports in %s: %v", file, err) 37 | } 38 | } 39 | 40 | if f.Echo { 41 | denada.UnparseTo(elems, os.Stdout) 42 | } 43 | 44 | fmt.Printf("File %s is syntactically correct Denada\n", file) 45 | return nil 46 | } 47 | -------------------------------------------------------------------------------- /denada_parser.go: -------------------------------------------------------------------------------- 1 | package denada 2 | 3 | import "io" 4 | 5 | import "log" 6 | import "fmt" 7 | import "bytes" 8 | import "strings" 9 | import "io/ioutil" 10 | import "encoding/json" 11 | 12 | import "github.com/bitly/go-simplejson" 13 | 14 | type Parser struct { 15 | src *strings.Reader 16 | lineNumber int 17 | colNumber int 18 | file string 19 | } 20 | 21 | func NewParser(s io.Reader, file string) (p *Parser, err error) { 22 | str, err := ioutil.ReadAll(s) 23 | if err != nil { 24 | return 25 | } 26 | src := strings.NewReader(string(str)) 27 | p = &Parser{src: src, lineNumber: 0, colNumber: 0, file: file} 28 | return 29 | } 30 | func (p *Parser) ParseFile() (ElementList, error) { 31 | ret := ElementList{} 32 | for { 33 | // Try to parse an Element 34 | elem, err := p.ParseElement(true) 35 | 36 | // If any other error 37 | if err != nil { 38 | return nil, err 39 | } 40 | 41 | // If elem is nil, that means there are no more elements to parse 42 | if elem == nil { 43 | return ret, nil 44 | } else { 45 | // Add element and continue 46 | ret = append(ret, elem) 47 | } 48 | } 49 | } 50 | 51 | // Returns an element if one found, nil on "}"...otherwise an error 52 | func (p *Parser) ParseElement(parsingFile bool) (ret *Element, err error) { 53 | ret = &Element{} 54 | 55 | // Get first token of the element 56 | t, err := p.nextNonWhiteToken() 57 | if err != nil { 58 | log.Printf("ParseElement is returning with error %v", err) 59 | return 60 | } 61 | 62 | // Depending on the context, the element list is terminated by 63 | // either an EOF or a }. Check for these... 64 | if t.Type == T_EOF { 65 | if parsingFile { 66 | // Expected, indicate no more elements 67 | return nil, nil 68 | } else { 69 | // Unexpected 70 | err = t.Expected("definition, declaration or '}'") 71 | return 72 | } 73 | } 74 | 75 | if t.Type == T_RBRACE { 76 | if parsingFile { 77 | // Unexpected 78 | err = t.Expected("definition, declaration or EOF") 79 | return 80 | } else { 81 | // Expected 82 | return nil, nil 83 | } 84 | } 85 | 86 | // Assuming there are more elements, the first thing should always 87 | // be an identifier 88 | if t.Type != T_IDENTIFIER { 89 | err = t.Expected("definition, declaration or EOF") 90 | return 91 | } 92 | ret.Name = t.String 93 | 94 | // Get next token 95 | t, err = p.nextNonWhiteToken() 96 | if err != nil { 97 | return 98 | } 99 | 100 | // Parse all remaining identifiers and white space 101 | for { 102 | if t.Type == T_IDENTIFIER { 103 | // More qualifiers 104 | ret.Qualifiers = append(ret.Qualifiers, ret.Name) 105 | ret.Name = t.String 106 | } else if t.Type == T_LPAREN || t.Type == T_QUOTE || t.Type == T_EQUALS || 107 | t.Type == T_SEMI || t.Type == T_LBRACE { 108 | // Expected next tokens 109 | break 110 | } else { 111 | err = UnexpectedToken{ 112 | Found: t, 113 | Expected: "identifier, whitespace, (, \", =, ; or {", 114 | } 115 | return 116 | } 117 | t, err = p.nextNonWhiteToken() 118 | if err != nil { 119 | return 120 | } 121 | } 122 | 123 | // Check if there is a modification (declaration or definition) 124 | if t.Type == T_LPAREN { 125 | ret.Modifications, err = p.ParseModifications() 126 | if err != nil { 127 | return 128 | } 129 | // Now get next token 130 | t, err = p.nextNonWhiteToken() 131 | if err != nil { 132 | return 133 | } 134 | } 135 | 136 | foundString := false 137 | 138 | // Read the description, if present 139 | if t.Type == T_QUOTE { 140 | ret.Description, err = p.ParseString() 141 | if err != nil { 142 | return 143 | } 144 | foundString = true 145 | 146 | // Now get next token 147 | t, err = p.nextNonWhiteToken() 148 | if err != nil { 149 | return 150 | } 151 | } 152 | 153 | // Is this a definition? 154 | if t.Type == T_LBRACE { 155 | // Definitely a definition, finish reading and return 156 | ret.definition = true 157 | for { 158 | e, terr := p.ParseElement(false) 159 | if terr != nil { 160 | err = terr 161 | return 162 | } 163 | // This means we are done 164 | if e == nil { 165 | err = nil 166 | return 167 | } 168 | ret.Contents = append(ret.Contents, e) 169 | } 170 | } 171 | 172 | // At this point, we know we have a declaration 173 | ret.definition = false 174 | 175 | // Check to see if it has a value 176 | if t.Type == T_EQUALS { 177 | // If we already parsed a string, then we shouldn't find an '=', we should 178 | // find a ';' 179 | if foundString { 180 | err = t.Expected(";") 181 | return 182 | } 183 | 184 | // Otherwise, parse the expression 185 | ret.Value, err = p.ParseExpr(false) 186 | if err != nil { 187 | return 188 | } 189 | 190 | // Grab the next token 191 | t, err = p.nextNonWhiteToken() 192 | if err != nil { 193 | return 194 | } 195 | 196 | // Check to see if there is a description after the value 197 | if t.Type == T_QUOTE { 198 | ret.Description, err = p.ParseString() 199 | if err != nil { 200 | return 201 | } 202 | 203 | t, err = p.nextNonWhiteToken() 204 | if err != nil { 205 | return 206 | } 207 | } 208 | } 209 | 210 | // This should a SEMI that terminates the declaration 211 | if t.Type != T_SEMI { 212 | err = t.Expected(";") 213 | } 214 | return 215 | } 216 | 217 | func (p *Parser) ParseContents() (ElementList, error) { 218 | ret := ElementList{} 219 | for { 220 | // Parse elements until there aren't any more 221 | elem, err := p.ParseElement(false) 222 | if err != nil { 223 | return nil, err 224 | } else { 225 | ret = append(ret, elem) 226 | } 227 | } 228 | } 229 | 230 | // Silly function to work around the fact that you can't 231 | // build a simplejson.Json object from an existing interface{} 232 | func makeJson(data interface{}) *simplejson.Json { 233 | tmp := simplejson.New() 234 | tmp.Set("tmp", data) 235 | return tmp.Get("tmp") 236 | } 237 | 238 | func (p *Parser) ParseExpr(modification bool) (expr *simplejson.Json, err error) { 239 | line := p.lineNumber 240 | col := p.colNumber 241 | // Read input stream keeping track of nesting of {}s and "s. The 242 | // next unquoted ',', ')' or ';' outside of quotes and outside an 243 | // object definition is the end of the JSON string. 244 | objcount := 0 245 | arraycount := 0 246 | quote := false 247 | escaped := false 248 | empty := true 249 | 250 | w := bytes.NewBuffer([]byte{}) 251 | 252 | for { 253 | ch, _, terr := p.src.ReadRune() 254 | err = terr 255 | if err == io.EOF { 256 | err = fmt.Errorf("Reached EOF while trying to read expression at (L%d, C%d)", 257 | p.lineNumber+1, p.colNumber+1) 258 | return 259 | } 260 | if err != nil { 261 | return 262 | } 263 | 264 | l := p.lineNumber 265 | c := p.colNumber 266 | white := p.updatePosition(ch) 267 | if !white { 268 | empty = false 269 | } 270 | 271 | if quote { 272 | if escaped { 273 | escaped = false 274 | } else { 275 | if ch == '"' { 276 | quote = false 277 | } 278 | if ch == '\\' { 279 | escaped = true 280 | } 281 | } 282 | } else { 283 | if objcount == 0 && arraycount == 0 && 284 | ((white && !empty) || ch == '/' || ch == ',' || ch == ')' || ch == ';') { 285 | p.src.UnreadRune() 286 | p.lineNumber = l 287 | p.colNumber = c 288 | 289 | // First, I use Go's native json encoding 290 | var data interface{} 291 | err = json.Unmarshal(w.Bytes(), &data) 292 | if err != nil { 293 | err = fmt.Errorf("Error parsing expression starting @ (L%d, C%d): %v", 294 | line+1, col+1, err) 295 | } 296 | 297 | // Now, convert it into a simplejson.Json object for 298 | // convenient access later. I don't use the 299 | // simplejson parsing routines because they turn on 300 | // "UseNumber" which stores integers as strings. 301 | // This, in turn, messes up JSON schema 302 | // representation. 303 | expr = makeJson(data) 304 | return 305 | } 306 | if ch == '"' { 307 | quote = true 308 | } 309 | if ch == '{' { 310 | objcount++ 311 | } 312 | if ch == '[' { 313 | arraycount++ 314 | } 315 | if ch == '}' { 316 | objcount-- 317 | } 318 | if ch == ']' { 319 | arraycount-- 320 | } 321 | } 322 | w.WriteRune(ch) 323 | } 324 | } 325 | 326 | // This is called if we've already parsed a "(" after a name 327 | func (p *Parser) ParseModifications() (mods Modifications, err error) { 328 | mods = Modifications{} 329 | first := true 330 | for { 331 | // Identifier 332 | nt, terr := p.nextNonWhiteToken() 333 | if terr != nil { 334 | err = terr 335 | return 336 | } 337 | if first && nt.Type == T_RPAREN { 338 | return 339 | } 340 | if nt.Type != T_IDENTIFIER { 341 | err = UnexpectedToken{ 342 | Found: nt, 343 | Expected: "identifier", 344 | } 345 | return 346 | } 347 | 348 | // = 349 | et, terr := p.nextNonWhiteToken() 350 | if terr != nil { 351 | err = terr 352 | return 353 | } 354 | if et.Type != T_EQUALS { 355 | err = UnexpectedToken{ 356 | Found: et, 357 | Expected: "=", 358 | } 359 | return 360 | } 361 | 362 | expr, terr := p.ParseExpr(true) 363 | if terr != nil { 364 | err = terr 365 | return 366 | } 367 | 368 | // , or ) 369 | tt, terr := p.nextNonWhiteToken() 370 | if terr != nil { 371 | err = terr 372 | return 373 | } 374 | if tt.Type != T_COMMA && tt.Type != T_RPAREN { 375 | err = UnexpectedToken{ 376 | Found: tt, 377 | Expected: ") or ,", 378 | } 379 | return 380 | } 381 | 382 | mods[nt.String] = expr 383 | first = false 384 | if tt.Type == T_RPAREN { 385 | break 386 | } 387 | } 388 | return 389 | } 390 | 391 | // Called when we've already read a '"'...read until we get to the closing '"' 392 | func (p *Parser) ParseString() (ret string, err error) { 393 | runes := []rune{} 394 | for { 395 | ch, _, terr := p.src.ReadRune() 396 | if terr != nil { 397 | err = terr 398 | return 399 | } 400 | p.updatePosition(ch) 401 | 402 | if ch == '"' { 403 | break 404 | } 405 | 406 | runes = append(runes, ch) 407 | } 408 | ret = string(runes) 409 | return 410 | } 411 | 412 | func (p *Parser) nextNonWhiteToken() (t Token, err error) { 413 | for { 414 | t, err = p.nextToken() 415 | if err != nil { 416 | return 417 | } 418 | // Ignore white space and comments 419 | if t.Type != T_WHITE && t.Type != T_SLCOMMENT && t.Type != T_MLCOMMENT { 420 | return 421 | } 422 | } 423 | } 424 | 425 | func (p *Parser) updatePosition(ch rune) bool { 426 | switch ch { 427 | case '\n': 428 | p.colNumber = 0 429 | p.lineNumber++ 430 | return true 431 | case '\t': 432 | p.colNumber += 4 // Assume tabs as four spaces 433 | return true 434 | case ' ': 435 | p.colNumber++ 436 | return true 437 | case '\r': 438 | p.colNumber = 0 439 | return true 440 | default: 441 | p.colNumber++ 442 | } 443 | return false 444 | } 445 | 446 | func (p *Parser) peek() (r rune, err error) { 447 | r, _, err = p.src.ReadRune() 448 | p.src.UnreadRune() 449 | if err != nil { 450 | return 451 | } 452 | return 453 | } 454 | 455 | func (p *Parser) nextToken() (t Token, err error) { 456 | // Record line number and column number at the start of this token 457 | line := p.lineNumber 458 | col := p.colNumber 459 | 460 | // Read the first character of the token 461 | ch, _, err := p.src.ReadRune() 462 | if err == io.EOF { 463 | t = Token{Type: T_EOF, String: "", Line: line, Column: col, File: p.file} 464 | err = nil 465 | return 466 | } 467 | if err != nil { 468 | return 469 | } 470 | 471 | // Assume this isn't white space 472 | white := p.updatePosition(ch) 473 | 474 | switch ch { 475 | case '/': 476 | nch, perr := p.peek() 477 | // Check if the next character is also a '/' 478 | if perr == nil && nch == '/' { 479 | // If so, read until end of line 480 | comment := []rune{} 481 | for { 482 | nch, _, err = p.src.ReadRune() 483 | if err != nil && err != io.EOF { 484 | return 485 | } 486 | p.updatePosition(nch) 487 | if err == io.EOF || nch == '\n' { 488 | t = Token{ 489 | Type: T_SLCOMMENT, 490 | String: string(comment), 491 | Line: line, 492 | Column: col, 493 | File: p.file} 494 | err = nil 495 | return 496 | } 497 | comment = append(comment, nch) 498 | } 499 | } 500 | // Check if the next character is also a '*' 501 | if perr == nil && nch == '*' { 502 | // If so, read until matching */ 503 | comment := []rune{'/'} 504 | star := false 505 | for { 506 | nch, _, err = p.src.ReadRune() 507 | if err != nil { 508 | err = fmt.Errorf("Error while reading multi-line comment: %v", err) 509 | return 510 | } 511 | p.updatePosition(nch) 512 | comment = append(comment, nch) 513 | if nch == '/' && star { 514 | t = Token{ 515 | Type: T_MLCOMMENT, 516 | String: string(comment), 517 | Line: line, 518 | Column: col, 519 | File: p.file} 520 | return 521 | } 522 | if nch == '*' { 523 | star = true 524 | } else { 525 | star = false 526 | } 527 | } 528 | } 529 | case '{': 530 | t = Token{Type: T_LBRACE, String: "{", Line: line, Column: col, File: p.file} 531 | return 532 | case '}': 533 | t = Token{Type: T_RBRACE, String: "}", Line: line, Column: col, File: p.file} 534 | return 535 | case '(': 536 | t = Token{Type: T_LPAREN, String: "(", Line: line, Column: col, File: p.file} 537 | return 538 | case ')': 539 | t = Token{Type: T_RPAREN, String: ")", Line: line, Column: col, File: p.file} 540 | return 541 | case '"': 542 | t = Token{Type: T_QUOTE, String: "\"", Line: line, Column: col, File: p.file} 543 | return 544 | case '=': 545 | t = Token{Type: T_EQUALS, String: "=", Line: line, Column: col, File: p.file} 546 | return 547 | case ';': 548 | t = Token{Type: T_SEMI, String: ";", Line: line, Column: col, File: p.file} 549 | return 550 | case ',': 551 | t = Token{Type: T_COMMA, String: ",", Line: line, Column: col, File: p.file} 552 | return 553 | } 554 | 555 | if white { 556 | // If this was white space, keep reading white space 557 | for { 558 | ch, _, err = p.src.ReadRune() 559 | if err == io.EOF { 560 | t = Token{Type: T_WHITE, String: " ", Line: line, Column: col, File: p.file} 561 | err = nil 562 | return 563 | } 564 | if err != nil { 565 | return 566 | } 567 | 568 | l := p.lineNumber 569 | c := p.colNumber 570 | // If not white space, we are done 571 | if !p.updatePosition(ch) { 572 | p.src.UnreadRune() 573 | p.lineNumber = l 574 | p.colNumber = c 575 | t = Token{Type: T_WHITE, String: " ", Line: line, Column: col, File: p.file} 576 | return 577 | } 578 | } 579 | } else { 580 | ret := []rune{ch} 581 | // If not white space, this is the start of an identifier 582 | // read until we hit a special character 583 | for { 584 | ch, _, err = p.src.ReadRune() 585 | if err == io.EOF { 586 | t = Token{Type: T_IDENTIFIER, String: string(ret), Line: line, Column: col, File: p.file} 587 | err = nil 588 | return 589 | } 590 | if err != nil { 591 | return 592 | } 593 | 594 | l := p.lineNumber 595 | c := p.colNumber 596 | 597 | if p.updatePosition(ch) || ch == '=' || ch == '/' || 598 | ch == '{' || ch == '}' || ch == '(' || 599 | ch == ')' || ch == ',' || ch == ';' { 600 | p.src.UnreadRune() 601 | p.lineNumber = l 602 | p.colNumber = c 603 | t = Token{Type: T_IDENTIFIER, String: string(ret), Line: line, Column: col, File: p.file} 604 | return 605 | } 606 | ret = append(ret, ch) 607 | } 608 | } 609 | } 610 | -------------------------------------------------------------------------------- /element.go: -------------------------------------------------------------------------------- 1 | package denada 2 | 3 | import "fmt" 4 | 5 | import "github.com/bitly/go-simplejson" 6 | 7 | type Modifications map[string]*simplejson.Json 8 | 9 | type Element struct { 10 | /* Common to all elements */ 11 | Qualifiers []string 12 | Name string 13 | Description string 14 | Modifications Modifications 15 | Contents ElementList // Used by definitions 16 | Value *simplejson.Json // Used by declarations 17 | 18 | rulepath string 19 | rule string 20 | definition bool 21 | } 22 | 23 | func NewDefinition(name string, desc string, qualifiers ...string) *Element { 24 | return &Element{ 25 | Qualifiers: qualifiers, 26 | Name: name, 27 | Description: desc, 28 | Modifications: Modifications{}, 29 | Contents: ElementList{}, 30 | Value: nil, 31 | definition: true, 32 | } 33 | } 34 | 35 | func NewDeclaration(name string, desc string, 36 | qualifiers ...string) *Element { 37 | return &Element{ 38 | Qualifiers: qualifiers, 39 | Name: name, 40 | Description: desc, 41 | Modifications: Modifications{}, 42 | Contents: ElementList{}, 43 | Value: nil, 44 | definition: false, 45 | } 46 | } 47 | 48 | // This checks whether a given element has EXACTLY the listed qualifiers (in the exact order) 49 | func (e Element) HasQualifiers(quals ...string) bool { 50 | if len(quals) != len(e.Qualifiers) { 51 | return false 52 | } 53 | for i, q := range e.Qualifiers { 54 | if quals[i] != q { 55 | return false 56 | } 57 | } 58 | return true 59 | } 60 | 61 | func (e Element) Unparse(rules bool) string { 62 | return UnparseElement(e, rules) 63 | } 64 | 65 | func (e Element) Clone() *Element { 66 | // TODO: Clone modifications and qualifiers 67 | 68 | children := []*Element{} 69 | if e.Contents != nil { 70 | children = append(children, e.Contents...) 71 | } else { 72 | children = nil 73 | } 74 | 75 | return &Element{ 76 | Qualifiers: e.Qualifiers, 77 | Name: e.Name, 78 | Description: e.Description, 79 | Modifications: e.Modifications, 80 | Contents: children, 81 | Value: e.Value, 82 | rule: e.rule, 83 | definition: e.definition, 84 | } 85 | } 86 | 87 | func (e Element) String() string { 88 | ret := "" 89 | for _, q := range e.Qualifiers { 90 | ret += q + " " 91 | } 92 | ret += e.Name 93 | 94 | if e.IsDefinition() { 95 | return fmt.Sprintf("%s { ... }", ret) 96 | } else { 97 | if e.Value != nil { 98 | return fmt.Sprintf("%s = %v;", ret, e.Value) 99 | } else { 100 | return fmt.Sprintf("%s;", ret) 101 | } 102 | } 103 | } 104 | 105 | func (e Element) Rule() string { 106 | return e.rule 107 | } 108 | 109 | func (e Element) RulePath() string { 110 | return e.rulepath 111 | } 112 | 113 | func (e Element) IsDefinition() bool { 114 | return e.definition 115 | } 116 | 117 | func (e Element) IsDeclaration() bool { 118 | return !e.definition 119 | } 120 | 121 | func equalValues(l *simplejson.Json, r *simplejson.Json) (bool, error) { 122 | lbytes, err := l.Encode() 123 | if err != nil { 124 | return false, err 125 | } 126 | rbytes, err := r.Encode() 127 | if err != nil { 128 | return false, err 129 | } 130 | return string(lbytes) == string(rbytes), nil 131 | } 132 | 133 | func (e *Element) Append(children ...*Element) error { 134 | if e.IsDefinition() { 135 | e.Contents = append(e.Contents, children...) 136 | return nil 137 | } else { 138 | return fmt.Errorf("Attempted to append elements to a declaration") 139 | } 140 | } 141 | 142 | func (e Element) Equals(o Element) error { 143 | // Check that they have the same number of qualifiers 144 | if len(e.Qualifiers) != len(o.Qualifiers) { 145 | return fmt.Errorf("Length mismatch (%d vs. %d)", 146 | len(e.Qualifiers), len(o.Qualifiers)) 147 | } 148 | 149 | // Then check that each qualifier is identical (and in identical order) 150 | for i, q := range e.Qualifiers { 151 | if q != o.Qualifiers[i] { 152 | return fmt.Errorf("Qualifier mismatch: %s vs %s", q, o.Qualifiers[i]) 153 | } 154 | } 155 | 156 | // Next, check that they have the same name 157 | if e.Name != o.Name { 158 | return fmt.Errorf("Name mismatch: %s vs %s", e.Name, o.Name) 159 | } 160 | 161 | // And then the same description 162 | if e.Description != o.Description { 163 | return fmt.Errorf("Description mismatch: %s vs %s", e.Description, o.Description) 164 | } 165 | 166 | // Now we check the modifications to make sure that all keys match and that the 167 | // value for each key is identical between both sets of modifications 168 | for k, v := range e.Modifications { 169 | ov, exists := o.Modifications[k] 170 | if !exists { 171 | return fmt.Errorf("Mismatch in modification for key %s missing from argument", k) 172 | } 173 | eq, err := equalValues(v, ov) 174 | if err != nil { 175 | return err 176 | } 177 | if !eq { 178 | return fmt.Errorf("Mismatch in value for key %s: %v vs %v", k, v, ov) 179 | } 180 | } 181 | for k, _ := range o.Modifications { 182 | _, exists := e.Modifications[k] 183 | if !exists { 184 | return fmt.Errorf("Mismatch in modification for key %s missing from object", k) 185 | } 186 | } 187 | 188 | err := e.Contents.Equals(o.Contents) 189 | if err != nil { 190 | return fmt.Errorf("Error in child elements comparing %v with %v: %v", 191 | e, o, err) 192 | } 193 | 194 | if e.Value != nil && o.Value != nil { 195 | // If they both have values, make sure they are equal 196 | eq, err := equalValues(e.Value, o.Value) 197 | if err != nil { 198 | return err 199 | } 200 | if !eq { 201 | return fmt.Errorf("Mismatch in values: %v vs %v", e.Value, o.Value) 202 | } 203 | } else { 204 | // If they don't both have values, then make sure that both are nil 205 | if e.Value != nil || o.Value != nil { 206 | return fmt.Errorf("Mismatch in value (one has a value, the other doesn't") 207 | } 208 | } 209 | // If we get here, nothing was unequal 210 | return nil 211 | } 212 | 213 | func (e *Element) SetModification(key string, data interface{}) { 214 | e.Modifications[key] = makeJson(data) 215 | } 216 | 217 | func (e *Element) SetValue(data interface{}) *simplejson.Json { 218 | val := makeJson(data) 219 | e.Value = val 220 | return val 221 | } 222 | 223 | func (e *Element) StringValueOf(defval string) string { 224 | if e == nil { 225 | return defval 226 | } 227 | if e.Value == nil { 228 | return defval 229 | } 230 | s, err := e.Value.String() 231 | if err != nil { 232 | return defval 233 | } 234 | return s 235 | } 236 | 237 | func (e *Element) FirstNamed(name string) *Element { 238 | if e.definition { 239 | return e.Contents.FirstNamed(name) 240 | } else { 241 | return nil 242 | } 243 | } 244 | -------------------------------------------------------------------------------- /elist.go: -------------------------------------------------------------------------------- 1 | package denada 2 | 3 | import ( 4 | "fmt" 5 | "github.com/bitly/go-simplejson" 6 | ) 7 | 8 | type ElementList []*Element 9 | 10 | func (e ElementList) Definition(name string, children ...string) (*Element, error) { 11 | for _, d := range e { 12 | if d.IsDefinition() && d.Name == name { 13 | if len(children) == 0 { 14 | return d, nil 15 | } else { 16 | return d.Contents.Definition(children[0], children[1:]...) 17 | } 18 | } 19 | } 20 | return nil, fmt.Errorf("Unable to find definition for %s", name) 21 | } 22 | 23 | func (e ElementList) Definitions() ElementList { 24 | ret := ElementList{} 25 | for _, elem := range e { 26 | if elem.IsDefinition() { 27 | ret = append(ret, elem) 28 | } 29 | } 30 | return ret 31 | } 32 | 33 | func (e ElementList) Declarations() ElementList { 34 | ret := ElementList{} 35 | for _, elem := range e { 36 | if elem.IsDeclaration() { 37 | ret = append(ret, elem) 38 | } 39 | } 40 | return ret 41 | } 42 | 43 | func (e ElementList) FirstNamed(name string) *Element { 44 | for _, elem := range e { 45 | if elem.Name == name { 46 | return elem 47 | } 48 | } 49 | return nil 50 | } 51 | 52 | func (e ElementList) QualifiedWith(name ...string) ElementList { 53 | ret := ElementList{} 54 | for _, elem := range e { 55 | if elem.HasQualifiers(name...) { 56 | ret = append(ret, elem) 57 | } 58 | } 59 | return ret 60 | } 61 | 62 | func (e ElementList) OfRule(name string, fqn bool) ElementList { 63 | ret := ElementList{} 64 | for _, elem := range e { 65 | if fqn { 66 | if elem.rulepath == name { 67 | ret = append(ret, elem) 68 | } 69 | } else { 70 | if elem.rule == name { 71 | ret = append(ret, elem) 72 | } 73 | } 74 | } 75 | return ret 76 | } 77 | 78 | func (e ElementList) AllElements() ElementList { 79 | ret := ElementList{} 80 | for _, elem := range e { 81 | ret = append(ret, elem) 82 | if elem.IsDefinition() { 83 | ret = append(ret, elem.Contents.AllElements()...) 84 | } 85 | } 86 | return ret 87 | } 88 | 89 | func (e ElementList) PopHead() (*Element, ElementList, error) { 90 | if len(e) == 0 { 91 | return nil, e, fmt.Errorf("Cannot pop the head of an empty element list") 92 | } 93 | ret := e[0] 94 | e = e[1:] 95 | return ret, e, nil 96 | } 97 | 98 | func (e ElementList) Equals(o ElementList) error { 99 | // Now make sure they have the same number of children 100 | if len(e) != len(o) { 101 | return fmt.Errorf("Mismatch in number of child elements: %d vs %d: %v vs %v", 102 | len(e), len(o), e, o) 103 | } 104 | 105 | // And that each child is equal 106 | for cn, child := range e { 107 | err := child.Equals(*o[cn]) 108 | if err != nil { 109 | return err 110 | } 111 | } 112 | return nil 113 | } 114 | 115 | // GetValue tries to find an element that matches the **fully qualified** rule name 116 | // provided. It returns nil if it cannot find a match (or it finds multiple matches), 117 | // otherwise it returns the value associated with that declaration. 118 | func (e ElementList) GetValue(rulename string) *simplejson.Json { 119 | elems := e.AllElements().OfRule(rulename, true) 120 | if len(elems) != 1 { 121 | return nil 122 | } 123 | return elems[0].Value 124 | } 125 | 126 | func (e ElementList) GetStringValue(rulename string, defaultValue string) string { 127 | v := e.GetValue(rulename) 128 | if v == nil { 129 | return defaultValue 130 | } 131 | ret := v.MustString() 132 | if ret == "" { 133 | return defaultValue 134 | } 135 | return ret 136 | } 137 | 138 | func (e ElementList) GetIntValue(rulename string, defaultValue int) int { 139 | v := e.GetValue(rulename) 140 | if v == nil { 141 | return defaultValue 142 | } 143 | ret, err := v.Int() 144 | if err != nil { 145 | return defaultValue 146 | } 147 | return ret 148 | } 149 | 150 | func (e ElementList) GetBoolValue(rulename string, defaultValue bool) bool { 151 | v := e.GetValue(rulename) 152 | if v == nil { 153 | return defaultValue 154 | } 155 | ret, err := v.Bool() 156 | if err != nil { 157 | return defaultValue 158 | } 159 | return ret 160 | } 161 | 162 | func MakeElementList() ElementList { 163 | return ElementList{} 164 | } 165 | -------------------------------------------------------------------------------- /grammar_test.go: -------------------------------------------------------------------------------- 1 | package denada 2 | 3 | import "testing" 4 | import . "github.com/onsi/gomega" 5 | 6 | func Test_QualifierMatch(t *testing.T) { 7 | RegisterTestingT(t) 8 | 9 | g := Element{Qualifiers: []string{"set"}, Name: "_", Description: "foo*", definition: false} 10 | i := Element{Qualifiers: []string{"var"}, Name: "x", definition: false} 11 | 12 | m := matchQualifiers(&i, &g) 13 | Expect(m).To(Equal(false)) 14 | 15 | gl := ElementList{&g} 16 | il := ElementList{&i} 17 | 18 | ml := Check(il, gl, false) 19 | Expect(ml).ToNot(BeNil()) 20 | } 21 | 22 | func Test_StringMatch(t *testing.T) { 23 | RegisterTestingT(t) 24 | 25 | match := matchString("abc", "abc") 26 | Expect(match).To(BeTrue()) 27 | match = matchString("abcabc", "(abc)+") 28 | Expect(match).To(BeTrue()) 29 | match = matchString("abc", "_") 30 | Expect(match).To(BeTrue()) 31 | match = matchString("abc", ".+") 32 | Expect(match).To(BeTrue()) 33 | 34 | match = matchString("abc", "def") 35 | Expect(match).To(BeFalse()) 36 | match = matchString("_", "abc") 37 | Expect(match).To(BeFalse()) 38 | } 39 | -------------------------------------------------------------------------------- /import_test.go: -------------------------------------------------------------------------------- 1 | package denada 2 | 3 | import "testing" 4 | import . "github.com/onsi/gomega" 5 | 6 | var importTest = ` 7 | import(file="testsuite/case1.dnd", recursive=false); 8 | 9 | import(file="testsuite/case1.dnd"); 10 | 11 | scoped { 12 | import(file="testsuite/case2.dnd", recursive=true); 13 | } 14 | ` 15 | 16 | var expectedResult = ` 17 | // Most basic syntactic example with just 2 declarations 18 | 19 | props(declarations=2); 20 | 21 | Real r; 22 | 23 | // Most basic syntactic example with just 2 declarations 24 | 25 | props(declarations=2); 26 | 27 | Real r; 28 | 29 | scoped { 30 | // Checking against a grammar file with different expression types 31 | 32 | props(grammar="config.grm", definitions=1, declarations=1) "props"; 33 | 34 | section Foo "section" { 35 | x = 1 "section.variable"; 36 | y = 1.0 "section.variable"; 37 | z = "test string" "section.variable"; 38 | json = {"this": "is a JSON expression!"} "section.variable"; 39 | } 40 | } 41 | ` 42 | 43 | func Test_NonRecursiveImport(t *testing.T) { 44 | RegisterTestingT(t) 45 | 46 | exp, err := ParseString(expectedResult) 47 | Expect(err).To(BeNil()) 48 | 49 | raw, err := ParseString(importTest) 50 | Expect(err).To(BeNil()) 51 | 52 | elab, err := ImportTransform(raw) 53 | Expect(err).To(BeNil()) 54 | 55 | eq := elab.Equals(exp) 56 | Expect(eq).To(BeNil()) 57 | } 58 | -------------------------------------------------------------------------------- /marshal.go: -------------------------------------------------------------------------------- 1 | package denada 2 | 3 | import "log" 4 | import "fmt" 5 | import "strings" 6 | import "reflect" 7 | 8 | func Marshal(data interface{}) (ElementList, error) { 9 | ret := []*Element{} 10 | typ := reflect.TypeOf(data) 11 | log.Printf("Type: %v", typ) 12 | for i := 0; i < typ.NumField(); i++ { 13 | f := typ.Field(i) 14 | log.Printf(" Field %d: %v", i, f) 15 | e, err := marshalField(f) 16 | if err != nil { 17 | err = fmt.Errorf("Error marshalling field #%d: %v", i, err) 18 | return nil, err 19 | } 20 | ret = append(ret, e) 21 | } 22 | 23 | return ret, nil 24 | } 25 | 26 | func marshalField(f reflect.StructField) (elem *Element, err error) { 27 | rule := f.Tag.Get("dndrule") 28 | if rule == "" { 29 | return nil, fmt.Errorf("Field %s has no dndrule field", f.Name) 30 | } 31 | _, err = ParseRuleName(rule) 32 | if err != nil { 33 | return nil, fmt.Errorf("Invalid rule '%s' associated with field %s", 34 | rule, f.Name) 35 | } 36 | 37 | quals := f.Tag.Get("dndquals") 38 | log.Printf("quals = %s", quals) 39 | 40 | return &Element{ 41 | Qualifiers: strings.Split(quals, " "), 42 | Name: "_", 43 | Description: rule, 44 | }, nil 45 | } 46 | -------------------------------------------------------------------------------- /marshal_test.go: -------------------------------------------------------------------------------- 1 | package denada 2 | 3 | import "fmt" 4 | import "testing" 5 | import . "github.com/onsi/gomega" 6 | 7 | type Case1 struct { 8 | //X string 9 | Reals []struct { 10 | Named string `dnd:"name=_"` 11 | Label string `dnd:"mod"` 12 | Units string `dnd:"mod"` 13 | Value int `dnd:"value"` 14 | } `dndrule:"Real*" dndquals:"Real"` 15 | Groups []struct { 16 | Named string `dnd:"name"` 17 | Label string `dnd:"mod"` 18 | Image string `dnd:"mod"` 19 | Contents []Case1 `dnd:"contents"` 20 | } `dndrule:"Groups*" dndquals:"group"` 21 | } 22 | 23 | type DenadaFormat interface { 24 | Grammar() ElementList 25 | Unmarshal(elems ElementList) error 26 | } 27 | 28 | /* 29 | type NamedStringVar struct { 30 | Name string 31 | Value string 32 | } 33 | 34 | func (n NamedStringVar) Grammar() Element { 35 | return Element{ 36 | Name: n.Name, 37 | Value: "$string", 38 | } 39 | } 40 | 41 | type Case2 struct { 42 | project NamedStringVar 43 | processes []NamedStringVar 44 | } 45 | */ 46 | 47 | func Test_MarshalCase1(t *testing.T) { 48 | RegisterTestingT(t) 49 | 50 | c := Case1{} 51 | elems, err := Marshal(c) 52 | fmt.Printf("%s\n", Unparse(elems, false)) 53 | Expect(err).To(BeNil()) 54 | Expect(len(elems)).To(Equal(2)) 55 | } 56 | -------------------------------------------------------------------------------- /parser_test.go: -------------------------------------------------------------------------------- 1 | package denada 2 | 3 | import "testing" 4 | import "strings" 5 | import "encoding/json" 6 | 7 | import . "github.com/onsi/gomega" 8 | 9 | var sample_noexprs = ` 10 | class ABC() "D1" { 11 | Real foo; 12 | Integer x; 13 | } 14 | 15 | class DEF "D2" { 16 | String y(); 17 | Boolean x "bool"; 18 | } 19 | ` 20 | 21 | var sample = ` 22 | printer 'ABC' { 23 | set location = "Mike's desk"; 24 | set model = "HP 8860"; 25 | } 26 | 27 | 'printer' DEF { 28 | set location = "Coffee machine"; 29 | set model = "HP 8860"; 30 | set networkName = "PrinterDEF"; 31 | } 32 | 33 | computer XYZ { 34 | set location = "Mike's desk"; 35 | set 'model' = "Mac Book Air"; 36 | } 37 | ` 38 | 39 | var sample_exprs = ` 40 | Real x = 5.0; 41 | Integer y = 1; 42 | String z = "This is a \"test\""; 43 | Object a = {"key1": 5, "\"test\"": 2, "nested": {"r": "another string"}}; 44 | Null b = null; 45 | Boolean c = [true, false]; 46 | Array d = [{"x": 5}, "foo", "\"test\"", true]; 47 | class Foo(x=5.0, y=1, z="This is a \"test\"", a={"key1": 5}, b=null, c=[true, false], 48 | d = [{"x": 5}, "foo"]) { 49 | Real x = 5.0; 50 | } 51 | ` 52 | 53 | func Test_SimpleDeclaration(t *testing.T) { 54 | RegisterTestingT(t) 55 | 56 | r := strings.NewReader("set x = 5 \"Description\";") 57 | elems, err := Parse(r) 58 | 59 | Expect(err).To(BeNil()) 60 | Expect(len(elems)).To(Equal(1)) 61 | 62 | elem := elems[0] 63 | 64 | Expect(elem.IsDeclaration()).To(BeTrue()) 65 | Expect(elem.IsDefinition()).To(BeFalse()) 66 | Expect(len(elem.Modifications)).To(Equal(0)) 67 | 68 | Expect(elem.Qualifiers).To(Equal([]string{"set"})) 69 | Expect(elem.Name).To(Equal("x")) 70 | Expect(elem.Description).To(Equal("Description")) 71 | v, err := elem.Value.Int() 72 | Expect(err).To(BeNil()) 73 | Expect(v).To(Equal(5)) 74 | } 75 | 76 | func Test_Errors(t *testing.T) { 77 | RegisterTestingT(t) 78 | r := strings.NewReader("set x = 5") 79 | 80 | _, err := Parse(r) 81 | 82 | Expect(err).ToNot(BeNil()) 83 | } 84 | 85 | func Test_SampleInput(t *testing.T) { 86 | RegisterTestingT(t) 87 | r := strings.NewReader(sample) 88 | 89 | el, err := Parse(r) 90 | 91 | Expect(err).To(BeNil()) 92 | 93 | Expect(len(el)).To(Equal(3)) 94 | Expect(el[0].IsDefinition()).To(BeTrue()) 95 | Expect(el[1].IsDefinition()).To(BeTrue()) 96 | Expect(el[2].IsDefinition()).To(BeTrue()) 97 | } 98 | 99 | func Test_SampleNoExprInput(t *testing.T) { 100 | RegisterTestingT(t) 101 | r := strings.NewReader(sample_noexprs) 102 | 103 | el, err := Parse(r) 104 | 105 | Expect(err).To(BeNil()) 106 | 107 | Expect(len(el)).To(Equal(2)) 108 | Expect(el[0].IsDefinition()).To(BeTrue()) 109 | Expect(el[1].IsDefinition()).To(BeTrue()) 110 | } 111 | 112 | func Test_JsonTypes(t *testing.T) { 113 | var expr interface{} 114 | str := `{"minItems": 1}` 115 | err := json.Unmarshal([]byte(str), &expr) 116 | Expect(err).To(BeNil()) 117 | asmap, ok := expr.(map[string]interface{}) 118 | Expect(ok).To(Equal(true)) 119 | v, exists := asmap["minItems"] 120 | Expect(exists).To(Equal(true)) 121 | Expect(v).To(Equal(1.0)) 122 | } 123 | 124 | func Test_NumbersInExpr(t *testing.T) { 125 | elems, err := ParseString(`var x = {"minItems": 1 };`) 126 | Expect(err).To(BeNil()) 127 | e := elems[0] 128 | v := e.Value 129 | 130 | asmap, err := v.Map() 131 | Expect(err).To(BeNil()) 132 | 133 | mif, exists := asmap["minItems"] 134 | Expect(exists).To(Equal(true)) 135 | Expect(mif).To(Equal(1.0)) 136 | 137 | mi := v.Get("minItems").MustInt() 138 | Expect(mi).To(Equal(1)) 139 | } 140 | 141 | func Test_SampleJSONInput(t *testing.T) { 142 | RegisterTestingT(t) 143 | r := strings.NewReader(sample_exprs) 144 | 145 | el, err := Parse(r) 146 | 147 | Expect(err).To(BeNil()) 148 | 149 | Expect(len(el)).To(Equal(8)) 150 | for i, e := range el { 151 | if i == 7 { 152 | Expect(e.IsDefinition()).To(BeTrue()) 153 | } else { 154 | Expect(e.IsDefinition()).To(BeFalse()) 155 | } 156 | } 157 | } 158 | -------------------------------------------------------------------------------- /rules.go: -------------------------------------------------------------------------------- 1 | package denada 2 | 3 | import "fmt" 4 | import "strings" 5 | 6 | type Cardinality int 7 | 8 | const ( 9 | Zero = iota 10 | Optional 11 | ZeroOrMore 12 | Singleton 13 | OneOrMore 14 | ) 15 | 16 | type RuleInfo struct { 17 | ContextPath []string 18 | Context RuleContext 19 | Name string 20 | Cardinality Cardinality 21 | } 22 | 23 | func (r RuleInfo) checkCount(count int) error { 24 | switch r.Cardinality { 25 | case Zero: 26 | if count != 0 { 27 | return fmt.Errorf("Expected zero of rule %s, found %d", r.Name, count) 28 | } 29 | case Optional: 30 | if count > 1 { 31 | return fmt.Errorf("Expected at most 1 of rule %s, found %d", r.Name, count) 32 | } 33 | case Singleton: 34 | if count != 1 { 35 | return fmt.Errorf("Expected at exactly 1 of rule %s, found %d", r.Name, count) 36 | } 37 | case OneOrMore: 38 | if count == 0 { 39 | return fmt.Errorf("Expected at least 1 of rule %s, found %d", r.Name, count) 40 | } 41 | } 42 | return nil 43 | } 44 | 45 | func ParseRuleName(desc string) (rule RuleInfo, err error) { 46 | return ParseRule(desc, NullContext()) 47 | } 48 | 49 | func ParseRule(desc string, context RuleContext) (rule RuleInfo, err error) { 50 | rule = RuleInfo{Cardinality: Zero} 51 | 52 | // Note a rule is of the form "myrule>childrule". If no ">" is present, 53 | // child rules are assumed to be indicated by the "contents" of the current 54 | // rule. 55 | 56 | parts := strings.Split(desc, ">") 57 | str := desc 58 | path := []string{"."} 59 | 60 | if len(parts) == 0 { 61 | err = fmt.Errorf("Empty rule string") 62 | return 63 | } else if len(parts) == 2 { 64 | str = parts[0] 65 | path = strings.Split(parts[1], "/") 66 | } else if len(parts) > 2 { 67 | err = fmt.Errorf("Rule contains multiple child rule indicators (>)") 68 | return 69 | } 70 | 71 | // Shorthand notation 72 | if str[0] == '^' { 73 | path = []string{".."} 74 | str = str[1:] 75 | } 76 | 77 | rctxt, ferr := context.Find(path...) 78 | if ferr != nil { 79 | err = fmt.Errorf("Error finding %v: %v", path, ferr) 80 | return 81 | } 82 | rule.Context = rctxt 83 | rule.ContextPath = path 84 | 85 | l := len(str) - 1 86 | lastchar := str[l] 87 | if lastchar == '-' { 88 | rule.Cardinality = Zero 89 | str = str[0:l] 90 | } else if lastchar == '+' { 91 | rule.Cardinality = OneOrMore 92 | str = str[0:l] 93 | } else if lastchar == '*' { 94 | rule.Cardinality = ZeroOrMore 95 | str = str[0:l] 96 | } else if lastchar == '?' { 97 | rule.Cardinality = Optional 98 | str = str[0:l] 99 | } else { 100 | rule.Cardinality = Singleton 101 | } 102 | rule.Name = str 103 | 104 | return 105 | } 106 | -------------------------------------------------------------------------------- /rules_test.go: -------------------------------------------------------------------------------- 1 | package denada 2 | 3 | import "log" 4 | import "testing" 5 | import . "github.com/smartystreets/goconvey/convey" 6 | 7 | func TestSingularRule(t *testing.T) { 8 | Convey("Testing Singular Rule", t, func() { 9 | info, err := ParseRuleName("singleton") 10 | So(err, ShouldBeNil) 11 | So(info.Context.This, ShouldResemble, ElementList{}) 12 | So(info.Name, ShouldEqual, "singleton") 13 | So(info.Cardinality, ShouldEqual, Cardinality(Singleton)) 14 | }) 15 | } 16 | 17 | func TestOptionalRule(t *testing.T) { 18 | Convey("Testing Optional Rule", t, func() { 19 | info, err := ParseRuleName("optional?") 20 | So(err, ShouldBeNil) 21 | So(info.Context.This, ShouldResemble, ElementList{}) 22 | So(info.Name, ShouldEqual, "optional") 23 | So(info.Cardinality, ShouldEqual, Cardinality(Optional)) 24 | }) 25 | } 26 | 27 | func TestZoMRule(t *testing.T) { 28 | Convey("Testing Zero-Or-More Rule", t, func() { 29 | info, err := ParseRuleName("zom*") 30 | So(err, ShouldBeNil) 31 | So(info.Context.This, ShouldResemble, ElementList{}) 32 | So(info.Name, ShouldEqual, "zom") 33 | So(info.Cardinality, ShouldEqual, Cardinality(ZeroOrMore)) 34 | }) 35 | } 36 | 37 | func TestOoMRule(t *testing.T) { 38 | Convey("Testing One-Or-More Rule", t, func() { 39 | info, err := ParseRuleName("oom+") 40 | So(err, ShouldBeNil) 41 | So(info.Context.This, ShouldResemble, ElementList{}) 42 | So(info.Name, ShouldEqual, "oom") 43 | So(info.Cardinality, ShouldEqual, Cardinality(OneOrMore)) 44 | }) 45 | } 46 | 47 | func TestRecursiveRule(t *testing.T) { 48 | Convey("Testing Recursive Rule", t, func() { 49 | dummy := NewDeclaration("dummy", "dummy*") 50 | root := ElementList{dummy} 51 | context := RootContext(root) 52 | 53 | info, err := ParseRule("recur>$root", context) 54 | So(err, ShouldBeNil) 55 | So(info.Context.This, ShouldResemble, root) 56 | So(info.Name, ShouldEqual, "recur") 57 | So(info.Cardinality, ShouldEqual, Cardinality(Singleton)) 58 | }) 59 | } 60 | 61 | func TestParentRule(t *testing.T) { 62 | Convey("Testing Parent Rule", t, func() { 63 | dummy := NewDeclaration("dummy", "dummy*") 64 | root := ElementList{dummy} 65 | context := RootContext(root) 66 | 67 | Convey("Check that root scope has no parent", func() { 68 | log.Printf("TestParentRule.context = %v", context) 69 | _, err := ParseRule("recur>..", context) 70 | So(err, ShouldNotBeNil) 71 | }) 72 | 73 | child := ChildContext(ElementList{}, &context) 74 | 75 | Convey("Check that child scope has a parent", func() { 76 | info, err := ParseRule("recur>..", child) 77 | So(err, ShouldBeNil) 78 | So(info.Context.This, ShouldResemble, root) 79 | So(info.Name, ShouldEqual, "recur") 80 | So(info.Cardinality, ShouldEqual, Cardinality(Singleton)) 81 | }) 82 | 83 | Convey("Check that parent of child of root is root", func() { 84 | info, err := ParseRule("recur>..", child) 85 | So(err, ShouldBeNil) 86 | rinfo, err := ParseRule("recur>..", child) 87 | So(err, ShouldBeNil) 88 | So(info.Context, ShouldResemble, rinfo.Context) 89 | }) 90 | }) 91 | } 92 | 93 | func TestCurrentRule(t *testing.T) { 94 | Convey("Testing Current Rule", t, func() { 95 | dummy := NewDeclaration("dummy", "dummy*") 96 | root := ElementList{dummy} 97 | context := RootContext(root) 98 | 99 | info, err := ParseRule("recur>.", context) 100 | So(err, ShouldBeNil) 101 | So(info.Context.This, ShouldResemble, root) 102 | So(info.Name, ShouldEqual, "recur") 103 | So(info.Cardinality, ShouldEqual, Cardinality(Singleton)) 104 | 105 | info, err = ParseRule("recur", context) 106 | So(err, ShouldBeNil) 107 | So(info.Context.This, ShouldResemble, root) 108 | So(info.Name, ShouldEqual, "recur") 109 | So(info.Cardinality, ShouldEqual, Cardinality(Singleton)) 110 | }) 111 | } 112 | 113 | func TestRecursiveComplexRule(t *testing.T) { 114 | Convey("Testing Complex Recursive Rule", t, func() { 115 | root := ElementList{new(Element)} 116 | context := RootContext(root) 117 | 118 | info, err := ParseRule("recur?>$root", context) 119 | So(err, ShouldBeNil) 120 | So(info.Context.This, ShouldResemble, root) 121 | So(info.Name, ShouldEqual, "recur") 122 | So(info.Cardinality, ShouldEqual, Cardinality(Optional)) 123 | }) 124 | } 125 | -------------------------------------------------------------------------------- /testsuite/case1.dnd: -------------------------------------------------------------------------------- 1 | // Most basic syntactic example with just 2 declarations 2 | 3 | props(declarations=2); 4 | 5 | Real r; 6 | -------------------------------------------------------------------------------- /testsuite/case2.dnd: -------------------------------------------------------------------------------- 1 | // Checking against a grammar file with different expression types 2 | 3 | props(grammar="config.grm", definitions=1, declarations=1) "props"; 4 | 5 | section Foo "section" { 6 | x = 1 "section.variable"; 7 | y = 1.0 "section.variable"; 8 | z = "test string" "section.variable"; 9 | json = {"this": "is a JSON expression!"} "section.variable"; 10 | } 11 | -------------------------------------------------------------------------------- /testsuite/case3.dnd: -------------------------------------------------------------------------------- 1 | /* This check case is to check comment parsing */ 2 | 3 | props(declarations=3, definitions=1 /*, ignored=5 */); 4 | 5 | definition /* With a few comments */ between /* Other things */ (attr=5/* everwhere */) { 6 | } 7 | 8 | var y/* Not an identifier */ = 2; 9 | 10 | var x = 5; // This should be ignored -------------------------------------------------------------------------------- /testsuite/case4.dnd: -------------------------------------------------------------------------------- 1 | // Adds a tricky comment case and more grammar checking 2 | 3 | props(grammar="config.grm", definitions=2, declarations=1) "props"; 4 | 5 | section Authentication "section" { 6 | username = "foo" "section.variable"; 7 | password/* Not an identifier */ = "bar" "section.variable"; 8 | } 9 | 10 | section DNS "section" { 11 | hostname = "localhost" "section.variable"; 12 | MTU = 1500 "section.variable"; 13 | } 14 | -------------------------------------------------------------------------------- /testsuite/case5.dnd: -------------------------------------------------------------------------------- 1 | // Checks support for multiple rules with the same name assuming their 2 | // cardinality matches 3 | 4 | props(grammar="multi.grm", definitions=0, declarations=2) "props"; 5 | 6 | var X "variable"; 7 | -------------------------------------------------------------------------------- /testsuite/case6.dnd: -------------------------------------------------------------------------------- 1 | // Checks support for multiple rules with the same name assuming their 2 | // cardinality matches 3 | 4 | props(grammar="multi.grm", definitions=0, declarations=2) "props"; 5 | 6 | set X "variable"; 7 | -------------------------------------------------------------------------------- /testsuite/case7.dnd: -------------------------------------------------------------------------------- 1 | // Some checks against a more complex JSON Schema set of rules 2 | // 3 | 4 | props(grammar="schema.grm", definitions=1, declarations=1) "props"; 5 | 6 | section A "section" { 7 | array = ["Hello"] "section.array"; 8 | // vector = [0, 1.0, 3.0] "section.vectory"; 9 | // mod(value=["this", "that"]) "section.mod"; 10 | } 11 | 12 | -------------------------------------------------------------------------------- /testsuite/case8.dnd: -------------------------------------------------------------------------------- 1 | // Use named rules 2 | // 3 | 4 | props(grammar="reference.grm", definitions=2, declarations=2) "props"; 5 | 6 | z = "This should work" "z"; 7 | 8 | case1 "case1" { 9 | props(grammar="reference.grm", definitions=2, declarations=2) "case1.props"; // Ignored 10 | z = "Allowed because case1 is recursive" "case1.z"; 11 | case1 "case1.case1" { // Allowed because of recursion but can be empty 12 | props(grammar="reference.grm", definitions=2, declarations=2) "case1.case1.props"; // Ignored 13 | } 14 | } 15 | 16 | // This is required 17 | case2 "case2" { 18 | nested "case2.nested" { 19 | x = "Allowed" "case2.nested.x"; // This is allowed here by named reference 20 | } 21 | also "case2.also" { // Simple recursion 22 | nested "case2.also.nested" { 23 | y = "Also allowed" "case2.also.nested.y"; 24 | } 25 | } 26 | } 27 | -------------------------------------------------------------------------------- /testsuite/config.grm: -------------------------------------------------------------------------------- 1 | props(grammar?="$string", definitions?="$int", declarations?={"type": "integer"}) "props"; 2 | 3 | section _ "section*" { 4 | _ = "$_" "variable*"; 5 | } 6 | -------------------------------------------------------------------------------- /testsuite/ecase1.dnd: -------------------------------------------------------------------------------- 1 | // This is missing a semi-colon 2 | 3 | Real x -------------------------------------------------------------------------------- /testsuite/ecase2.dnd: -------------------------------------------------------------------------------- 1 | props(grammar="config.grm", definitions=2); 2 | 3 | // Error: multiple occurences of singleton qualifier 4 | section section Authentication "section" { 5 | username = "foo" "variable"; 6 | password = "bar" "variable"; 7 | } 8 | 9 | section DNS "section" { 10 | hostname = "localhost" "variable"; 11 | MTU = 1500 "variable"; 12 | } 13 | -------------------------------------------------------------------------------- /testsuite/ecase3.dnd: -------------------------------------------------------------------------------- 1 | props(grammar="config.grm", definitions=2); 2 | 3 | // Error: illegal qualifier 'extra' 4 | extra section Authentication "section" { 5 | username = "foo" "variable"; 6 | password = "bar" "variable"; 7 | } 8 | 9 | section DNS "section"{ 10 | hostname = "localhost" "variable"; 11 | MTU = 1500 "variable"; 12 | } 13 | -------------------------------------------------------------------------------- /testsuite/ecase4.dnd: -------------------------------------------------------------------------------- 1 | props(grammar="config.grm", definitions=2); 2 | 3 | section Authentication "section" { 4 | username = "foo" "variable"; 5 | password = "bar" "variable"; 6 | } 7 | 8 | section DNS "section" { 9 | // Error: unexpected qualifier 10 | var hostname = "localhost" "variable"; 11 | MTU = 1500 "variable"; 12 | } 13 | -------------------------------------------------------------------------------- /testsuite/ecase5.dnd: -------------------------------------------------------------------------------- 1 | // Test matching of input element descriptions with and matched rule names 2 | 3 | props(grammar="config.grm", definitions=2, declarations=1) "props"; 4 | 5 | section Authentication "section" { 6 | username = "foo" "variable"; 7 | password/* Not an identifier */ = "bar" "assignment"; // Description doesn't match rule name 8 | } 9 | 10 | section DNS "section" { 11 | hostname = "localhost" "variable"; 12 | MTU = 1500 "variable"; 13 | } 14 | -------------------------------------------------------------------------------- /testsuite/ecase6.dnd: -------------------------------------------------------------------------------- 1 | // Checks support for multiple rules with the same name assuming their 2 | // cardinality matches 3 | 4 | props(grammar="multi.grm", definitions=2, declarations=1) "props"; 5 | 6 | var X "variable"; 7 | set Y "variable"; // Multiple...not allowed 8 | -------------------------------------------------------------------------------- /testsuite/ecase7.dnd: -------------------------------------------------------------------------------- 1 | // Checks support for multiple rules with the same name assuming their 2 | // cardinality matches 3 | 4 | props(grammar="multi.grm", definitions=2, declarations=1) "props"; 5 | 6 | set X "variable"; 7 | set Y "variable"; // Multiple...not allowed 8 | -------------------------------------------------------------------------------- /testsuite/ecase8.dnd: -------------------------------------------------------------------------------- 1 | // Use named rules (should fail) 2 | // 3 | 4 | props(grammar="reference.grm", definitions=1, declarations=1) "props"; 5 | 6 | z = "This should work"; 7 | 8 | // This should not be allowed for cardinality reasons 9 | namedRules { 10 | x = "hello"; 11 | } 12 | 13 | case2 { 14 | } 15 | -------------------------------------------------------------------------------- /testsuite/multi.grm: -------------------------------------------------------------------------------- 1 | props(grammar?="$string", definitions?="$int", declarations?={"type": "integer"}) "props"; 2 | 3 | var _ "variable?"; 4 | set _ "variable?"; 5 | -------------------------------------------------------------------------------- /testsuite/reference.grm: -------------------------------------------------------------------------------- 1 | props(grammar?="$string", definitions?="$int", declarations?={"type": "integer"}) "props"; 2 | 3 | namedRules "namedRules-" { 4 | x = "$string" "x?"; 5 | y = "$_" "y?"; 6 | } 7 | 8 | z = "$string" "z?"; 9 | 10 | case1 "case1?>$root" {} 11 | 12 | case2 "case2?" { 13 | nested "nested?>$root/namedRules" { // Should be nested?namedRules 14 | } 15 | also "^also?" { 16 | } 17 | } 18 | -------------------------------------------------------------------------------- /testsuite/schema.grm: -------------------------------------------------------------------------------- 1 | props(grammar?="$string", definitions?="$int", declarations?={"type": "integer"}) "props"; 2 | 3 | section _ "section+" { 4 | array = { 5 | "type": "array", 6 | "minItems": 1, 7 | "items": { "type": "string" }, 8 | "uniqueItems": false 9 | } "array*"; 10 | 11 | vector = { 12 | "type": "array", 13 | "minItems": 1, 14 | "items": { "type": "number" }, 15 | "uniqueItems": false 16 | } "vector*"; 17 | 18 | mod(value={ 19 | "type": "array", 20 | "minItems": 0, 21 | "items": { "type": "string" }, 22 | "uniqueItems": false 23 | }) "mod*"; 24 | } 25 | -------------------------------------------------------------------------------- /testsuite_test.go: -------------------------------------------------------------------------------- 1 | package denada 2 | 3 | import "os" 4 | import "log" 5 | import "fmt" 6 | import "path" 7 | import "testing" 8 | import "strings" 9 | 10 | import . "github.com/smartystreets/goconvey/convey" 11 | 12 | func ReparseFile(name string) error { 13 | filename := path.Join("testsuite", name) 14 | 15 | elems, err := ParseFile(filename) 16 | if err != nil { 17 | return err 18 | } 19 | 20 | str := Unparse(elems, false) 21 | 22 | relems, err := ParseString(str) 23 | if err != nil { 24 | return fmt.Errorf("Error in unparsed code: %v", err) 25 | } 26 | 27 | err = elems.Equals(relems) 28 | if err != nil { 29 | return fmt.Errorf("Inequality in reparsing of %s: %v", filename, err) 30 | } 31 | return nil 32 | } 33 | 34 | func CheckFile(name string) error { 35 | filename := path.Join("testsuite", name) 36 | 37 | elems, err := ParseFile(filename) 38 | if err != nil { 39 | return err 40 | } 41 | 42 | if len(elems) == 0 { 43 | return fmt.Errorf("Empty file") 44 | } 45 | 46 | props := elems[0] 47 | 48 | declsv, exists := props.Modifications["declarations"] 49 | var edecls int = 0 50 | if exists { 51 | edecls = declsv.MustInt(0) 52 | } 53 | 54 | defsv, exists := props.Modifications["definitions"] 55 | var edefs int = 0 56 | if exists { 57 | edefs = defsv.MustInt(0) 58 | } 59 | 60 | var adecls int = 0 61 | var adefs int = 0 62 | for _, e := range elems { 63 | if e.IsDeclaration() { 64 | adecls++ 65 | } 66 | if e.IsDefinition() { 67 | adefs++ 68 | } 69 | } 70 | 71 | if adecls != edecls { 72 | return fmt.Errorf("Expected %d declarations, found %d", edecls, adecls) 73 | } 74 | 75 | if adefs != edefs { 76 | return fmt.Errorf("Expected %d definitions, found %d", edefs, adefs) 77 | } 78 | 79 | grmv, exists := props.Modifications["grammar"] 80 | if exists { 81 | gfile := grmv.MustString() 82 | g, err := ParseFile(path.Join("testsuite", gfile)) 83 | if err != nil { 84 | return err 85 | } 86 | err = Check(elems, g, false) 87 | // Check if descriptions on input elements matche expected rule names 88 | if err == nil { 89 | for _, e := range elems.AllElements() { 90 | if e.Description == "" { 91 | err = fmt.Errorf("Input element %v in %s didn't have a description", 92 | e, name) 93 | return err 94 | } else if e.RulePath() == "" { 95 | err = fmt.Errorf("Input element %v in %s didn't seem to match anything", 96 | e, name) 97 | return err 98 | } else { 99 | if e.RulePath() != e.Description { 100 | err = fmt.Errorf("Input element %v matched rule %s but description implies a match with %s", e, e.RulePath(), e.Description) 101 | return err 102 | } 103 | } 104 | } 105 | } 106 | return err 107 | } 108 | 109 | return nil 110 | } 111 | 112 | func CheckError(name string) { 113 | err := CheckFile(name) 114 | So(err, ShouldBeNil) 115 | } 116 | 117 | func Test_TestSuite(t *testing.T) { 118 | Convey("Running TestSuite", t, func() { 119 | cur, err := os.Open("testsuite") 120 | So(err, ShouldBeNil) 121 | 122 | files, err := cur.Readdir(0) 123 | So(err, ShouldBeNil) 124 | 125 | for _, f := range files { 126 | name := f.Name() 127 | if !strings.HasSuffix(name, ".dnd") { 128 | continue 129 | } 130 | Convey("Processing "+name, func() { 131 | if strings.HasPrefix(name, "case") { 132 | err := CheckFile(name) 133 | if err != nil { 134 | log.Printf("Case %s: Failed: %v", name, err) 135 | } 136 | So(err, ShouldBeNil) 137 | err = ReparseFile(name) 138 | if err != nil { 139 | log.Printf("Case %s: Reparse failed: %v", name, err) 140 | } 141 | So(err, ShouldBeNil) 142 | return 143 | } 144 | if strings.HasPrefix(name, "ecase") { 145 | err := CheckFile(name) 146 | if err == nil { 147 | log.Printf("Error Case %s: FAILED", name) 148 | } 149 | So(err, ShouldNotBeNil) 150 | return 151 | } 152 | log.Printf("Unrecognized file type in test suite: %s", name) 153 | }) 154 | } 155 | }) 156 | } 157 | -------------------------------------------------------------------------------- /token.go: -------------------------------------------------------------------------------- 1 | package denada 2 | 3 | import "fmt" 4 | 5 | type TokenType int 6 | 7 | const ( 8 | T_IDENTIFIER TokenType = iota 9 | T_RBRACE 10 | T_LBRACE 11 | T_LPAREN 12 | T_RPAREN 13 | T_QUOTE 14 | T_EQUALS 15 | T_SEMI 16 | T_COMMA 17 | T_EOF 18 | T_WHITE 19 | T_SLCOMMENT 20 | T_MLCOMMENT 21 | T_UNKNOWN 22 | ) 23 | 24 | func (tt TokenType) String() string { 25 | switch tt { 26 | case T_IDENTIFIER: 27 | return "" 28 | case T_RBRACE: 29 | return "}" 30 | case T_LBRACE: 31 | return "{" 32 | case T_LPAREN: 33 | return "(" 34 | case T_RPAREN: 35 | return ")" 36 | case T_QUOTE: 37 | return "\"" 38 | case T_EQUALS: 39 | return "=" 40 | case T_SEMI: 41 | return ";" 42 | case T_COMMA: 43 | return "," 44 | case T_WHITE: 45 | return "" 46 | case T_SLCOMMENT: 47 | return "" 48 | case T_MLCOMMENT: 49 | return "" 50 | case T_EOF: 51 | return "EOF" 52 | case T_UNKNOWN: 53 | fallthrough 54 | default: 55 | return "" 56 | } 57 | } 58 | 59 | type UnexpectedToken struct { 60 | Found Token 61 | Expected string 62 | } 63 | 64 | func (u UnexpectedToken) Error() string { 65 | if u.Found.File != "" { 66 | return fmt.Sprintf("Expecting %s, found '%v' @ (%d, %d) in %s", u.Expected, u.Found.Type, 67 | u.Found.Line, u.Found.Column, u.Found.File) 68 | } else { 69 | return fmt.Sprintf("Expecting %s, found '%v' @ (%d, %d)", u.Expected, u.Found.Type, 70 | u.Found.Line, u.Found.Column) 71 | } 72 | } 73 | 74 | type Token struct { 75 | Type TokenType 76 | String string 77 | Line int 78 | Column int 79 | File string 80 | } 81 | 82 | func (t Token) Expected(expected string) UnexpectedToken { 83 | return UnexpectedToken{ 84 | Found: t, 85 | Expected: expected, 86 | } 87 | } 88 | -------------------------------------------------------------------------------- /transforms.go: -------------------------------------------------------------------------------- 1 | package denada 2 | 3 | import "fmt" 4 | 5 | var importGrammar = ` 6 | import(file="$string", recursive?="$bool") "import*"; 7 | ` 8 | 9 | func ImportTransform(root ElementList) (ret ElementList, err error) { 10 | g, err := ParseString(importGrammar) 11 | if err != nil { 12 | err = fmt.Errorf("Error parsing import statement grammar: %v", err) 13 | return 14 | } 15 | 16 | ret = ElementList{} 17 | for _, e := range root { 18 | match := matchElement(e, g[0], g[0].Contents, false, "", "", NullContext()) 19 | if match == nil { 20 | file := e.Modifications["file"].MustString() 21 | insert, err := ParseFile(file) 22 | if err != nil { 23 | return nil, fmt.Errorf("Error parsing import contents from %s: %v", file, err) 24 | } 25 | rval, present := e.Modifications["recursive"] 26 | recursive := false 27 | if present { 28 | recursive = rval.MustBool() 29 | } 30 | 31 | if recursive { 32 | insert, err = ImportTransform(insert) 33 | if err != nil { 34 | return nil, err 35 | } 36 | } 37 | for _, i := range insert { 38 | ret = append(ret, i) 39 | } 40 | } else { 41 | if e.IsDefinition() { 42 | newchildren, err := ImportTransform(e.Contents) 43 | if err != nil { 44 | return nil, err 45 | } 46 | newe := e.Clone() 47 | newe.Contents = newchildren 48 | ret = append(ret, newe) 49 | } else { 50 | ret = append(ret, e) 51 | } 52 | } 53 | } 54 | return ret, nil 55 | } 56 | -------------------------------------------------------------------------------- /unparse.go: -------------------------------------------------------------------------------- 1 | package denada 2 | 3 | import "io" 4 | import "fmt" 5 | import "bytes" 6 | import "strings" 7 | 8 | import "github.com/bitly/go-simplejson" 9 | 10 | func Unparse(elems ElementList, rules bool) string { 11 | w := bytes.NewBuffer([]byte{}) 12 | UnparseTo(elems, w, rules) 13 | return w.String() 14 | } 15 | 16 | func UnparseTo(elems ElementList, w io.Writer, rules bool) { 17 | unparse(elems, "", w, rules) 18 | } 19 | 20 | func unparse(elems ElementList, prefix string, w io.Writer, rules bool) { 21 | for _, e := range elems { 22 | unparseElement(*e, prefix, w, rules) 23 | fmt.Fprintf(w, prefix+"\n") 24 | } 25 | } 26 | 27 | func unparseValue(v *simplejson.Json, prefix string) string { 28 | enc, err := v.EncodePretty() 29 | if err != nil { 30 | panic(err) 31 | } 32 | estr := string(enc) 33 | estr = strings.Replace(estr, "\n", "\n"+prefix, -1) 34 | return estr 35 | } 36 | 37 | func UnparseElement(e Element, rules bool) string { 38 | w := bytes.NewBuffer([]byte{}) 39 | unparseElement(e, "", w, rules) 40 | return w.String() 41 | } 42 | 43 | func UnparseElementTo(e Element, w io.Writer, rules bool) { 44 | unparseElement(e, "", w, rules) 45 | } 46 | 47 | func unparseElement(e Element, prefix string, w io.Writer, rules bool) { 48 | fmt.Fprintf(w, prefix) 49 | for _, q := range e.Qualifiers { 50 | fmt.Fprintf(w, "%s ", q) 51 | } 52 | fmt.Fprintf(w, "%s", e.Name) 53 | if len(e.Modifications) > 0 { 54 | first := true 55 | fmt.Fprintf(w, "(") 56 | for k, v := range e.Modifications { 57 | if !first { 58 | fmt.Fprintf(w, ", ") 59 | } 60 | if v != nil { 61 | estr := unparseValue(v, prefix) 62 | fmt.Fprintf(w, "%s=%s", k, estr) 63 | } 64 | first = false 65 | } 66 | fmt.Fprintf(w, ")") 67 | } 68 | if e.IsDefinition() { 69 | fmt.Fprintf(w, " ") 70 | if e.Description != "" { 71 | fmt.Fprintf(w, "\"%s\" ", strings.Replace(e.Description, "\"", "\\\"", 0)) 72 | } 73 | if rules && (e.rulepath != "" || e.rule != "") { 74 | fmt.Fprintf(w, "[%s:%s] ", e.rulepath, e.rule) 75 | } 76 | fmt.Fprintf(w, "{\n") 77 | if e.Contents != nil { 78 | unparse(e.Contents, prefix+" ", w, rules) 79 | } 80 | fmt.Fprintf(w, "%s}", prefix) 81 | } else { 82 | if e.Value != nil { 83 | estr := unparseValue(e.Value, prefix) 84 | fmt.Fprintf(w, "=%s", estr) 85 | } 86 | if e.Description != "" { 87 | fmt.Fprintf(w, " \"%s\"", strings.Replace(e.Description, "\"", "\\\"", 0)) 88 | } 89 | if rules && (e.rulepath != "" || e.rule != "") { 90 | fmt.Fprintf(w, " [%s:%s]", e.rulepath, e.rule) 91 | } 92 | fmt.Fprintf(w, ";") 93 | } 94 | } 95 | -------------------------------------------------------------------------------- /utils.go: -------------------------------------------------------------------------------- 1 | package denada 2 | 3 | import "github.com/bitly/go-simplejson" 4 | 5 | func getString(v *simplejson.Json) (string, bool) { 6 | str, err := v.String() 7 | return str, err == nil 8 | } 9 | 10 | func getStringArray(v *simplejson.Json) ([]string, bool) { 11 | str, err := v.StringArray() 12 | return str, err == nil 13 | } 14 | 15 | /* 16 | A special utility function to quickly get the value (if it exists) of constructs 17 | like this: 18 | 19 | fmu SomeName { 20 | parameter := "Contents"; 21 | } 22 | */ 23 | func GetStringParameter(app *Element, key string) (string, bool) { 24 | e := app.Contents.Declarations().FirstNamed(key) 25 | if e == nil { 26 | return "", false 27 | } 28 | return getString(e.Value) 29 | } 30 | 31 | func GetStringValue(app *Element) (string, bool) { 32 | val := app.Value 33 | if val == nil { 34 | return "", false 35 | } 36 | return getString(val) 37 | } 38 | 39 | func GetStringModification(app *Element, key string) (string, bool) { 40 | val, exists := app.Modifications[key] 41 | if !exists { 42 | return "", false 43 | } 44 | return getString(val) 45 | } 46 | 47 | func GetStringArrayParameter(app *Element, key string) ([]string, bool) { 48 | e := app.Contents.Declarations().FirstNamed(key) 49 | if e == nil { 50 | return []string{}, false 51 | } 52 | return getStringArray(e.Value) 53 | } 54 | 55 | func GetStringArrayValue(app *Element) ([]string, bool) { 56 | val := app.Value 57 | if val == nil { 58 | return []string{}, false 59 | } 60 | return getStringArray(val) 61 | } 62 | 63 | func GetStringArrayModification(app *Element, key string) ([]string, bool) { 64 | val, exists := app.Modifications[key] 65 | if !exists { 66 | return []string{}, false 67 | } 68 | return getStringArray(val) 69 | } 70 | --------------------------------------------------------------------------------