├── .gitignore ├── .isort.cfg ├── .travis.yml ├── LICENSE ├── README.md ├── bin └── pystachio_repl ├── pystachio ├── __init__.py ├── base.py ├── basic.py ├── choice.py ├── composite.py ├── config.py ├── container.py ├── matcher.py ├── naming.py ├── parsing.py └── typing.py ├── setup.cfg ├── setup.py ├── tests ├── test_base.py ├── test_basic_types.py ├── test_bigger_examples.py ├── test_choice.py ├── test_composite_types.py ├── test_config.py ├── test_container_types.py ├── test_matching.py ├── test_naming.py ├── test_parsing.py └── test_typing.py └── tox.ini /.gitignore: -------------------------------------------------------------------------------- 1 | *~ 2 | *.pyc 3 | *.egg-info 4 | build/* 5 | dist/* 6 | .tox 7 | .coverage 8 | .pytest_cache/ 9 | tests/htmlcov 10 | TODO 11 | -------------------------------------------------------------------------------- /.isort.cfg: -------------------------------------------------------------------------------- 1 | [settings] 2 | line_length=100 3 | known_first_party=pystachio 4 | multi_line_output=3 5 | default_section=THIRDPARTY 6 | -------------------------------------------------------------------------------- /.travis.yml: -------------------------------------------------------------------------------- 1 | # Enables support for a docker container-based build, 2 | # see: http://docs.travis-ci.com/user/workers/container-based-infrastructure/ 3 | sudo: false 4 | 5 | language: python 6 | dist: trusty 7 | 8 | python: 9 | - "3.6" 10 | - "3.8" 11 | - "pypy3" 12 | 13 | install: 14 | - pip install tox-travis 15 | script: 16 | - tox 17 | - tox -e isort-check 18 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | Copyright (C) 2011 Brian Wickman 2 | 3 | Permission is hereby granted, free of charge, to any person obtaining a copy of 4 | this software and associated documentation files (the "Software"), to deal in 5 | the Software without restriction, including without limitation the rights to 6 | use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies 7 | of the Software, and to permit persons to whom the Software is furnished to do 8 | so, subject to the following conditions: 9 | 10 | The above copyright notice and this permission notice shall be included in all 11 | copies or substantial portions of the Software. 12 | 13 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 14 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 15 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 16 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 17 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 18 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 19 | SOFTWARE. 20 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Pystachio # 2 | [![Build Status](https://travis-ci.org/wickman/pystachio.svg)](https://travis-ci.org/wickman/pystachio) 3 | 4 | ### tl;dr ### 5 | 6 | Pystachio is a type-checked dictionary templating library. 7 | 8 | ### why? ### 9 | 10 | Its primary use is for the construction of miniature domain-specific 11 | configuration languages. Schemas defined by Pystachio can themselves be 12 | serialized and reconstructed into other Python interpreters. Pystachio 13 | objects are tailored via Mustache templates, as explained in the section on 14 | templating. 15 | 16 | ## Similar projects ## 17 | 18 | This project is unrelated to the defunct Javascript Python interpreter. 19 | 20 | Notable related projects: 21 | 22 | * [dictshield](http://github.com/j2labs/dictshield) 23 | * [remoteobjects](http://github.com/saymedia/remoteobjects) 24 | * Django's [model.Model](https://docs.djangoproject.com/en/dev/ref/models/instances/) 25 | 26 | ## Requirements ## 27 | 28 | Tested and works in CPython3 and PyPy3. 29 | 30 | ## Overview ## 31 | 32 | You can define a structured type through the 'Struct' type: 33 | 34 | from pystachio import ( 35 | Integer, 36 | String, 37 | Struct) 38 | 39 | class Employee(Struct): 40 | first = String 41 | last = String 42 | age = Integer 43 | 44 | 45 | By default all fields are optional: 46 | 47 | >>> Employee().check() 48 | TypeCheck(OK) 49 | 50 | >>> Employee(first = 'brian') 51 | Employee(first=brian) 52 | >>> Employee(first = 'brian').check() 53 | TypeCheck(OK) 54 | 55 | 56 | But it is possible to make certain fields required: 57 | 58 | from pystachio import Required 59 | 60 | class Employee(Struct): 61 | first = Required(String) 62 | last = Required(String) 63 | age = Integer 64 | 65 | We can still instantiate objects with empty fields: 66 | 67 | >>> Employee() 68 | Employee() 69 | 70 | 71 | But they will fail type checks: 72 | 73 | >>> Employee().check() 74 | TypeCheck(FAILED): Employee[last] is required. 75 | 76 | 77 | Struct objects are purely functional and hence immutable after 78 | constructed, however they are composable like functors: 79 | 80 | >>> brian = Employee(first = 'brian') 81 | >>> brian(last = 'wickman') 82 | Employee(last=wickman, first=brian) 83 | >>> brian 84 | Employee(first=brian) 85 | >>> brian = brian(last='wickman') 86 | >>> brian.check() 87 | TypeCheck(OK) 88 | 89 | 90 | Object fields may also acquire defaults: 91 | 92 | class Employee(Struct): 93 | first = Required(String) 94 | last = Required(String) 95 | age = Integer 96 | location = Default(String, "San Francisco") 97 | 98 | >>> Employee() 99 | Employee(location=San Francisco) 100 | 101 | 102 | Schemas wouldn't be terribly useful without the ability to be hierarchical: 103 | 104 | class Location(Struct): 105 | city = String 106 | state = String 107 | country = String 108 | 109 | class Employee(Struct): 110 | first = Required(String) 111 | last = Required(String) 112 | age = Integer 113 | location = Default(Location, Location(city = "San Francisco")) 114 | 115 | >>> Employee(first="brian", last="wickman") 116 | Employee(last=wickman, location=Location(city=San Francisco), first=brian) 117 | 118 | >>> Employee(first="brian", last="wickman").check() 119 | TypeCheck(OK) 120 | 121 | 122 | ## The type system ## 123 | 124 | There are five basic types, two basic container types and then the `Struct` and `Choice` types. 125 | 126 | ### Basic Types ### 127 | 128 | There are five basic types: `String`, `Integer`, `Float`, `Boolean` and `Enum`. The first four behave as expected: 129 | 130 | >>> Float(1.0).check() 131 | TypeCheck(OK) 132 | >>> String("1.0").check() 133 | TypeCheck(OK) 134 | >>> Integer(1).check() 135 | TypeCheck(OK) 136 | >>> Boolean(False).check() 137 | TypeCheck(OK) 138 | 139 | They also make a best effort to coerce into the appropriate type: 140 | 141 | >>> Float("1.0") 142 | Float(1.0) 143 | >>> String(1.0) 144 | String(1.0) 145 | >>> Integer("1") 146 | Integer(1) 147 | >>> Boolean("true") 148 | Boolean(True) 149 | 150 | Though the same gotchas apply as standard coercion in Python: 151 | 152 | >>> int("1.0") 153 | ValueError: invalid literal for int() with base 10: '1.0' 154 | >>> Integer("1.0") 155 | pystachio.objects.CoercionError: Cannot coerce '1.0' to Integer 156 | 157 | with the exception of `Boolean` which accepts "false" as falsy. 158 | 159 | 160 | Enum is a factory that produces new enumeration types: 161 | 162 | >>> Enum('Red', 'Green', 'Blue') 163 | 164 | >>> Color = Enum('Red', 'Green', 'Blue') 165 | >>> Color('Red') 166 | Enum_Red_Green_Blue(Red) 167 | >>> Color('Brown') 168 | Traceback (most recent call last): 169 | File "", line 1, in 170 | File "/Users/wickman/clients/pystachio/pystachio/basic.py", line 208, in __init__ 171 | self.__class__.__name__, ', '.join(self.VALUES))) 172 | ValueError: Enum_Red_Green_Blue only accepts the following values: Red, Green, Blue 173 | 174 | Enums can also be constructed using `namedtuple` syntax to generate more illustrative class names: 175 | 176 | >>> Enum('Color', ('Red', 'Green', 'Blue')) 177 | 178 | >>> Color = Enum('Color', ('Red', 'Green', 'Blue')) 179 | >>> Color('Red') 180 | Color(Red) 181 | 182 | 183 | ### Choices #### 184 | 185 | Choice types represent alternatives - values that can have one of some set of values. 186 | 187 | >>> C = Choice([Integer, String]) 188 | >>> c1 = C("abc") 189 | >>> c2 = C(343) 190 | 191 | ### Container types ### 192 | 193 | There are two container types: the `List` type and the `Map` type. Lists 194 | are parameterized by the type they contain, and Maps are parameterized from 195 | a key type to a value type. 196 | 197 | #### Lists #### 198 | 199 | You construct a `List` by specifying its type (it actually behaves like a 200 | metaclass, since it produces a type): 201 | 202 | >>> List(String) 203 | 204 | >>> List(String)([]) 205 | StringList() 206 | >>> List(String)(["a", "b", "c"]) 207 | StringList(a, b, c) 208 | 209 | 210 | They compose like expected: 211 | 212 | >>> li = List(Integer) 213 | >>> li 214 | 215 | >>> List(li) 216 | 217 | >>> List(li)([li([1,"2",3]), li([' 2', '3 ', 4])]) 218 | IntegerListList(IntegerList(1, 2, 3), IntegerList(2, 3, 4)) 219 | 220 | 221 | Type checking is done recursively: 222 | 223 | >> List(li)([li([1,"2",3]), li([' 2', '3 ', 4])]).check() 224 | TypeCheck(OK) 225 | 226 | 227 | #### Maps #### 228 | 229 | You construct a `Map` by specifying the source and destination types: 230 | 231 | >>> ages = Map(String, Integer)({ 232 | ... 'brian': 30, 233 | ... 'ian': 15, 234 | ... 'robey': 5000 235 | ... }) 236 | >>> ages 237 | StringIntegerMap(brian => 28, ian => 15, robey => 5000) 238 | >>> ages.check() 239 | TypeCheck(OK) 240 | 241 | Much like all other types, these types are immutable. The only way to 242 | "mutate" would be to create a whole new Map. Technically speaking these 243 | types are hashable as well, so you can construct stranger composite types 244 | (added indentation for clarity.) 245 | 246 | >>> fake_ages = Map(String, Integer)({ 247 | ... 'brian': 28, 248 | ... 'ian': 15, 249 | ... 'robey': 5000 250 | ... }) 251 | >>> real_ages = Map(String, Integer)({ 252 | ... 'brian': 30, 253 | ... 'ian': 21, 254 | ... 'robey': 35 255 | ... }) 256 | >>> believability = Map(Map(String, Integer), Float)({ 257 | ... fake_ages: 0.2, 258 | ... real_ages: 0.9 259 | ... }) 260 | >>> believability 261 | StringIntegerMapFloatMap( 262 | StringIntegerMap(brian => 28, ian => 15, robey => 5000) => 0.2, 263 | StringIntegerMap(brian => 30, ian => 21, robey => 35) => 0.9) 264 | 265 | 266 | 267 | ## Object scopes ## 268 | 269 | Objects have "environments": a set of bound scopes that follow the Object 270 | around. Objects are still immutable. The act of binding a variable to an 271 | Object just creates a new object with an additional variable scope. You can 272 | print the scopes by using the `scopes` function: 273 | 274 | >>> String("hello").scopes() 275 | () 276 | 277 | You can bind variables to that object with the `bind` function: 278 | 279 | >>> String("hello").bind(herp = "derp") 280 | String(hello) 281 | 282 | The environment variables of an object do not alter equality, for example: 283 | 284 | >>> String("hello") == String("hello") 285 | True 286 | >>> String("hello").bind(foo = "bar") == String("hello") 287 | True 288 | 289 | The object appears to be the same but it carries that scope around with it: 290 | 291 | >>> String("hello").bind(herp = "derp").scopes() 292 | (Environment({Ref(herp): 'derp'}),) 293 | 294 | 295 | Furthermore you can bind multiple times: 296 | 297 | >>> String("hello").bind(herp = "derp").bind(herp = "extra derp").scopes() 298 | (Environment({Ref(herp): 'extra derp'}), Environment({Ref(herp): 'derp'})) 299 | 300 | 301 | You can use keyword arguments, but you can also pass dictionaries directly: 302 | 303 | >>> String("hello").bind({"herp": "derp"}).scopes() 304 | (Environment({Ref(herp): 'derp'}),) 305 | 306 | Think of this as a "mount table" for mounting objects at particular points 307 | in a namespace. This namespace is hierarchical: 308 | 309 | >>> String("hello").bind(herp = "derp", metaherp = {"a": 1, "b": {"c": 2}}).scopes() 310 | (Environment({Ref(herp): 'derp', Ref(metaherp.b.c): '2', Ref(metaherp.a): '1'}),) 311 | 312 | In fact, you can bind any `Namable` object, including `List`, `Map`, and 313 | `Struct` types directly: 314 | 315 | >>> class Person(Struct) 316 | ... first = String 317 | ... last = String 318 | ... 319 | >>> String("hello").bind(Person(first="brian")).scopes() 320 | (Person(first=brian),) 321 | 322 | The `Environment` object is simply a mechanism to bind arbitrary strings 323 | into a namespace compatible with `Namable` objects. 324 | 325 | Because you can bind multiple times, scopes just form a name-resolution order: 326 | 327 | >>> (String("hello").bind(Person(first="brian"), first="john") 328 | .bind({'first': "jake"}, Person(first="jane"))).scopes() 329 | (Person(first=jane), 330 | Environment({Ref(first): 'jake'}), 331 | Environment({Ref(first): 'john'}), 332 | Person(first=brian)) 333 | 334 | The later a variable is bound, the "higher priority" its name resolution 335 | becomes. Binding to an object is to achieve the effect of local overriding. 336 | But you can also do a lower-priority "global" bindings via `in_scope`: 337 | 338 | >>> env = Environment(globalvar = "global variable", sharedvar = "global shared variable") 339 | >>> obj = String("hello").bind(localvar = "local variable", sharedvar = "local shared variable") 340 | >>> obj.scopes() 341 | (Environment({Ref(localvar): 'local variable', Ref(sharedvar): 'local shared variable'}),) 342 | 343 | Now we can bind `env` directly into `obj` as if they were local variables using `bind`: 344 | 345 | >>> obj.bind(env).scopes() 346 | (Environment({Ref(globalvar): 'global variable', Ref(sharedvar): 'global shared variable'}), 347 | Environment({Ref(localvar): 'local variable', Ref(sharedvar): 'local shared variable'})) 348 | 349 | Alternatively we can bind `env` into `obj` as if they were global variables using `in_scope`: 350 | 351 | >>> obj.in_scope(env).scopes() 352 | (Environment({Ref(localvar): 'local variable', Ref(sharedvar): 'local shared variable'}), 353 | Environment({Ref(globalvar): 'global variable', Ref(sharedvar): 'global shared variable'})) 354 | 355 | You can see the local variables take precedence. The use of scoping will 356 | become more obvious when in the context of templating. 357 | 358 | 359 | ## Templating ## 360 | 361 | ### Simple templates ### 362 | 363 | As briefly mentioned at the beginning, Mustache templates are first class 364 | "language" features. Let's look at the simple case of a `String` to see how 365 | Mustache templates might behave. 366 | 367 | >>> String('echo {{hello_message}}') 368 | String(echo {{hello_message}}) 369 | 370 | OK, seems reasonable enough. Now let's look at the more complicated version 371 | of a `Float`: 372 | 373 | >>> Float('not.floaty') 374 | CoercionError: Cannot coerce 'not.floaty' to Float 375 | 376 | But if we template it, it behaves differently: 377 | 378 | >>> Float('{{not}}.{{floaty}}') 379 | Float({{not}}.{{floaty}}) 380 | 381 | Pystachio understands that by introducing a Mustache template, that we 382 | should lazily coerce the `Float` only once it's fully specified by its 383 | environment. For example: 384 | 385 | >>> not_floaty = Float('{{not}}.{{floaty}}') 386 | >>> not_floaty.bind({'not': 1}) 387 | Float(1.{{floaty}}) 388 | 389 | We've bound a variable into the environment of `not_floaty`. It's still not 390 | floaty: 391 | 392 | >>> not_floaty.bind({'not': 1}).check() 393 | TypeCheck(FAILED): u'1.{{floaty}}' not a float 394 | 395 | However, once it becomes fully specified, the picture changes: 396 | 397 | >>> floaty = not_floaty.bind({'not': 1, 'floaty': 0}) 398 | >>> floaty 399 | Float(1.0) 400 | >>> floaty.check() 401 | TypeCheck(OK) 402 | 403 | Of course, the coercion can only take place if the environment is legit: 404 | 405 | >>> not_floaty.bind({'not': 1, 'floaty': 'GARBAGE'}) 406 | CoercionError: Cannot coerce '1.GARBAGE' to Float 407 | 408 | It's worth noting that `floaty` has not been coerced permanently: 409 | 410 | >>> floaty 411 | Float(1.0) 412 | >>> floaty.bind({'not': 2}) 413 | Float(2.0) 414 | 415 | In fact, `floaty` continues to store the template; it's just hidden from 416 | view and interpolated on-demand: 417 | 418 | >>> floaty._value 419 | '{{not}}.{{floaty}}' 420 | 421 | 422 | As we mentioned before, objects have scopes. Let's look at the case of floaty: 423 | 424 | >>> floaty = not_floaty.bind({'not': 1, 'floaty': 0}) 425 | >>> floaty 426 | Float(1.0) 427 | >>> floaty.scopes() 428 | (Environment({Ref(not): '1', Ref(floaty): '0'}),) 429 | 430 | 431 | But if we bind `not = 2`: 432 | 433 | >>> floaty.bind({'not': 2}) 434 | Float(2.0) 435 | >>> floaty.bind({'not': 2}).scopes() 436 | (Environment({Ref(not): '2'}), Environment({Ref(floaty): '0', Ref(not): '1'})) 437 | 438 | If we had merely just evaluated floaty in the scope of `not = 2`, it would have behaved differently: 439 | 440 | >>> floaty.in_scope({'not': 2}) 441 | Float(1.0) 442 | 443 | The interpolation of template variables happens in scope order from top 444 | down. Ultimately `bind` just prepends a scope to the list of scopes and 445 | `in_scope` appends a scope to the end of the list of scopes. 446 | 447 | ### Complex templates ### 448 | 449 | Remember however that you can bind any `Namable` object, which includes 450 | `List`, `Map`, `Struct` and `Environment` types, and these are hierarchical. 451 | Take for example a schema that defines a UNIX process: 452 | 453 | class Process(Struct): 454 | name = Default(String, '{{config.name}}') 455 | cmdline = String 456 | 457 | class ProcessConfig(Struct): 458 | name = String 459 | ports = Map(String, Integer) 460 | 461 | The expectation could be that `Process` structures are always interpolated 462 | in an environment where `config` is set to the `ProcessConfig`. 463 | 464 | For example: 465 | 466 | >>> webserver = Process(cmdline = "bin/tfe --listen={{config.ports[http]}} --health={{config.ports[health]}}") 467 | >>> webserver 468 | Process(cmdline=bin/tfe --listen={{config.ports[http]}} --health={{config.ports[health]}}, name={{config.name}}) 469 | 470 | Now let's define its configuration: 471 | 472 | >>> app_config = ProcessConfig(name = "tfe", ports = {'http': 80, 'health': 8888}) 473 | >>> app_config 474 | ProcessConfig(name=tfe, ports=StringIntegerMap(health => 8888, http => 80)) 475 | 476 | And let's evaluate the configuration: 477 | 478 | >>> webserver % Environment(config = app_config) 479 | Process(cmdline=bin/tfe --listen=80 --health=8888, name=tfe) 480 | 481 | The `%`-based interpolation is just shorthand for `in_scope`. 482 | 483 | `List` types and `Map` types are dereferenced as expected in the context of 484 | `{{}}`-style mustache templates, using `[index]` for `List` types and 485 | `[value]` for `Map` types. `Struct` types are dereferenced using `.`-notation. 486 | 487 | For example, `{{foo.bar[23][baz].bang}}` translates to a name lookup chain 488 | of `foo (Struct) => bar (List or Map) => 23 (Map) => baz (Struct) => bang`, 489 | ensuring the type consistency at each level of the lookup chain. 490 | 491 | ## Templating scope inheritance ## 492 | 493 | The use of templating is most powerful in the use of `Struct` types where 494 | parent object scope is inherited by all children during interpolation. 495 | 496 | Let's look at the example of building a phone book type. 497 | 498 | class PhoneBookEntry(Struct): 499 | name = Required(String) 500 | number = Required(Integer) 501 | 502 | class PhoneBook(Struct): 503 | city = Required(String) 504 | people = List(PhoneBookEntry) 505 | 506 | >>> sf = PhoneBook(city = "San Francisco").bind(areacode = 415) 507 | >>> sj = PhoneBook(city = "San Jose").bind(areacode = 408) 508 | 509 | We met a girl last night in a bar, her name was Jenny, and her number was 8 510 | 6 7 5 3 oh nayee-aye-in. But in the bay area, you never know what her area 511 | code could be, so we template it: 512 | 513 | >>> jenny = PhoneBookEntry(name = "Jenny", number = "{{areacode}}8675309") 514 | 515 | But `brian` is a Nebraskan farm boy from the 402 and took his number with him: 516 | 517 | >>> brian = PhoneBookEntry(name = "Brian", number = "{{areacode}}5551234") 518 | >>> brian = brian.bind(areacode = 402) 519 | 520 | If we assume that Jenny is from San Francisco, then we look her up in the San Francisco phone book: 521 | 522 | >>> sf(people = [jenny]) 523 | PhoneBook(city=San Francisco, people=PhoneBookEntryList(PhoneBookEntry(name=Jenny, number=4158675309))) 524 | 525 | 526 | But it's equally likely that she could be from San Jose: 527 | 528 | >>> sj(people = [jenny]) 529 | PhoneBook(city=San Jose, people=PhoneBookEntryList(PhoneBookEntry(name=Jenny, number=4088675309))) 530 | 531 | 532 | If we bind `jenny` to one of the phone books, she inherits the area code 533 | from her parent object. Of course, `brian` is from Nebraska and he kept his 534 | number, so San Jose or San Francisco, his number remains the same: 535 | 536 | >>> sf(people = [jenny, brian]) 537 | PhoneBook(city=San Francisco, 538 | people=PhoneBookEntryList(PhoneBookEntry(name=Jenny, number=4158675309), 539 | PhoneBookEntry(name=Brian, number=4025551234))) 540 | 541 | 542 | ## Dictionary type-checking ## 543 | 544 | Because of how `Struct` based schemas are created, the constructor of 545 | such a schema behaves like a deserialization mechanism from a straight 546 | Python dictionary. In a sense, deserialization comes for free. Take the 547 | schema defined below: 548 | 549 | class Resources(Struct): 550 | cpu = Required(Float) 551 | ram = Required(Integer) 552 | disk = Default(Integer, 2 * 2**30) 553 | 554 | class Process(Struct): 555 | name = Required(String) 556 | resources = Required(Resources) 557 | cmdline = String 558 | max_failures = Default(Integer, 1) 559 | 560 | class Task(Struct): 561 | name = Required(String) 562 | processes = Required(List(Process)) 563 | max_failures = Default(Integer, 1) 564 | 565 | Let's write out a task as a dictionary, as we would expect to see from the schema: 566 | 567 | task = { 568 | 'name': 'basic', 569 | 'processes': [ 570 | { 571 | 'resources': { 572 | 'cpu': 1.0, 573 | 'ram': 100 574 | }, 575 | 'cmdline': 'echo hello world' 576 | } 577 | ] 578 | } 579 | 580 | And instantiate it as a Task (indentation provided for clarity): 581 | 582 | >>> tsk = Task(task) 583 | >>> tsk 584 | Task(processes=ProcessList(Process(cmdline=echo hello world, max_failures=1, 585 | resources=Resources(disk=2147483648, ram=100, cpu=1.0))), 586 | max_failures=1, name=basic) 587 | 588 | 589 | The schema that we defined as a Python class structure is applied 590 | to the dictionary. We can use this schema to type-check the dictionary: 591 | 592 | >>> tsk.check() 593 | TypeCheck(FAILED): Task[processes] failed: Element in ProcessList failed check: Process[name] is required. 594 | 595 | It turns out that we forgot to specify the name of the `Process` in our 596 | process list, and it was a `Required` field. If we update the dictionary to 597 | specify 'name', it will type check successfully. 598 | 599 | 600 | ## Type construction ## 601 | 602 | 603 | It is possible to serialize constructed types, pickle them and send them 604 | around with your dictionary data in order to do portable type checking. 605 | 606 | ### Serialization ### 607 | 608 | 609 | Every type in Pystachio has a `serialize_type` method which is used to 610 | describe the type in a portable way. The basic types are uninteresting: 611 | 612 | >>> String.serialize_type() 613 | ('String',) 614 | >>> Integer.serialize_type() 615 | ('Integer',) 616 | >>> Float.serialize_type() 617 | ('Float',) 618 | 619 | The notation is simply: `String` types are produced by the `"String"` type 620 | factory. They are not parameterized types so they need no additional type 621 | parameters. However, Lists and Maps are parameterized: 622 | 623 | >>> List(String).serialize_type() 624 | ('List', ('String',)) 625 | >>> Map(Integer,String).serialize_type() 626 | ('Map', ('Integer',), ('String',)) 627 | >>> Map(Integer,List(String)).serialize_type() 628 | ('Map', ('Integer',), ('List', ('String',))) 629 | 630 | Furthermore, composite types created with `Struct` are also serializable. 631 | Take the composite types defined in the previous section: `Task`, `Process` and `Resources`. 632 | 633 | >>> from pprint import pprint 634 | >>> pprint(Resources.serialize_type(), indent=2, width=100) 635 | ( 'Struct', 636 | 'Resources', 637 | ('cpu', (True, (), True, ('Float',))), 638 | ('disk', (False, 2147483648, False, ('Integer',))), 639 | ('ram', (True, (), True, ('Integer',)))) 640 | 641 | In other words, the `Struct` factory is producing a type with a set of type 642 | parameters: `Resources` is the name of the struct, `cpu`, `disk` and `ram` 643 | are attributes of the type. 644 | 645 | If you serialize `Task`, it recursively serializes its children types: 646 | 647 | >>> pprint(Task.serialize_type(), indent=2, width=100) 648 | ( 'Struct', 649 | 'Task', 650 | ('max_failures', (False, 1, False, ('Integer',))), 651 | ('name', (True, (), True, ('String',))), 652 | ( 'processes', 653 | ( True, 654 | (), 655 | True, 656 | ( 'List', 657 | ( 'Struct', 658 | 'Process', 659 | ('cmdline', (False, (), True, ('String',))), 660 | ('max_failures', (False, 1, False, ('Integer',))), 661 | ('name', (True, (), True, ('String',))), 662 | ( 'resources', 663 | ( True, 664 | (), 665 | True, 666 | ( 'Struct', 667 | 'Resources', 668 | ('cpu', (True, (), True, ('Float',))), 669 | ('disk', (False, 2147483648, False, ('Integer',))), 670 | ('ram', (True, (), True, ('Integer',))))))))))) 671 | 672 | 673 | ### Deserialization ### 674 | 675 | Given a type tuple produced by serialize_type, you can then use 676 | `TypeFactory.load` from `pystachio.typing` to load a type into an interpreter. For example: 677 | 678 | >>> pprint(TypeFactory.load(Resources.serialize_type())) 679 | {'Float': , 680 | 'Integer': , 681 | 'Resources': } 682 | 683 | `TypeFactory.load` returns a map from type name to the fully reified type for all types required to 684 | describe the serialized type, including children. In the example of `Task` above: 685 | 686 | >>> pprint(TypeFactory.load(Task.serialize_type())) 687 | {'Float': , 688 | 'Integer': , 689 | 'Process': , 690 | 'ProcessList': , 691 | 'Resources': , 692 | 'String': , 693 | 'Task': } 694 | 695 | `TypeFactory.load` also takes an `into` keyword argument, so you can do 696 | `TypeFactory.load(type, into=globals())` in order to deposit them into your interpreter: 697 | 698 | >>> from pystachio import * 699 | >>> TypeFactory.load(( 'Struct', 700 | ... 'Task', 701 | ... ('max_failures', (False, 1, False, ('Integer',))), 702 | ... ('name', (True, (), True, ('String',))), 703 | ... ( 'processes', 704 | ... ( True, 705 | ... (), 706 | ... True, 707 | ... ( 'List', 708 | ... ( 'Struct', 709 | ... 'Process', 710 | ... ('cmdline', (False, (), True, ('String',))), 711 | ... ('max_failures', (False, 1, False, ('Integer',))), 712 | ... ('name', (True, (), True, ('String',))), 713 | ... ( 'resources', 714 | ... ( True, 715 | ... (), 716 | ... True, 717 | ... ( 'Struct', 718 | ... 'Resources', 719 | ... ('cpu', (True, (), True, ('Float',))), 720 | ... ('disk', (False, 2147483648, False, ('Integer',))), 721 | ... ('ram', (True, (), True, ('Integer',))))))))))), into=globals()) 722 | >>> Task 723 | 724 | >>> Process 725 | 726 | >>> Task().check() 727 | TypeCheck(FAILED): Task[processes] is required. 728 | >>> Resources().check() 729 | TypeCheck(FAILED): Resources[ram] is required. 730 | >>> Resources(cpu = 1.0, ram = 1024, disk = 1024).check() 731 | TypeCheck(OK) 732 | 733 | 734 | ## Equivalence ## 735 | 736 | Types produced by `TypeFactory.load` are reified types but they are not 737 | identical to each other. This could be provided in the future via type 738 | memoization but that would require keeping some amount of state around. 739 | 740 | Instead, `__instancecheck__` has been provided, so that you can do 741 | `isinstance` checks: 742 | 743 | >>> Task 744 | 745 | >>> Task == TypeFactory.new({}, *Task.serialize_type()) 746 | False 747 | >>> isinstance(Task(), TypeFactory.new({}, *Task.serialize_type())) 748 | True 749 | 750 | 751 | ## Author ## 752 | 753 | @wickman (Brian Wickman) 754 | 755 | Thanks to @marius for some of the original design ideas, @benh, @jsirois, 756 | @wfarner and others for constructive comments. 757 | -------------------------------------------------------------------------------- /bin/pystachio_repl: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python3 2 | 3 | from pystachio import * 4 | 5 | from code import interact 6 | interact('Pystachio REPL', local=locals()) 7 | -------------------------------------------------------------------------------- /pystachio/__init__.py: -------------------------------------------------------------------------------- 1 | __author__ = 'Brian Wickman' 2 | __version__ = '0.8.3' 3 | __license__ = 'MIT' 4 | 5 | 6 | import sys 7 | 8 | from .base import Environment 9 | from .basic import Boolean, Enum, Float, Integer, String 10 | from .choice import Choice 11 | from .composite import Default, Empty, Required, Struct 12 | from .container import List, Map 13 | from .naming import Namable, Ref 14 | from .parsing import MustacheParser 15 | from .typing import Type, TypeCheck, TypeFactory 16 | -------------------------------------------------------------------------------- /pystachio/base.py: -------------------------------------------------------------------------------- 1 | import copy 2 | from pprint import pformat 3 | 4 | from .naming import Namable, Ref 5 | from .parsing import MustacheParser 6 | from .typing import TypeCheck 7 | 8 | 9 | class Environment(Namable): 10 | """ 11 | A mount table for Refs pointing to Objects or arbitrary string substitutions. 12 | """ 13 | __slots__ = ('_table',) 14 | 15 | @staticmethod 16 | def wrap(value): 17 | if isinstance(value, dict): 18 | return Environment(value) 19 | elif isinstance(value, (Environment, Object)): 20 | return value 21 | else: 22 | if isinstance(value, (int, float, str)): 23 | return str(value) 24 | else: 25 | raise ValueError( 26 | 'Environment values must be strings, numbers, Objects or other Environments. ' 27 | 'Got %s instead.' % type(value)) 28 | 29 | def _assimilate_dictionary(self, d): 30 | for key, val in d.items(): 31 | val = Environment.wrap(val) 32 | rkey = Ref.wrap(key) 33 | if isinstance(val, Environment): 34 | for vkey, vval in val._table.items(): 35 | self._table[rkey + vkey] = vval 36 | else: 37 | self._table[rkey] = val 38 | 39 | def _assimilate_table(self, mt): 40 | for key, val in mt._table.items(): 41 | self._table[key] = val 42 | 43 | def __init__(self, *dicts, **kw): 44 | self._table = {} 45 | for d in list(dicts) + [kw]: 46 | if isinstance(d, dict): 47 | self._assimilate_dictionary(d) 48 | elif isinstance(d, Environment): 49 | self._assimilate_table(d) 50 | else: 51 | raise ValueError("Environment expects dict or Environment, got %s" % repr(d)) 52 | 53 | def find(self, ref): 54 | if ref in self._table: 55 | return self._table[ref] 56 | targets = [key for key in self._table if Ref.subscope(key, ref)] 57 | if not targets: 58 | raise Namable.NotFound(self, ref) 59 | else: 60 | for key in sorted(targets, reverse=True): 61 | scope = self._table[key] 62 | if not isinstance(scope, Namable): 63 | continue 64 | subscope = Ref.subscope(key, ref) 65 | # If subscope is empty, then we should've found it in the ref table. 66 | assert not subscope.is_empty() 67 | try: 68 | resolved = scope.find(subscope) 69 | return resolved 70 | except Namable.Error: 71 | continue 72 | raise Namable.NotFound(self, ref) 73 | 74 | def __repr__(self): 75 | return 'Environment(%s)' % pformat(self._table) 76 | 77 | 78 | class Object(object): 79 | """ 80 | Object base class, encapsulating a set of variable bindings scoped to this object. 81 | """ 82 | __slots__ = ('_scopes',) 83 | 84 | class CoercionError(ValueError): 85 | def __init__(self, src, dst, message=None): 86 | error = "Cannot coerce '%s' to %s" % (src, dst.__name__) 87 | ValueError.__init__(self, '%s: %s' % (error, message) if message else error) 88 | 89 | class InterpolationError(Exception): pass 90 | 91 | @classmethod 92 | def checker(cls, obj): 93 | raise NotImplementedError 94 | 95 | def __init__(self): 96 | self._scopes = () 97 | 98 | def get(self): 99 | raise NotImplementedError 100 | 101 | def __hash__(self): 102 | si, _ = self.interpolate() 103 | return hash(si.get()) 104 | 105 | def copy(self): 106 | """ 107 | Return a copy of this object. 108 | """ 109 | self_copy = self.dup() 110 | self_copy._scopes = copy.copy(self._scopes) 111 | return self_copy 112 | 113 | @staticmethod 114 | def translate_to_scopes(*args, **kw): 115 | scopes = [arg if isinstance(arg, Namable) else Environment.wrap(arg) 116 | for arg in args] 117 | if kw: 118 | scopes.append(Environment(kw)) 119 | return tuple(scopes) 120 | 121 | def bind(self, *args, **kw): 122 | """ 123 | Bind environment variables into this object's scope. 124 | """ 125 | new_self = self.copy() 126 | new_scopes = Object.translate_to_scopes(*args, **kw) 127 | new_self._scopes = tuple(reversed(new_scopes)) + new_self._scopes 128 | return new_self 129 | 130 | def in_scope(self, *args, **kw): 131 | """ 132 | Scope this object to a parent environment (like bind but reversed.) 133 | """ 134 | new_self = self.copy() 135 | new_scopes = Object.translate_to_scopes(*args, **kw) 136 | new_self._scopes = new_self._scopes + new_scopes 137 | return new_self 138 | 139 | def scopes(self): 140 | return self._scopes 141 | 142 | def check(self): 143 | """ 144 | Type check this object. 145 | """ 146 | try: 147 | si, uninterp = self.interpolate() 148 | # TODO(wickman) This should probably be pushed out to the interpolate leaves. 149 | except (Object.CoercionError, MustacheParser.Uninterpolatable) as e: 150 | return TypeCheck(False, "Unable to interpolate: %s" % e) 151 | return self.checker(si) 152 | 153 | def __ne__(self, other): 154 | return not (self == other) 155 | 156 | def __mod__(self, namable): 157 | if isinstance(namable, dict): 158 | namable = Environment.wrap(namable) 159 | interp, _ = self.bind(namable).interpolate() 160 | return interp 161 | 162 | def interpolate(self): 163 | """ 164 | Interpolate this object in the context of the Object's environment. 165 | 166 | Should return a 2-tuple: 167 | The object with as much interpolated as possible. 168 | The remaining unbound Refs necessary to fully interpolate the object. 169 | 170 | If the object is fully interpolated, it should be typechecked prior to 171 | return. 172 | """ 173 | raise NotImplementedError 174 | -------------------------------------------------------------------------------- /pystachio/basic.py: -------------------------------------------------------------------------------- 1 | from .base import Object 2 | from .parsing import MustacheParser 3 | from .typing import Type, TypeCheck, TypeFactory, TypeMetaclass 4 | 5 | 6 | class SimpleObject(Object, Type): 7 | """ 8 | A simply-valued (unnamable) object. 9 | """ 10 | __slots__ = ('_value',) 11 | 12 | def __init__(self, value): 13 | self._value = value 14 | super(SimpleObject, self).__init__() 15 | 16 | def get(self): 17 | return self._value 18 | 19 | def dup(self): 20 | return self.__class__(self._value) 21 | 22 | def _my_cmp(self, other): 23 | if self.__class__ != other.__class__: 24 | return -1 25 | si, _ = self.interpolate() 26 | oi, _ = other.interpolate() 27 | if si._value < oi._value: 28 | return -1 29 | elif si._value > oi._value: 30 | return 1 31 | else: 32 | return 0 33 | 34 | def __hash__(self): 35 | return hash(self._value) 36 | 37 | def __eq__(self, other): 38 | return self._my_cmp(other) == 0 39 | 40 | def __lt__(self, other): 41 | return self._my_cmp(other) == -1 42 | 43 | def __gt__(self, other): 44 | return self._my_cmp(other) == 1 45 | 46 | def __le__(self, other): 47 | return self._my_cmp(other) <= 0 48 | 49 | def __ge__(self, other): 50 | return self._my_cmp(other) >= 0 51 | 52 | def __unicode__(self): 53 | si, _ = self.interpolate() 54 | return unicode(si._value) 55 | 56 | def __str__(self): 57 | si, _ = self.interpolate() 58 | return str(si._value) 59 | 60 | def __repr__(self): 61 | return '%s(%s)' % (self.__class__.__name__, str(self)) 62 | 63 | def interpolate(self): 64 | if not isinstance(self._value, str): 65 | return self.__class__(self.coerce(self._value)), [] 66 | else: 67 | joins, unbound = MustacheParser.resolve(self._value, *self.scopes()) 68 | if unbound: 69 | return self.__class__(joins), unbound 70 | else: 71 | # XXX 72 | self_copy = self.copy() 73 | self_copy._value = self_copy.coerce(joins) 74 | return self_copy, unbound 75 | 76 | @classmethod 77 | def type_factory(cls): 78 | return cls.__name__ 79 | 80 | @classmethod 81 | def type_parameters(cls): 82 | return () 83 | 84 | 85 | class String(SimpleObject): 86 | @classmethod 87 | def checker(cls, obj): 88 | assert isinstance(obj, String) 89 | if isinstance(obj._value, str): 90 | return TypeCheck.success() 91 | else: 92 | # TODO(wickman) Perhaps we should mark uninterpolated Mustache objects as 93 | # intrinsically non-stringy, because String will never typecheck false given 94 | # its input constraints. 95 | return TypeCheck.failure("%s not a string" % repr(obj._value)) 96 | 97 | @classmethod 98 | def coerce(cls, value): 99 | if not isinstance(value, (str, int, float)): 100 | raise cls.CoercionError(value, cls) 101 | return str(value) 102 | 103 | 104 | class StringFactory(TypeFactory): 105 | PROVIDES = 'String' 106 | @staticmethod 107 | def create(type_dict, *type_parameters): 108 | return String 109 | 110 | 111 | class Integer(SimpleObject): 112 | @classmethod 113 | def checker(cls, obj): 114 | assert isinstance(obj, Integer) 115 | if isinstance(obj._value, int): 116 | return TypeCheck.success() 117 | else: 118 | return TypeCheck.failure("%s not an integer" % repr(obj._value)) 119 | 120 | @classmethod 121 | def coerce(cls, value): 122 | if not isinstance(value, (str, int, float)): 123 | raise cls.CoercionError(value, cls) 124 | try: 125 | return int(value) 126 | except ValueError: 127 | raise cls.CoercionError(value, cls) 128 | 129 | 130 | class IntegerFactory(TypeFactory): 131 | PROVIDES = 'Integer' 132 | @staticmethod 133 | def create(type_dict, *type_parameters): 134 | return Integer 135 | 136 | 137 | class Float(SimpleObject): 138 | @classmethod 139 | def checker(cls, obj): 140 | assert isinstance(obj, Float) 141 | if isinstance(obj._value, (int, float)): 142 | return TypeCheck.success() 143 | else: 144 | return TypeCheck.failure("%s not a float" % repr(obj._value)) 145 | 146 | @classmethod 147 | def coerce(cls, value): 148 | if not isinstance(value, (str, int, float)): 149 | raise cls.CoercionError(value, cls) 150 | try: 151 | return float(value) 152 | except ValueError: 153 | raise cls.CoercionError(value, cls) 154 | 155 | class FloatFactory(TypeFactory): 156 | PROVIDES = 'Float' 157 | @staticmethod 158 | def create(type_dict, *type_parameters): 159 | return Float 160 | 161 | 162 | class Boolean(SimpleObject): 163 | @classmethod 164 | def checker(cls, obj): 165 | assert isinstance(obj, Boolean) 166 | if isinstance(obj._value, bool): 167 | return TypeCheck.success() 168 | else: 169 | return TypeCheck.failure("%s not a boolean" % repr(obj._value)) 170 | 171 | @classmethod 172 | def coerce(cls, value): 173 | if not isinstance(value, (bool, str, int, float)): 174 | raise cls.CoercionError(value, cls) 175 | 176 | if isinstance(value, bool): 177 | return value 178 | elif isinstance(value, str): 179 | if value.lower() in ("true", "1"): 180 | return True 181 | elif value.lower() in ("false", "0"): 182 | return False 183 | else: 184 | raise cls.CoercionError(value, cls) 185 | else: 186 | return bool(value) 187 | 188 | class BooleanFactory(TypeFactory): 189 | PROVIDES = 'Boolean' 190 | @staticmethod 191 | def create(type_dict, *type_parameters): 192 | return Boolean 193 | 194 | 195 | class EnumContainer(SimpleObject): 196 | def __init__(self, value): 197 | stringish = String(value) 198 | _, refs = stringish.interpolate() 199 | if not refs and value not in self.VALUES: 200 | raise ValueError('%s only accepts the following values: %s' % ( 201 | self.__class__.__name__, ', '.join(self.VALUES))) 202 | super(EnumContainer, self).__init__(value) 203 | 204 | @classmethod 205 | def checker(cls, obj): 206 | assert isinstance(obj, EnumContainer) 207 | if isinstance(obj._value, str) and obj._value in cls.VALUES: 208 | return TypeCheck.success() 209 | else: 210 | return TypeCheck.failure("%s not in the enumeration (%s)" % (repr(obj._value), 211 | ', '.join(cls.VALUES))) 212 | 213 | @classmethod 214 | def coerce(cls, value): 215 | if not isinstance(value, str) or value not in cls.VALUES: 216 | raise cls.CoercionError(value, cls, '%s is not one of %s' % ( 217 | value, ', '.join(cls.VALUES))) 218 | return str(value) 219 | 220 | @classmethod 221 | def type_factory(cls): 222 | return 'Enum' 223 | 224 | @classmethod 225 | def type_parameters(cls): 226 | return (cls.__name__, cls.VALUES) 227 | 228 | 229 | class EnumFactory(TypeFactory): 230 | PROVIDES = 'Enum' 231 | 232 | @staticmethod 233 | def create(type_dict, *type_parameters): 234 | """ 235 | EnumFactory.create(*type_parameters) expects: 236 | enumeration name, (enumeration values) 237 | """ 238 | name, values = type_parameters 239 | assert isinstance(values, (list, tuple)) 240 | for value in values: 241 | assert isinstance(value, str) 242 | return TypeMetaclass(str(name), (EnumContainer,), { 'VALUES': values }) 243 | 244 | 245 | def Enum(*stuff): 246 | # TODO(wickman) Check input 247 | if len(stuff) == 2 and isinstance(stuff[0], str) and ( 248 | isinstance(stuff[1], (list, tuple))): 249 | name, values = stuff 250 | return TypeFactory.new({}, EnumFactory.PROVIDES, name, values) 251 | else: 252 | return TypeFactory.new({}, EnumFactory.PROVIDES, 'Enum_' + '_'.join(stuff), stuff) 253 | -------------------------------------------------------------------------------- /pystachio/choice.py: -------------------------------------------------------------------------------- 1 | # Choice types: types that can take one of a group of selected types. 2 | from .base import Object 3 | from .typing import Type, TypeCheck, TypeFactory, TypeMetaclass 4 | 5 | 6 | class ChoiceFactory(TypeFactory): 7 | """A Pystachio type representing a value which can be one of several 8 | different types. 9 | For example, a field which could be either an integer, or an integer 10 | expression (where IntegerExpression is a struct type) could be written 11 | Choice("IntOrExpr", (Integer, IntegerExpression)) 12 | """ 13 | PROVIDES = 'Choice' 14 | 15 | @staticmethod 16 | def create(type_dict, *type_parameters): 17 | """ 18 | type_parameters should be: 19 | (name, (alternative1, alternative2, ...)) 20 | where name is a string, and the alternatives are all valid serialized 21 | types. 22 | """ 23 | assert len(type_parameters) == 2 24 | name = type_parameters[0] 25 | alternatives = type_parameters[1] 26 | assert isinstance(name, str) 27 | assert isinstance(alternatives, (list, tuple)) 28 | choice_types = [] 29 | for c in alternatives: 30 | choice_types.append(TypeFactory.new(type_dict, *c)) 31 | return TypeMetaclass(str(name), (ChoiceContainer,), {'CHOICES': choice_types, 'TYPE_PARAMETERS': (str(name), tuple(t.serialize_type() for t in choice_types)) 32 | }) 33 | 34 | 35 | class ChoiceContainer(Object, Type): 36 | """The inner implementation of a choice type value. 37 | 38 | This just stores a value, and then tries to coerce it into one of the alternatives when 39 | it's checked or interpolated. 40 | """ 41 | __slots__ = ('_value',) 42 | 43 | def __init__(self, val): 44 | super(ChoiceContainer, self).__init__() 45 | self._value = val 46 | 47 | def get(self): 48 | return self.unwrap().get() 49 | 50 | def unwrap(self): 51 | """Get the Pystachio value that's wrapped in this choice.""" 52 | return self.interpolate()[0] 53 | 54 | def dup(self): 55 | return self.__class__(self._value) 56 | 57 | def __hash__(self): 58 | return hash(self.get()) 59 | 60 | def __unicode__(self): 61 | return unicode(self.unwrap()) 62 | 63 | def __str__(self): 64 | return str(self.unwrap()) 65 | 66 | def __repr__(self): 67 | return '%s(%s)' % (self.__class__.__name__, 68 | repr(self._value)) 69 | 70 | def __eq__(self, other): 71 | if not isinstance(other, ChoiceContainer): 72 | return False 73 | if len(self.CHOICES) != len(other.CHOICES): 74 | return False 75 | for myalt, otheralt in zip(self.CHOICES, other.CHOICES): 76 | if myalt.serialize_type() != otheralt.serialize_type(): 77 | return False 78 | si, _ = self.interpolate() 79 | oi, _ = other.interpolate() 80 | return si == oi 81 | 82 | def _unwrap(self, ret_fun, err_fun): 83 | """Iterate over the options in the choice type, and try to perform some 84 | action on them. If the action fails (returns None or raises either CoercionError 85 | or ValueError), then it goes on to the next type. 86 | Args: 87 | ret_fun: a function that takes a wrapped option value, and either returns a successful 88 | return value or fails. 89 | err_fun: a function that takes the unwrapped value of this choice, and generates 90 | an appropriate error. 91 | Returns: the return value from a successful invocation of ret_fun on one of the 92 | type options. If no invocation fails, then returns the value of invoking err_fun. 93 | """ 94 | for opt in self.CHOICES: 95 | if isinstance(self._value, opt): 96 | return ret_fun(self._value) 97 | else: 98 | try: 99 | o = opt(self._value) 100 | ret = ret_fun(o) 101 | if ret: 102 | return ret 103 | except (self.CoercionError, ValueError): 104 | pass 105 | return err_fun(self._value) 106 | 107 | def check(self): 108 | # Try each of the options in sequence: 109 | # There are three cases for matching depending on the value: 110 | # (1) It's a pystachio value, and its type is the type alternative. Then typecheck 111 | # succeeds. 112 | # (2) It's a pystachio value, but its type is not the current alternative. Then the 113 | # typecheck proceeds to the next alternative. 114 | # (3) It's not a pystachio value. Then we try to coerce it to the type alternative. 115 | # If it succeeds, then the typecheck succeeds. Otherwise, it proceeds to the next 116 | # type alternative. 117 | # If none of the type alternatives succeed, then the check fails. match 118 | def _check(v): 119 | tc = v.in_scope(*self.scopes()).check() 120 | if tc.ok(): 121 | return tc 122 | 123 | def _err(v): 124 | return TypeCheck.failure( 125 | "%s typecheck failed: value %s did not match any of its alternatives" % 126 | (self.__class__.__name__, v)) 127 | 128 | return self._unwrap(_check, _err) 129 | 130 | def interpolate(self): 131 | def _inter(v): 132 | return v.in_scope(*self.scopes()).interpolate() 133 | 134 | def _err(v): 135 | raise self.CoercionError(self._value, self.__class__) 136 | 137 | return self._unwrap(_inter, _err) 138 | 139 | @classmethod 140 | def type_factory(cls): 141 | return 'Choice' 142 | 143 | @classmethod 144 | def type_parameters(cls): 145 | return cls.TYPE_PARAMETERS 146 | 147 | @classmethod 148 | def serialize_type(cls): 149 | return (cls.type_factory(),) + cls.type_parameters() 150 | 151 | 152 | def Choice(*args): 153 | """Helper function for creating new choice types. 154 | This can be called either as: 155 | Choice(Name, [Type1, Type2, ...]) 156 | or: 157 | Choice([Type1, Type2, ...]) 158 | In the latter case, the name of the new type will be autogenerated, and will 159 | look like "Choice_Type1_Type2". 160 | """ 161 | if len(args) == 2: 162 | name, alternatives = args 163 | else: 164 | name = "Choice_" + "_".join(a.__name__ for a in args[0]) 165 | alternatives = args[0] 166 | assert isinstance(name, str) 167 | assert all(issubclass(t, Type) for t in alternatives) 168 | return TypeFactory.new({}, ChoiceFactory.PROVIDES, name, 169 | tuple(t.serialize_type() for t in alternatives)) 170 | -------------------------------------------------------------------------------- /pystachio/composite.py: -------------------------------------------------------------------------------- 1 | import copy 2 | import json 3 | from collections.abc import Mapping 4 | from inspect import isclass 5 | 6 | from .base import Environment, Object 7 | from .naming import Namable, frozendict 8 | from .typing import Type, TypeCheck, TypeFactory, TypeMetaclass 9 | 10 | 11 | class Empty(object): 12 | """The Empty sentinel representing an unspecified field.""" 13 | pass 14 | 15 | 16 | class TypeSignature(object): 17 | """ 18 | Type metadata for composite type schemas. 19 | """ 20 | 21 | def __init__(self, cls, required=False, default=Empty): 22 | """Create an instance of a type signature. 23 | Args: 24 | cls (Class): the "type" of the object this signature represents. 25 | required (bool): 26 | default(object): an instance of the type for a default value. This 27 | should be either an instance of cls or something coercable to cls. 28 | """ 29 | assert isclass(cls) 30 | assert issubclass(cls, Object) 31 | if default is not Empty and not isinstance(default, cls): 32 | self._default = cls(default) 33 | else: 34 | self._default = default 35 | self._cls = cls 36 | self._required = required 37 | 38 | def serialize(self): 39 | return (self.required, 40 | self.default.get() if not self.empty else (), 41 | self.empty, 42 | self.klazz.serialize_type()) 43 | 44 | @staticmethod 45 | def deserialize(sig, type_dict): 46 | req, default, empty, klazz_schema = sig 47 | real_class = TypeFactory.new(type_dict, *klazz_schema) 48 | if not empty: 49 | return TypeSignature(real_class, default=real_class(default), required=req) 50 | else: 51 | return TypeSignature(real_class, required=req) 52 | 53 | def __hash__(self): 54 | return hash( 55 | self.klazz.serialize_type(), 56 | self.required, 57 | self.default, 58 | self.empty) 59 | 60 | def __eq__(self, other): 61 | return (self.klazz.serialize_type() == other.klazz.serialize_type() and 62 | self.required == other.required and 63 | self.default == other.default and 64 | self.empty == other.empty) 65 | 66 | def __ne__(self, other): 67 | return not (self == other) 68 | 69 | def __repr__(self): 70 | return 'TypeSignature(%s, required: %s, default: %s, empty: %s)' % ( 71 | self.klazz.__name__, self.required, self.default, self.empty) 72 | 73 | @property 74 | def klazz(self): 75 | return self._cls 76 | 77 | @property 78 | def required(self): 79 | return self._required 80 | 81 | @property 82 | def default(self): 83 | return self._default 84 | 85 | @property 86 | def empty(self): 87 | return self._default is Empty 88 | 89 | @staticmethod 90 | def wrap(sig): 91 | """Convert a Python class into a type signature.""" 92 | if isclass(sig) and issubclass(sig, Object): 93 | return TypeSignature(sig) 94 | elif isinstance(sig, TypeSignature): 95 | return sig 96 | 97 | 98 | def Required(cls): 99 | """ 100 | Helper to make composite types read succintly. Wrap a type and make its 101 | specification required during type-checking of composite types. 102 | """ 103 | return TypeSignature(cls, required=True) 104 | 105 | 106 | def Default(cls, default): 107 | """ 108 | Helper to make composite types read succintly. Wrap a type and assign it a 109 | default if it is unspecified in the construction of the composite type. 110 | """ 111 | return TypeSignature(cls, required=False, default=default) 112 | 113 | 114 | class StructFactory(TypeFactory): 115 | PROVIDES = 'Struct' 116 | 117 | @staticmethod 118 | def create(type_dict, *type_parameters, class_cell=None): 119 | """ 120 | StructFactory.create(*type_parameters) expects: 121 | 122 | class name, 123 | ((binding requirement1,), 124 | (binding requirement2, bound_to_scope), 125 | ...), 126 | ((attribute_name1, attribute_sig1 (serialized)), 127 | (attribute_name2, attribute_sig2 ...), 128 | ... 129 | (attribute_nameN, ...)) 130 | class_cell inherited from StructMetaclass 131 | """ 132 | name, parameters = type_parameters 133 | for param in parameters: 134 | assert isinstance(param, tuple) 135 | typemap = dict((attr, TypeSignature.deserialize(param, type_dict)) 136 | for attr, param in parameters) 137 | attributes = {'TYPEMAP': typemap, 'TYPE_PARAMETERS': (str(name), tuple(sorted([(attr, sig.serialize()) for attr, sig in typemap.items()])))} 138 | if class_cell: 139 | attributes['__classcell__'] = class_cell 140 | return TypeMetaclass(str(name), (Structural,), attributes) 141 | 142 | 143 | class StructMetaclass(type): 144 | """ 145 | Schema-extracting metaclass for Struct objects. 146 | """ 147 | @staticmethod 148 | def attributes_to_parameters(attributes): 149 | parameters = [] 150 | for attr_name, attr_value in attributes.items(): 151 | sig = TypeSignature.wrap(attr_value) 152 | if sig: 153 | parameters.append((attr_name, sig.serialize())) 154 | return tuple(parameters) 155 | 156 | def __new__(mcs, name, parents, attributes): 157 | if any(parent.__name__ == 'Struct' for parent in parents): 158 | type_parameters = StructMetaclass.attributes_to_parameters(attributes) 159 | return TypeFactory.new({}, 'Struct', name, type_parameters, class_cell=attributes.pop('__classcell__', None)) 160 | else: 161 | return type.__new__(mcs, name, parents, attributes) 162 | 163 | class IsNotMappingError(ValueError): 164 | """Raised when a composite argument is not a Mapping""" 165 | 166 | def __init__(self, arg): 167 | self.arg = arg 168 | 169 | def __repr__(self): 170 | return 'IsNotMappingError({!r})'.format(self.arg) 171 | 172 | StructMetaclassWrapper = StructMetaclass('StructMetaclassWrapper', (object,), {}) 173 | class Structural(Object, Type, Namable): 174 | """A Structural base type for composite objects.""" 175 | __slots__ = ('_schema_data',) 176 | 177 | def __init__(self, *args, **kw): 178 | self._schema_data = frozendict((attr, value.default) for (attr, value) in self.TYPEMAP.items()) 179 | for arg in args: 180 | if not isinstance(arg, Mapping): 181 | raise IsNotMappingError(arg) 182 | self._update_schema_data(**arg) 183 | self._update_schema_data(**copy.copy(kw)) 184 | super(Structural, self).__init__() 185 | 186 | def get(self): 187 | return frozendict((k, v.get()) for k, v in self._schema_data.items() if v is not Empty) 188 | 189 | def _process_schema_attribute(self, attr, value): 190 | if attr not in self.TYPEMAP: 191 | raise AttributeError('Unknown schema attribute %s' % attr) 192 | schema_type = self.TYPEMAP[attr] 193 | if value is Empty: 194 | return Empty 195 | elif isinstance(value, schema_type.klazz): 196 | return value 197 | else: 198 | return schema_type.klazz(value) 199 | 200 | def _update_schema_data(self, **kw): 201 | for attr, value in kw.items(): 202 | self._schema_data[attr] = self._process_schema_attribute(attr, value) 203 | 204 | def dup(self): 205 | return self.__class__(**self._schema_data) 206 | 207 | def __call__(self, **kw): 208 | new_self = self.copy() 209 | new_self._update_schema_data(**copy.copy(kw)) 210 | return new_self 211 | 212 | def __hash__(self): 213 | return hash(self.get()) 214 | 215 | def __eq__(self, other): 216 | if not isinstance(other, Structural): return False 217 | if self.TYPEMAP != other.TYPEMAP: return False 218 | si = self.interpolate() 219 | oi = other.interpolate() 220 | return si[0]._schema_data == oi[0]._schema_data 221 | 222 | def __repr__(self): 223 | si, _ = self.interpolate() 224 | return '%s(%s)' % ( 225 | self.__class__.__name__, 226 | (',\n%s' % (' ' * (len(self.__class__.__name__) + 1))).join( 227 | '%s=%s' % (key, val) for key, val in si._schema_data.items() if val is not Empty) 228 | ) 229 | 230 | def __getattr__(self, attr): 231 | if not hasattr(self, 'TYPEMAP'): 232 | raise AttributeError 233 | 234 | if attr.startswith('has_'): 235 | if attr[4:] in self.TYPEMAP: 236 | return lambda: self._schema_data[attr[4:]] != Empty 237 | 238 | if attr not in self.TYPEMAP: 239 | raise AttributeError("%s has no attribute %s" % (self.__class__.__name__, attr)) 240 | 241 | return lambda: self.interpolate_key(attr) 242 | 243 | def check(self): 244 | scopes = self.scopes() 245 | for name, signature in self.TYPEMAP.items(): 246 | if self._schema_data[name] is Empty and signature.required: 247 | return TypeCheck.failure('%s[%s] is required.' % (self.__class__.__name__, name)) 248 | elif self._schema_data[name] is not Empty: 249 | type_check = self._schema_data[name].in_scope(*scopes).check() 250 | if type_check.ok(): 251 | continue 252 | else: 253 | return TypeCheck.failure('%s[%s] failed: %s' % (self.__class__.__name__, name, 254 | type_check.message())) 255 | return TypeCheck.success() 256 | 257 | @classmethod 258 | def _cast_scopes_to_child(cls, scopes): 259 | return tuple(Environment({'super': scope}) for scope in scopes) 260 | 261 | def _self_scope(self): 262 | return Environment(dict((key, value) for (key, value) in self._schema_data.items() 263 | if value is not Empty)) 264 | 265 | def scopes(self): 266 | self_scope = self._self_scope() 267 | return (Environment({'self': self_scope}), self_scope,) + self._scopes + ( 268 | self._cast_scopes_to_child(self._scopes)) 269 | 270 | def interpolate(self): 271 | unbound = set() 272 | interpolated_schema_data = {} 273 | scopes = self.scopes() 274 | for key, value in self._schema_data.items(): 275 | if value is Empty: 276 | interpolated_schema_data[key] = Empty 277 | else: 278 | vinterp, vunbound = value.in_scope(*scopes).interpolate() 279 | unbound.update(vunbound) 280 | interpolated_schema_data[key] = vinterp 281 | return self.__class__(**interpolated_schema_data), list(unbound) 282 | 283 | def interpolate_key(self, attribute): 284 | if self._schema_data[attribute] is Empty: 285 | return Empty 286 | vinterp, _ = self._schema_data[attribute].in_scope(*self.scopes()).interpolate() 287 | return self._process_schema_attribute(attribute, vinterp) 288 | 289 | @classmethod 290 | def type_factory(cls): 291 | return 'Struct' 292 | 293 | @classmethod 294 | def type_parameters(cls): 295 | return cls.TYPE_PARAMETERS 296 | 297 | @classmethod 298 | def _filter_against_schema(cls, values): 299 | result = {} 300 | for key, val in values.items(): 301 | if key not in cls.TYPEMAP: 302 | continue 303 | if issubclass(cls.TYPEMAP[key].klazz, Structural): 304 | result[key] = cls.TYPEMAP[key].klazz._filter_against_schema(val) 305 | else: 306 | result[key] = val 307 | return result 308 | 309 | @classmethod 310 | def json_load(cls, fp, strict=False): 311 | return cls(json.load(fp) if strict else cls._filter_against_schema(json.load(fp))) 312 | 313 | @classmethod 314 | def json_loads(cls, json_string, strict=False): 315 | return cls(json.loads(json_string) if strict 316 | else cls._filter_against_schema(json.loads(json_string))) 317 | 318 | def json_dump(self, fp): 319 | d, _ = self.interpolate() 320 | return json.dump(d.get(), fp) 321 | 322 | def json_dumps(self): 323 | d, _ = self.interpolate() 324 | return json.dumps(d.get()) 325 | 326 | def find(self, ref): 327 | if not ref.is_dereference(): 328 | raise Namable.NamingError(self, ref) 329 | name = ref.action().value 330 | if name not in self.TYPEMAP or self._schema_data[name] is Empty: 331 | raise Namable.NotFound(self, ref) 332 | else: 333 | namable = self._schema_data[name] 334 | if ref.rest().is_empty(): 335 | return namable.in_scope(*self.scopes()) 336 | else: 337 | if not isinstance(namable, Namable): 338 | raise Namable.Unnamable(namable) 339 | else: 340 | return namable.in_scope(*self.scopes()).find(ref.rest()) 341 | 342 | 343 | class Struct(StructMetaclassWrapper, Structural): 344 | """ 345 | Schema-based composite objects, e.g. 346 | 347 | class Employee(Struct): 348 | first = Required(String) 349 | last = Required(String) 350 | email = Required(String) 351 | phone = String 352 | 353 | Employee(first = "brian", last = "wickman", email = "wickman@twitter.com").check() 354 | 355 | They're purely functional data structures and behave more like functors. 356 | In other words they're immutable: 357 | 358 | >>> brian = Employee(first = "brian") 359 | >>> brian(last = "wickman") 360 | Employee(last=String(wickman), first=String(brian)) 361 | >>> brian 362 | Employee(first=String(brian)) 363 | """ 364 | pass 365 | -------------------------------------------------------------------------------- /pystachio/config.py: -------------------------------------------------------------------------------- 1 | import os 2 | from functools import reduce 3 | 4 | try: 5 | import pkg_resources 6 | except ImportError: 7 | pkg_resources = None 8 | 9 | 10 | def relativize(from_path, include_path): 11 | return os.path.join(os.path.dirname(from_path), include_path) 12 | 13 | 14 | def exec_function(ast, globals_map): 15 | locals_map = globals_map 16 | exec(ast, globals_map, locals_map) 17 | return locals_map 18 | 19 | 20 | class ConfigContext(object): 21 | ROOT = '' 22 | 23 | # Make JSON-friendly keys -- since JSON can't encode tuples as dictionary keys. 24 | @classmethod 25 | def key(cls, from_path, include_string): 26 | return '\0'.join([from_path, include_string]) 27 | 28 | @classmethod 29 | def from_key(cls, key): 30 | return key.split('\0') 31 | 32 | def __init__(self, environment, loadables): 33 | self.environment = environment 34 | self.loadables = loadables 35 | 36 | def compile(self, from_path, include_string, data): 37 | self.loadables[self.key(from_path, include_string)] = data 38 | exec_function(compile(data, include_string, 'exec'), self.environment) 39 | 40 | 41 | class ConfigExecutor(object): 42 | ROOT = ConfigContext.ROOT 43 | 44 | @classmethod 45 | def matches(cls, loadable): 46 | return False 47 | 48 | @classmethod 49 | def get(cls, loadable): 50 | """return the include function and the root include object.""" 51 | raise NotImplementedError 52 | 53 | 54 | class FileExecutor(ConfigExecutor): 55 | @classmethod 56 | def matches(cls, loadable): 57 | return isinstance(loadable, str) and os.path.isfile(loadable) 58 | 59 | @classmethod 60 | def compile_into(cls, context, from_path, config_file): 61 | actual_file = relativize(from_path, config_file) 62 | with open(actual_file) as fp: 63 | context.compile(from_path, config_file, fp.read()) 64 | 65 | @classmethod 66 | def get(cls, loadable): 67 | deposit_stack = [cls.ROOT] 68 | def ast_executor(config_file, context): 69 | from_path = deposit_stack[-1] 70 | actual_file = relativize(from_path, config_file) 71 | deposit_stack.append(actual_file) 72 | cls.compile_into(context, from_path, config_file) 73 | deposit_stack.pop() 74 | return ast_executor, loadable 75 | 76 | 77 | class ResourceExecutor(FileExecutor): 78 | @classmethod 79 | def resource_exists(cls, loadable): 80 | if pkg_resources is None: 81 | return False 82 | module_base, module_file = os.path.split(loadable) 83 | module_base = module_base.replace(os.sep, '.') 84 | try: 85 | return pkg_resources.resource_exists(module_base, module_file) and ( 86 | not pkg_resources.resource_isdir(module_base, module_file)) 87 | except (ValueError, ImportError): 88 | return False 89 | 90 | @classmethod 91 | def matches(cls, loadable): 92 | return isinstance(loadable, str) and cls.resource_exists(loadable) 93 | 94 | @classmethod 95 | def compile_into(cls, context, from_path, config_file): 96 | actual_file = relativize(from_path, config_file) 97 | module_base, module_file = os.path.split(actual_file) 98 | module_base = module_base.replace(os.sep, '.') 99 | context.compile(from_path, config_file, 100 | pkg_resources.resource_string(module_base, module_file)) 101 | 102 | 103 | class LoadableMapExecutor(ConfigExecutor): 104 | @classmethod 105 | def matches(cls, loadable): 106 | return isinstance(loadable, dict) 107 | 108 | @classmethod 109 | def find_root_file(cls, loadable): 110 | for key in loadable.keys(): 111 | from_file, config_file = ConfigContext.from_key(key) 112 | if from_file == cls.ROOT: 113 | return config_file 114 | 115 | @classmethod 116 | def from_filename(cls, stack): 117 | return reduce(relativize, stack, '') 118 | 119 | @classmethod 120 | def get(cls, loadable): 121 | deposit_stack = [cls.ROOT] 122 | 123 | def ast_executor(config_file, context): 124 | from_file = cls.from_filename(deposit_stack) 125 | deposit_stack.append(config_file) 126 | context.compile(from_file, config_file, loadable[ConfigContext.key(from_file, config_file)]) 127 | deposit_stack.pop() 128 | 129 | return ast_executor, cls.find_root_file(loadable) 130 | 131 | 132 | class FilelikeExecutor(ConfigExecutor): 133 | @classmethod 134 | def matches(cls, loadable): 135 | return hasattr(loadable, 'read') and callable(loadable.read) 136 | 137 | @classmethod 138 | def get(cls, loadable): 139 | def ast_executor(config, context): 140 | if config is not loadable: 141 | raise ValueError('You may not include() anything from filelike objects.') 142 | context.compile(cls.ROOT, '' % loadable, loadable.read()) 143 | return ast_executor, loadable 144 | 145 | 146 | class Config(object): 147 | class Error(Exception): pass 148 | class InvalidConfigError(Error): pass 149 | class NotFound(Error): pass 150 | 151 | DEFAULT_SCHEMA = 'from pystachio import *' 152 | EXECUTORS = [ 153 | FileExecutor, 154 | ResourceExecutor, 155 | FilelikeExecutor, 156 | LoadableMapExecutor 157 | ] 158 | ROOT = None 159 | 160 | @classmethod 161 | def choose_executor(cls, obj): 162 | for executor in cls.EXECUTORS: 163 | if executor.matches(obj): 164 | return executor.get(obj) 165 | raise cls.NotFound('Could not load resource %s' % obj) 166 | 167 | @classmethod 168 | def load_schema(cls, environment, schema=None): 169 | exec_function( 170 | compile(schema or cls.DEFAULT_SCHEMA, "", "exec"), environment) 171 | 172 | def __init__(self, loadable, schema=None): 173 | self._environment = {} 174 | self._loadables = {} 175 | self.load_schema(self._environment, schema) 176 | root_executor, initial_config = self.choose_executor(loadable) 177 | context = ConfigContext(self._environment, self._loadables) 178 | self._environment.update(include=lambda fn: root_executor(fn, context)) 179 | try: 180 | root_executor(initial_config, context) 181 | except (SyntaxError, ValueError) as e: 182 | raise self.InvalidConfigError(str(e)) 183 | 184 | @property 185 | def loadables(self): 186 | return self._loadables 187 | 188 | @property 189 | def environment(self): 190 | return self._environment 191 | -------------------------------------------------------------------------------- /pystachio/container.py: -------------------------------------------------------------------------------- 1 | import copy 2 | from collections.abc import Iterable, Mapping, Sequence 3 | from inspect import isclass 4 | 5 | from .base import Object 6 | from .naming import Namable, frozendict 7 | from .typing import Type, TypeCheck, TypeFactory, TypeMetaclass 8 | 9 | 10 | class ListFactory(TypeFactory): 11 | PROVIDES = 'List' 12 | 13 | @staticmethod 14 | def create(type_dict, *type_parameters): 15 | """ 16 | Construct a List containing type 'klazz'. 17 | """ 18 | assert len(type_parameters) == 1 19 | klazz = TypeFactory.new(type_dict, *type_parameters[0]) 20 | assert isclass(klazz) 21 | assert issubclass(klazz, Object) 22 | return TypeMetaclass('%sList' % klazz.__name__, (ListContainer,), {'TYPE': klazz, 'TYPE_PARAMETERS': (klazz.serialize_type(),)}) 23 | 24 | 25 | class ListContainer(Object, Namable, Type): 26 | """ 27 | The List container type. This is the base class for all user-generated 28 | List types. It won't function as-is, since it requires cls.TYPE to be 29 | set to the contained type. If you want a concrete List type, see the 30 | List() function. 31 | """ 32 | __slots__ = ('_values',) 33 | 34 | def __init__(self, vals): 35 | self._values = self._coerce_values(copy.copy(vals)) 36 | super(ListContainer, self).__init__() 37 | 38 | def get(self): 39 | return tuple(v.get() for v in self._values) 40 | 41 | def dup(self): 42 | return self.__class__(self._values) 43 | 44 | def __hash__(self): 45 | return hash(self.get()) 46 | 47 | def __repr__(self): 48 | si, _ = self.interpolate() 49 | return '%s(%s)' % (self.__class__.__name__, 50 | ', '.join(str(v) for v in si._values)) 51 | 52 | def __iter__(self): 53 | si, _ = self.interpolate() 54 | return iter(si._values) 55 | 56 | def __getitem__(self, index_or_slice): 57 | si, _ = self.interpolate() 58 | return si._values[index_or_slice] 59 | 60 | def __contains__(self, item): 61 | si, _ = self.interpolate() 62 | if isinstance(item, self.TYPE): 63 | return item in si._values 64 | else: 65 | return item in si.get() 66 | 67 | def __eq__(self, other): 68 | if not isinstance(other, ListContainer): return False 69 | if self.TYPE.serialize_type() != other.TYPE.serialize_type(): return False 70 | si, _ = self.interpolate() 71 | oi, _ = other.interpolate() 72 | return si._values == oi._values 73 | 74 | @staticmethod 75 | def isiterable(values): 76 | return isinstance(values, Sequence) and not isinstance(values, str) 77 | 78 | def _coerce_values(self, values): 79 | if not ListContainer.isiterable(values): 80 | raise ValueError("ListContainer expects an iterable, got %s" % repr(values)) 81 | def coerced(value): 82 | return value if isinstance(value, self.TYPE) else self.TYPE(value) 83 | return tuple([coerced(v) for v in values]) 84 | 85 | def check(self): 86 | assert ListContainer.isiterable(self._values) 87 | scopes = self.scopes() 88 | for element in self._values: 89 | assert isinstance(element, self.TYPE) 90 | typecheck = element.in_scope(*scopes).check() 91 | if not typecheck.ok(): 92 | return TypeCheck.failure("Element in %s failed check: %s" % (self.__class__.__name__, 93 | typecheck.message())) 94 | return TypeCheck.success() 95 | 96 | def interpolate(self): 97 | unbound = set() 98 | interpolated = [] 99 | scopes = self.scopes() 100 | for element in self._values: 101 | einterp, eunbound = element.in_scope(*scopes).interpolate() 102 | interpolated.append(einterp) 103 | unbound.update(eunbound) 104 | return self.__class__(interpolated), list(unbound) 105 | 106 | def find(self, ref): 107 | if not ref.is_index(): 108 | raise Namable.NamingError(self, ref) 109 | try: 110 | intvalue = int(ref.action().value) 111 | except ValueError: 112 | raise Namable.NamingError(self, ref) 113 | if len(self._values) <= intvalue: 114 | raise Namable.NotFound(self, ref) 115 | else: 116 | namable = self._values[intvalue] 117 | if ref.rest().is_empty(): 118 | return namable.in_scope(*self.scopes()) 119 | else: 120 | if not isinstance(namable, Namable): 121 | raise Namable.Unnamable(namable) 122 | else: 123 | return namable.in_scope(*self.scopes()).find(ref.rest()) 124 | 125 | @classmethod 126 | def type_factory(cls): 127 | return 'List' 128 | 129 | @classmethod 130 | def type_parameters(cls): 131 | return cls.TYPE_PARAMETERS 132 | 133 | List = TypeFactory.wrapper(ListFactory) 134 | 135 | 136 | class MapFactory(TypeFactory): 137 | PROVIDES = 'Map' 138 | 139 | @staticmethod 140 | def create(type_dict, *type_parameters): 141 | assert len(type_parameters) == 2, 'Type parameters: %s' % repr(type_parameters) 142 | key_klazz, value_klazz = type_parameters 143 | key_klazz, value_klazz = (TypeFactory.new(type_dict, *key_klazz), 144 | TypeFactory.new(type_dict, *value_klazz)) 145 | assert isclass(key_klazz) and isclass(value_klazz) 146 | assert issubclass(key_klazz, Object) and issubclass(value_klazz, Object) 147 | return TypeMetaclass('%s%sMap' % (key_klazz.__name__, value_klazz.__name__), (MapContainer,), 148 | {'KEYTYPE': key_klazz, 'VALUETYPE': value_klazz, 'TYPE_PARAMETERS': (key_klazz.serialize_type(), value_klazz.serialize_type())}) 149 | 150 | 151 | # TODO(wickman) Technically it's possible to do the following: 152 | # 153 | # >>> my_map = Map(Boolean,Integer)((True,2), (False,3), (False, 2)) 154 | # >>> my_map 155 | # BooleanIntegerMap(True => 2, False => 3, False => 2) 156 | # >>> my_map.get() 157 | # frozendict({False: 2, True: 2}) 158 | # >>> my_map[True] 159 | # Integer(2) 160 | # >>> my_map.get()[True] 161 | # 2 162 | # we should filter tuples for uniqueness. 163 | class MapContainer(Object, Namable, Type): 164 | """ 165 | The Map container type. This is the base class for all user-generated 166 | Map types. It won't function as-is, since it requires cls.KEYTYPE and 167 | cls.VALUETYPE to be set to the appropriate types. If you want a 168 | concrete Map type, see the Map() function. 169 | 170 | __init__(dict) => translates to list of tuples & sanity checks 171 | __init__(tuple) => sanity checks 172 | """ 173 | __slots__ = ('_map',) 174 | 175 | def __init__(self, *args): 176 | """ 177 | Construct a map. 178 | 179 | Input: 180 | sequence of tuples _or_ a dictionary 181 | """ 182 | if len(args) == 1 and isinstance(args[0], Mapping): 183 | self._map = self._coerce_map(copy.copy(args[0])) 184 | elif all(isinstance(arg, Iterable) and len(arg) == 2 for arg in args): 185 | self._map = self._coerce_tuple(args) 186 | else: 187 | raise ValueError("Unexpected input to MapContainer: %s" % repr(args)) 188 | super(MapContainer, self).__init__() 189 | 190 | def get(self): 191 | return frozendict((k.get(), v.get()) for (k, v) in self._map) 192 | 193 | def _coerce_wrapper(self, key, value): 194 | coerced_key = key if isinstance(key, self.KEYTYPE) else self.KEYTYPE(key) 195 | coerced_value = value if isinstance(value, self.VALUETYPE) else self.VALUETYPE(value) 196 | return (coerced_key, coerced_value) 197 | 198 | def _coerce_map(self, input_map): 199 | return tuple(self._coerce_wrapper(key, value) for key, value in input_map.items()) 200 | 201 | def _coerce_tuple(self, input_tuple): 202 | return tuple(self._coerce_wrapper(key, value) for key, value in input_tuple) 203 | 204 | def __hash__(self): 205 | return hash(self.get()) 206 | 207 | def __iter__(self): 208 | si, _ = self.interpolate() 209 | return (t[0] for t in si._map) 210 | 211 | def __getitem__(self, key): 212 | if not isinstance(key, self.KEYTYPE): 213 | try: 214 | key = self.KEYTYPE(key) 215 | except ValueError: 216 | raise KeyError("%s is not coercable to %s" % self.KEYTYPE.__name__) 217 | # TODO(wickman) The performance of this should be improved. 218 | si, _ = self.interpolate() 219 | for tup in si._map: 220 | if key == tup[0]: 221 | return tup[1] 222 | raise KeyError("%s not found" % key) 223 | 224 | def __contains__(self, item): 225 | try: 226 | self[item] 227 | return True 228 | except KeyError: 229 | return False 230 | 231 | def dup(self): 232 | return self.__class__(*self._map) 233 | 234 | def __repr__(self): 235 | si, _ = self.interpolate() 236 | return '%s(%s)' % (self.__class__.__name__, 237 | ', '.join('%s => %s' % (key, val) for key, val in si._map)) 238 | 239 | def __eq__(self, other): 240 | if not isinstance(other, MapContainer): return False 241 | if self.KEYTYPE.serialize_type() != other.KEYTYPE.serialize_type(): return False 242 | if self.VALUETYPE.serialize_type() != other.VALUETYPE.serialize_type(): return False 243 | si, _ = self.interpolate() 244 | oi, _ = other.interpolate() 245 | return si._map == oi._map 246 | 247 | def check(self): 248 | assert isinstance(self._map, tuple) 249 | scopes = self.scopes() 250 | for key, value in self._map: 251 | assert isinstance(key, self.KEYTYPE) 252 | assert isinstance(value, self.VALUETYPE) 253 | keycheck = key.in_scope(*scopes).check() 254 | valuecheck = value.in_scope(*scopes).check() 255 | if not keycheck.ok(): 256 | return TypeCheck.failure("%s key %s failed check: %s" % (self.__class__.__name__, 257 | key, keycheck.message())) 258 | if not valuecheck.ok(): 259 | return TypeCheck.failure("%s[%s] value %s failed check: %s" % (self.__class__.__name__, 260 | key, value, valuecheck.message())) 261 | return TypeCheck.success() 262 | 263 | def interpolate(self): 264 | unbound = set() 265 | interpolated = [] 266 | scopes = self.scopes() 267 | for key, value in self._map: 268 | kinterp, kunbound = key.in_scope(*scopes).interpolate() 269 | vinterp, vunbound = value.in_scope(*scopes).interpolate() 270 | unbound.update(kunbound) 271 | unbound.update(vunbound) 272 | interpolated.append((kinterp, vinterp)) 273 | return self.__class__(*interpolated), list(unbound) 274 | 275 | def find(self, ref): 276 | if not ref.is_index(): 277 | raise Namable.NamingError(self, ref) 278 | kvalue = self.KEYTYPE(ref.action().value) 279 | scopes = self.scopes() 280 | for key, namable in self._map: 281 | if kvalue == key: 282 | if ref.rest().is_empty(): 283 | return namable.in_scope(*scopes) 284 | else: 285 | if not isinstance(namable, Namable): 286 | raise Namable.Unnamable(namable) 287 | else: 288 | return namable.in_scope(*scopes).find(ref.rest()) 289 | raise Namable.NotFound(self, ref) 290 | 291 | @classmethod 292 | def type_factory(cls): 293 | return 'Map' 294 | 295 | @classmethod 296 | def type_parameters(cls): 297 | return cls.TYPE_PARAMETERS 298 | 299 | Map = TypeFactory.wrapper(MapFactory) 300 | -------------------------------------------------------------------------------- /pystachio/matcher.py: -------------------------------------------------------------------------------- 1 | import copy 2 | import re 3 | 4 | from .naming import Ref 5 | 6 | try: 7 | from itertools import izip_longest as zipl 8 | except ImportError: 9 | from itertools import zip_longest as zipl 10 | 11 | 12 | 13 | class Any(object): 14 | pass 15 | 16 | 17 | class Matcher(object): 18 | """ 19 | Matcher of Ref patterns. 20 | """ 21 | 22 | __slots__ = ('_components', '_resolver') 23 | 24 | class Error(Exception): pass 25 | class NoMatch(Error): pass 26 | 27 | @classmethod 28 | def escape(cls, pattern): 29 | if not pattern.endswith('$'): 30 | pattern = pattern + '$' 31 | if not pattern.startswith('^'): 32 | pattern = '^' + pattern 33 | return re.compile(pattern) 34 | 35 | def __init__(self, root=None): 36 | if root is None: 37 | self._components = [] 38 | return 39 | if not isinstance(root, str) and root is not Any: 40 | raise ValueError('Invalid root match value: %s' % root) 41 | self._components = [Ref.Dereference(self.escape('.*' if root is Any else root))] 42 | 43 | def __extend(self, value): 44 | new_match = Matcher() 45 | new_match._components = copy.copy(self._components) + [value] 46 | return new_match 47 | 48 | def __getattr__(self, pattern): 49 | if pattern == '_': 50 | return lambda pattern: self.__extend(Ref.Dereference(self.escape(pattern))) 51 | elif pattern == 'Any': 52 | return self.__extend(Ref.Dereference(self.escape('.*'))) 53 | else: 54 | return self.__extend(Ref.Dereference(self.escape(pattern))) 55 | 56 | def __getitem__(self, pattern): 57 | if pattern is Any: 58 | return self.__extend(Ref.Index(self.escape('.*'))) 59 | else: 60 | return self.__extend(Ref.Index(self.escape(pattern))) 61 | 62 | def __repr__(self): 63 | return 'Match(%s)' % '+'.join(map(str, self._components)) 64 | 65 | def match(self, pystachio_object): 66 | _, refs = pystachio_object.interpolate() 67 | for ref in refs: 68 | args = [] 69 | zips = list(zipl(self._components, ref.components())) 70 | for pattern, component in zips[:len(self._components)]: 71 | if pattern.__class__ != component.__class__ or not pattern.value.match(component.value): 72 | break 73 | args.append(component.value) 74 | else: 75 | yield tuple(args) 76 | 77 | def __translate(self, match_tuple): 78 | components = [] 79 | for component, match in zip(self._components, match_tuple): 80 | components.append(component.__class__(match)) 81 | return Ref(components) 82 | 83 | def apply(self, binder, pystachio_object): 84 | if not callable(binder): 85 | raise TypeError('binder must be a callable') 86 | for match in self.match(pystachio_object): 87 | pystachio_object = pystachio_object.bind({self.__translate(match): binder(*match)}) 88 | return pystachio_object 89 | -------------------------------------------------------------------------------- /pystachio/naming.py: -------------------------------------------------------------------------------- 1 | from functools import lru_cache 2 | import re 3 | 4 | 5 | class frozendict(dict): 6 | """A hashable dictionary.""" 7 | def __key(self): 8 | return tuple((k, self[k]) for k in sorted(self)) 9 | 10 | def __hash__(self): 11 | return hash(self.__key()) 12 | 13 | def __eq__(self, other): 14 | return self.__key() == other.__key() 15 | 16 | def __ne__(self, other): 17 | return self.__key() != other.__key() 18 | 19 | def __repr__(self): 20 | return 'frozendict(%s)' % dict.__repr__(self) 21 | 22 | 23 | class Namable(object): 24 | """ 25 | An object that can be named/dereferenced. 26 | """ 27 | class Error(Exception): pass 28 | 29 | class Unnamable(Error): 30 | def __init__(self, obj): 31 | super(Namable.Unnamable, self).__init__('Object is not indexable: %s' % 32 | obj.__class__.__name__) 33 | 34 | class NamingError(Error): 35 | def __init__(self, obj, ref): 36 | super(Namable.NamingError, self).__init__('Cannot dereference object %s by %s' % ( 37 | obj.__class__.__name__, ref.action())) 38 | 39 | class NotFound(Error): 40 | def __init__(self, obj, ref): 41 | super(Namable.NotFound, self).__init__('Could not find %s in object %s' % (ref.action().value, 42 | obj.__class__.__name__)) 43 | 44 | def find(self, ref): 45 | """ 46 | Given a ref, return the value referencing that ref. 47 | Raises Namable.NotFound if not found. 48 | Raises Namable.NamingError if try to dereference object in an invalid way. 49 | Raises Namable.Unnamable if try to dereference into an unnamable type. 50 | """ 51 | raise NotImplementedError 52 | 53 | 54 | class Ref(object): 55 | """ 56 | A reference into to a hierarchically named object. 57 | """ 58 | # ref re 59 | # ^[^\d\W]\w*\Z 60 | _DEREF_RE = r'[^\d\W]\w*' 61 | _INDEX_RE = r'[\w\-\./]+' 62 | _REF_RE = re.compile(r'(\.' + _DEREF_RE + r'|\[' + _INDEX_RE + r'\])') 63 | _VALID_START = re.compile(r'[a-zA-Z_]') 64 | _COMPONENT_SEPARATOR = '.' 65 | 66 | class Component(object): 67 | def __init__(self, value): 68 | self._value = value 69 | 70 | @property 71 | def value(self): 72 | return self._value 73 | 74 | def __hash__(self): 75 | return hash(self.value) 76 | 77 | def __eq__(self, other): 78 | return self.__class__ == other.__class__ and self.value == other.value 79 | 80 | def __ne__(self, other): 81 | return not (self == other) 82 | 83 | def __lt__(self, other): 84 | return self.value < other.value 85 | 86 | def __gt__(self, other): 87 | return self.value > other.value 88 | 89 | class Index(Component): 90 | RE = re.compile(r'^[\w\-\./]+$') 91 | 92 | def __repr__(self): 93 | return '[%s]' % self._value 94 | 95 | class Dereference(Component): 96 | RE = re.compile(r'^[^\d\W]\w*$') 97 | 98 | def __repr__(self): 99 | return '.%s' % self._value 100 | 101 | class InvalidRefError(Exception): pass 102 | class UnnamableError(Exception): pass 103 | 104 | @staticmethod 105 | def wrap(value): 106 | if isinstance(value, Ref): 107 | return value 108 | else: 109 | return Ref.from_address(value) 110 | 111 | @staticmethod 112 | @lru_cache(maxsize=128) 113 | def from_address(address): 114 | components = [] 115 | if not address or not isinstance(address, str): 116 | raise Ref.InvalidRefError('Invalid address: %s' % repr(address)) 117 | if not (address.startswith('[') or address.startswith('.')): 118 | if Ref._VALID_START.match(address[0]): 119 | components = Ref.split_components('.' + address) 120 | else: 121 | raise Ref.InvalidRefError(address) 122 | else: 123 | components = Ref.split_components(address) 124 | return Ref(components) 125 | 126 | def __init__(self, components): 127 | self._components = tuple(components) 128 | self._hash = None 129 | 130 | def components(self): 131 | return self._components 132 | 133 | def action(self): 134 | return self._components[0] 135 | 136 | def is_index(self): 137 | return isinstance(self.action(), Ref.Index) 138 | 139 | def is_dereference(self): 140 | return isinstance(self.action(), Ref.Dereference) 141 | 142 | def is_empty(self): 143 | return len(self.components()) == 0 144 | 145 | def rest(self): 146 | return Ref(self.components()[1:]) 147 | 148 | @lru_cache(maxsize=128) 149 | def __add__(self, other): 150 | sc = self.components() 151 | oc = other.components() 152 | return Ref(sc + oc) 153 | 154 | @staticmethod 155 | @lru_cache(maxsize=10000) 156 | def subscope(ref1, ref2): 157 | rc = ref1.components() 158 | sc = ref2.components() 159 | if rc == sc[0:len(rc)]: 160 | if len(sc) > len(rc): 161 | return Ref(sc[len(rc):]) 162 | 163 | def scoped_to(self, ref): 164 | return Ref.subscope(self, ref) 165 | 166 | @staticmethod 167 | def split_components(address): 168 | def map_to_namable(component): 169 | if (component.startswith('[') and component.endswith(']') and 170 | Ref.Index.RE.match(component[1:-1])): 171 | return Ref.Index(component[1:-1]) 172 | elif component.startswith('.') and Ref.Dereference.RE.match(component[1:]): 173 | return Ref.Dereference(component[1:]) 174 | else: 175 | raise Ref.InvalidRefError('Address %s has bad component %s' % (address, component)) 176 | splits = Ref._REF_RE.split(address) 177 | if any(splits[0::2]): 178 | raise Ref.InvalidRefError('Badly formed address %s' % address) 179 | splits = splits[1::2] 180 | return [map_to_namable(spl) for spl in splits] 181 | 182 | def address(self): 183 | joined = ''.join(str(comp) for comp in self._components) 184 | if joined.startswith('.'): 185 | return joined[1:] 186 | else: 187 | return joined 188 | 189 | def __str__(self): 190 | return '{{%s}}' % self.address() 191 | 192 | def __repr__(self): 193 | return 'Ref(%s)' % self.address() 194 | 195 | def __eq__(self, other): 196 | return self.components() == other.components() 197 | 198 | def __ne__(self, other): 199 | return self.components() != other.components() 200 | 201 | @staticmethod 202 | def compare(self, other): 203 | if len(self.components()) < len(other.components()): 204 | return -1 205 | elif len(self.components()) > len(other.components()): 206 | return 1 207 | else: 208 | return (self.components() > other.components()) - (self.components() < other.components()) 209 | 210 | def __lt__(self, other): 211 | return Ref.compare(self, other) == -1 212 | 213 | def __gt__(self, other): 214 | return Ref.compare(self, other) == 1 215 | 216 | def __hash__(self): 217 | if not self._hash: 218 | self._hash = hash(self.components()) 219 | return self._hash 220 | -------------------------------------------------------------------------------- /pystachio/parsing.py: -------------------------------------------------------------------------------- 1 | import re 2 | 3 | from .naming import Namable, Ref 4 | 5 | 6 | class MustacheParser(object): 7 | """ 8 | Split strings on Mustache-style templates: 9 | a {{foo}} bar {{baz}} b => ['a ', Ref('foo'), ' bar ', Ref('baz'), ' b'] 10 | 11 | To suppress parsing of individual tags, you can use {{&foo}} which emits '{{foo}}' 12 | instead of Ref('foo') or Ref('&foo'). As such, template variables cannot 13 | begin with '&'. 14 | """ 15 | 16 | _ADDRESS_DELIMITER = '&' 17 | _MUSTACHE_RE = re.compile(r"{{(%c)?([^{}]+?)\1?}}" % _ADDRESS_DELIMITER) 18 | MAX_ITERATIONS = 100 19 | 20 | class Error(Exception): pass 21 | class Uninterpolatable(Error): pass 22 | 23 | @classmethod 24 | def split(cls, string, keep_aliases=False): 25 | splits = cls._MUSTACHE_RE.split(string) 26 | first_split = splits.pop(0) 27 | outsplits = [first_split] if first_split else [] 28 | assert len(splits) % 3 == 0 29 | for k in range(0, len(splits), 3): 30 | if splits[k] == cls._ADDRESS_DELIMITER: 31 | outsplits.append('{{%s%s}}' % ( 32 | cls._ADDRESS_DELIMITER if keep_aliases else '', 33 | splits[k + 1])) 34 | elif splits[k] is None: 35 | outsplits.append(Ref.from_address(splits[k + 1])) 36 | else: 37 | raise Exception("Unexpected parsing error in Mustache: splits[%s] = '%s'" % ( 38 | k, splits[k])) 39 | if splits[k + 2]: 40 | outsplits.append(splits[k + 2]) 41 | return outsplits 42 | 43 | @classmethod 44 | def join(cls, splits, *namables, found_refs = dict()): 45 | """ 46 | Interpolate strings. 47 | 48 | :params splits: The output of Parser.split(string) 49 | :params namables: A sequence of Namable objects in which the interpolation should take place. 50 | 51 | Returns 2-tuple containing: 52 | joined string, list of unbound object ids (potentially empty) 53 | """ 54 | isplits = [] 55 | unbound = [] 56 | for ref in splits: 57 | if isinstance(ref, Ref): 58 | if ref not in found_refs: 59 | for namable in namables: 60 | try: 61 | found_refs[ref] = namable.find(ref) 62 | break 63 | except Namable.Error: 64 | continue 65 | if ref in found_refs: 66 | isplits.append(found_refs[ref]) 67 | else: 68 | isplits.append(ref) 69 | unbound.append(ref) 70 | else: 71 | isplits.append(ref) 72 | return (''.join(map(str, isplits)), unbound) 73 | 74 | @classmethod 75 | def resolve(cls, stream, *namables): 76 | found_refs = dict() 77 | def iterate(st, keep_aliases=True): 78 | refs = cls.split(st, keep_aliases=keep_aliases) 79 | unbound = [ref for ref in refs if isinstance(ref, Ref)] 80 | repl, interps = cls.join(refs, *namables, found_refs=found_refs) 81 | return repl, interps, unbound 82 | 83 | for _ in range(cls.MAX_ITERATIONS): 84 | stream, interps, unbound = iterate(stream, keep_aliases=True) 85 | if interps == unbound: 86 | break 87 | else: 88 | raise cls.Uninterpolatable('Unable to interpolate %s! Maximum replacements reached.' 89 | % stream) 90 | 91 | stream, _, unbound = iterate(stream, keep_aliases=False) 92 | return stream, unbound 93 | -------------------------------------------------------------------------------- /pystachio/typing.py: -------------------------------------------------------------------------------- 1 | from .naming import frozendict 2 | 3 | 4 | class TypeCheck(object): 5 | """ 6 | Encapsulate the results of a type check pass. 7 | """ 8 | class Error(Exception): 9 | pass 10 | 11 | @staticmethod 12 | def success(): 13 | return TypeCheck(True, "") 14 | 15 | @staticmethod 16 | def failure(msg): 17 | return TypeCheck(False, msg) 18 | 19 | def __init__(self, success, message): 20 | self._success = success 21 | self._message = message 22 | 23 | def message(self): 24 | return self._message 25 | 26 | def ok(self): 27 | return self._success 28 | 29 | def __repr__(self): 30 | if self.ok(): 31 | return 'TypeCheck(OK)' 32 | else: 33 | return 'TypeCheck(FAILED): %s' % self._message 34 | 35 | 36 | class TypeFactoryType(type): 37 | _TYPE_FACTORIES = {} 38 | 39 | def __new__(mcs, name, parents, attributes): 40 | """Args: 41 | mcs(metaclass): the class object to create an instance of. Since this is actually 42 | creating an instance of a type factory class, it's really a metaclass. 43 | name (str): the name of the type to create. 44 | parents (list(class)): the superclasses. 45 | attributes (map(string, value)): 46 | """ 47 | if 'PROVIDES' not in attributes: 48 | return type.__new__(mcs, name, parents, attributes) 49 | else: 50 | provides = attributes['PROVIDES'] 51 | new_type = type.__new__(mcs, name, parents, attributes) 52 | TypeFactoryType._TYPE_FACTORIES[provides] = new_type 53 | return new_type 54 | 55 | 56 | TypeFactoryClass = TypeFactoryType('TypeFactoryClass', (object,), {}) 57 | class TypeFactory(TypeFactoryClass): 58 | @staticmethod 59 | def get_factory(type_name): 60 | assert type_name in TypeFactoryType._TYPE_FACTORIES, ( 61 | 'Unknown type: %s, Existing factories: %s' % ( 62 | type_name, TypeFactoryType._TYPE_FACTORIES.keys())) 63 | return TypeFactoryType._TYPE_FACTORIES[type_name] 64 | 65 | @staticmethod 66 | def create(type_dict, *type_parameters, **kwargs): 67 | """ 68 | Implemented by the TypeFactory to produce a new type. 69 | 70 | Should return: 71 | reified type 72 | (with usable type.__name__) 73 | """ 74 | raise NotImplementedError("create unimplemented for: %s" % repr(type_parameters)) 75 | 76 | @staticmethod 77 | def new(type_dict, type_factory, *type_parameters, **kwargs): 78 | """ 79 | Create a fully reified type from a type schema. 80 | """ 81 | type_tuple = (type_factory,) + type_parameters 82 | if type_tuple not in type_dict: 83 | factory = TypeFactory.get_factory(type_factory) 84 | reified_type = factory.create(type_dict, *type_parameters, **kwargs) 85 | type_dict[type_tuple] = reified_type 86 | return type_dict[type_tuple] 87 | 88 | @staticmethod 89 | def wrapper(factory): 90 | assert issubclass(factory, TypeFactory) 91 | def wrapper_function(*type_parameters): 92 | return TypeFactory.new({}, factory.PROVIDES, *tuple( 93 | [typ.serialize_type() for typ in type_parameters])) 94 | return wrapper_function 95 | 96 | @staticmethod 97 | def load(type_tuple, into=None): 98 | """ 99 | Determine all types touched by loading the type and deposit them into 100 | the particular namespace. 101 | """ 102 | type_dict = {} 103 | TypeFactory.new(type_dict, *type_tuple) 104 | deposit = into if (into is not None and isinstance(into, dict)) else {} 105 | for reified_type in type_dict.values(): 106 | deposit[reified_type.__name__] = reified_type 107 | return deposit 108 | 109 | @staticmethod 110 | def load_json(json_list, into=None): 111 | """ 112 | Determine all types touched by loading the type and deposit them into 113 | the particular namespace. 114 | """ 115 | def l2t(obj): 116 | if isinstance(obj, list): 117 | return tuple(l2t(L) for L in obj) 118 | elif isinstance(obj, dict): 119 | return frozendict(obj) 120 | else: 121 | return obj 122 | return TypeFactory.load(l2t(json_list), into=into) 123 | 124 | @staticmethod 125 | def load_file(filename, into=None): 126 | import json 127 | with open(filename) as fp: 128 | return TypeFactory.load_json(json.load(fp), into=into) 129 | 130 | 131 | class TypeMetaclass(type): 132 | def __instancecheck__(cls, other): 133 | if not hasattr(other, 'type_parameters'): 134 | return False 135 | if not hasattr(other, '__class__'): 136 | return False 137 | if cls.__name__ != other.__class__.__name__: 138 | return False 139 | return cls.type_factory() == other.type_factory() and ( 140 | cls.type_parameters() == other.type_parameters()) 141 | 142 | def __new__(mcls, name, parents, attributes): 143 | """Creates a new Type object (an instance of TypeMetaclass). 144 | Args: 145 | name (str): the name of the new type. 146 | parents (list(str)): a list of superclasses. 147 | attributes: (???): a map from name to value for "parameters" for defining 148 | the new type. 149 | """ 150 | return type.__new__(mcls, name, parents, attributes) 151 | 152 | 153 | class Type(object): 154 | @classmethod 155 | def type_factory(cls): 156 | """ Return the name of the factory that produced this class. """ 157 | raise NotImplementedError 158 | 159 | @classmethod 160 | def type_parameters(cls): 161 | """ Return the type parameters used to produce this class. """ 162 | raise NotImplementedError 163 | 164 | @classmethod 165 | def serialize_type(cls): 166 | return (cls.type_factory(),) + cls.type_parameters() 167 | 168 | @classmethod 169 | def dump(cls, fp): 170 | import json 171 | json.dump(cls.serialize_type(), fp) 172 | 173 | def check(self): 174 | """ 175 | Returns a TypeCheck object explaining whether or not a particular 176 | instance of this object typechecks. 177 | """ 178 | raise NotImplementedError 179 | -------------------------------------------------------------------------------- /setup.cfg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/wickman/pystachio/37baee1361d11746abb25a1bbd74feddd4074ce7/setup.cfg -------------------------------------------------------------------------------- /setup.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | from setuptools import Command, find_packages, setup 3 | 4 | version = '0.8.10' 5 | 6 | 7 | class PyTest(Command): 8 | user_options = [] 9 | 10 | def initialize_options(self): 11 | pass 12 | 13 | def finalize_options(self): 14 | pass 15 | 16 | def run(self): 17 | import sys, subprocess 18 | try: 19 | from py import test as pytest 20 | except ImportError: 21 | raise Exception('Running tests requires pytest.') 22 | errno = subprocess.call([sys.executable, '-m', 'py.test']) 23 | raise SystemExit(errno) 24 | 25 | 26 | setup( 27 | name = 'pystachio', 28 | version = version, 29 | description = 'type-checked dictionary templating library', 30 | url = 'http://github.com/wickman/pystachio', 31 | author = 'Brian Wickman', 32 | author_email = 'wickman@gmail.com', 33 | license = 'MIT', 34 | packages = find_packages(), 35 | py_modules = ['pystachio'], 36 | zip_safe = True, 37 | cmdclass = { 38 | 'test': PyTest 39 | }, 40 | scripts = [ 41 | 'bin/pystachio_repl' 42 | ], 43 | classifiers = [ 44 | 'Programming Language :: Python :: 3', 45 | 'Intended Audience :: Developers', 46 | 'License :: OSI Approved :: MIT License', 47 | 'Operating System :: OS Independent', 48 | ], 49 | ) 50 | -------------------------------------------------------------------------------- /tests/test_base.py: -------------------------------------------------------------------------------- 1 | import pytest 2 | 3 | from pystachio.base import Environment, Object 4 | from pystachio.basic import Integer 5 | from pystachio.container import List 6 | from pystachio.naming import Namable, Ref 7 | 8 | 9 | def dtd(d): 10 | return dict((Ref.from_address(key), str(val)) for key, val in d.items()) 11 | 12 | def ref(address): 13 | return Ref.from_address(address) 14 | 15 | 16 | def test_environment_constructors(): 17 | oe = Environment(a = 1, b = 2) 18 | assert oe._table == dtd({'a': 1, 'b': 2}) 19 | 20 | oe = Environment({'a': 1, 'b': 2}) 21 | assert oe._table == dtd({'a': 1, 'b': 2}) 22 | 23 | oe = Environment({'a': 1}, b = 2) 24 | assert oe._table == dtd({'a': 1, 'b': 2}) 25 | 26 | oe = Environment({'a': 1}, a = 2) 27 | assert oe._table == dtd({'a': 2}), "last update should win" 28 | 29 | oe = Environment({'b': 1}, a = 2) 30 | assert oe._table == dtd({'a': 2, 'b': 1}) 31 | 32 | oe = Environment(oe, a = 3) 33 | assert oe._table == dtd({'a': 3, 'b': 1}) 34 | 35 | bad_values = [None, 3, 'a', type, ()] 36 | for value in bad_values: 37 | with pytest.raises(ValueError): 38 | Environment(value) 39 | bad_values = [None, type, ()] 40 | for value in bad_values: 41 | with pytest.raises(ValueError): 42 | Environment(foo = value) 43 | 44 | 45 | def test_environment_find(): 46 | oe1 = Environment(a = { 'b': 1 }) 47 | oe2 = Environment(a = { 'b': { 'c': List(Integer)([1,2,3]) } } ) 48 | oe = Environment(oe1, oe2) 49 | assert oe.find(ref('a.b')) == '1' 50 | assert oe.find(ref('a.b.c[0]')) == Integer(1) 51 | assert oe.find(ref('a.b.c[1]')) == Integer(2) 52 | assert oe.find(ref('a.b.c[2]')) == Integer(3) 53 | 54 | missing_refs = [ref('b'), ref('b.c'), ref('a.c'), ref('a.b.c[3]')] 55 | for r in missing_refs: 56 | with pytest.raises(Namable.NotFound): 57 | oe.find(r) 58 | 59 | oe = Environment(a = { 'b': { 'c': 5 } } ) 60 | assert oe.find(ref('a.b.c')) == '5' 61 | 62 | 63 | def test_environment_merge(): 64 | oe1 = Environment(a = 1) 65 | oe2 = Environment(b = 2) 66 | assert Environment(oe1, oe2)._table == { 67 | ref('a'): '1', 68 | ref('b'): '2' 69 | } 70 | 71 | oe1 = Environment(a = 1, b = 2) 72 | oe2 = Environment(a = 1, b = {'c': 2}) 73 | assert Environment(oe1, oe2)._table == { 74 | ref('a'): '1', 75 | ref('b'): '2', 76 | ref('b.c'): '2' 77 | } 78 | 79 | oe1 = Environment(a = 1, b = 2) 80 | oe2 = Environment(a = 1, b = {'c': 2}) 81 | assert Environment(oe2, oe1)._table == { 82 | ref('a'): '1', 83 | ref('b'): '2', 84 | ref('b.c'): '2' 85 | } 86 | 87 | oe1 = Environment(a = { 'b': 1 }) 88 | oe2 = Environment(a = { 'c': 2 }) 89 | assert Environment(oe1, oe2)._table == { 90 | ref('a.b'): '1', 91 | ref('a.c'): '2' 92 | } 93 | assert Environment(oe2, oe1)._table == { 94 | ref('a.b'): '1', 95 | ref('a.c'): '2' 96 | } 97 | 98 | 99 | def test_environment_bad_values(): 100 | bad_values = [None, type, object()] 101 | for val in bad_values: 102 | with pytest.raises(ValueError): 103 | Environment(a = val) 104 | 105 | 106 | def test_object_unimplementeds(): 107 | o = Object() 108 | with pytest.raises(NotImplementedError): 109 | Object.checker(o) 110 | with pytest.raises(NotImplementedError): 111 | o.get() 112 | with pytest.raises(NotImplementedError): 113 | oi = o.interpolate() 114 | -------------------------------------------------------------------------------- /tests/test_basic_types.py: -------------------------------------------------------------------------------- 1 | import pytest 2 | 3 | from pystachio.basic import Boolean, Enum, Float, Integer, SimpleObject, String 4 | 5 | 6 | def unicodey(s): 7 | from sys import version_info 8 | if version_info[0] == 2: 9 | return unicode(s) 10 | else: 11 | return s 12 | 13 | 14 | def test_bad_inputs(): 15 | for typ in Float, Integer, String, Boolean: 16 | with pytest.raises(TypeError): 17 | typ() 18 | with pytest.raises(TypeError): 19 | typ("1", "2") 20 | with pytest.raises(TypeError): 21 | typ(foo = '123') 22 | 23 | bad_inputs = [ {1:2}, None, type, Float, Integer, String, Boolean, 24 | Float(1), Integer(1), String(1), Boolean(1) ] 25 | for inp in bad_inputs: 26 | with pytest.raises(SimpleObject.CoercionError): 27 | '%s' % typ(inp) 28 | 29 | 30 | def test_string_constructors(): 31 | good_inputs = [ 32 | '', 'a b c', '{{a}} b {{c}}', '%d', unicodey('unic\u215bde should work too, yo!'), 33 | 1.0, 1, 1e3, 1.0e3 34 | ] 35 | 36 | for input in good_inputs: 37 | str(String(input)) 38 | repr(String(input)) 39 | 40 | 41 | def test_float_constructors(): 42 | bad_inputs = ['', 'a b c', unicodey('a b c'), unicodey('')] 43 | good_inputs = [unicodey('{{foo}}'), '1 ', ' 1', unicodey(' 1e5'), ' {{herp}}.{{derp}} ', 0, 0.0, 1e5] 44 | 45 | for input in bad_inputs: 46 | with pytest.raises(SimpleObject.CoercionError): 47 | str(Float(input)) 48 | with pytest.raises(SimpleObject.CoercionError): 49 | repr(Float(input)) 50 | 51 | for input in good_inputs: 52 | str(Float(input)) 53 | repr(Float(input)) 54 | 55 | assert Float(unicodey(' {{herp}}.{{derp}} ')) % {'herp': 1, 'derp': '2e3'} == Float(1.2e3) 56 | assert Float(123).check().ok() 57 | assert Float('123.123').check().ok() 58 | assert not Float('{{foo}}').check().ok() 59 | 60 | def test_integer_constructors(): 61 | bad_inputs = ['', 'a b c', unicodey('a b c'), unicodey(''), '1e5'] 62 | good_inputs = [unicodey('{{foo}}'), '1 ', ' 1', ' {{herp}}.{{derp}} ', 0, 0.0, 1e5] 63 | 64 | for input in bad_inputs: 65 | with pytest.raises(SimpleObject.CoercionError): 66 | str(Integer(input)) 67 | with pytest.raises(SimpleObject.CoercionError): 68 | repr(Integer(input)) 69 | 70 | for input in good_inputs: 71 | str(Integer(input)) 72 | repr(Integer(input)) 73 | 74 | assert Integer(123).check().ok() 75 | assert Integer('500').check().ok() 76 | assert not Integer('{{foo}}').check().ok() 77 | 78 | 79 | def test_boolean_constructors(): 80 | bad_inputs = ['', 'a b c', unicodey('a b c'), unicodey(''), '1e5', ' 0'] 81 | good_inputs = [unicodey('{{foo}}'), -1, 0, 1, 2, 'true', 'false', '0', '1', True, False] 82 | 83 | for input in bad_inputs: 84 | with pytest.raises(SimpleObject.CoercionError): 85 | str(Boolean(input)) 86 | with pytest.raises(SimpleObject.CoercionError): 87 | repr(Boolean(input)) 88 | 89 | for input in good_inputs: 90 | str(Boolean(input)) 91 | repr(Boolean(input)) 92 | 93 | assert Boolean(0) == Boolean(False) 94 | assert Boolean(0) != Boolean(True) 95 | assert Boolean(1) == Boolean(True) 96 | assert Boolean(1) != Boolean(False) 97 | assert Boolean("0") == Boolean(False) 98 | assert Boolean("1") == Boolean(True) 99 | assert not Boolean("2").check().ok() 100 | assert Boolean(123).check().ok() 101 | assert Boolean('true').check().ok() 102 | assert Boolean('false').check().ok() 103 | assert Boolean(True).check().ok() 104 | assert Boolean(False).check().ok() 105 | assert not Boolean('{{foo}}').check().ok() 106 | assert Boolean('{{foo}}').bind(foo=True).check().ok() 107 | 108 | 109 | def test_cmp(): 110 | assert not Float(1) == Integer(1) 111 | assert Float(1) != Integer(1) 112 | assert not String(1) == Integer(1) 113 | assert Integer(1) < Integer(2) 114 | assert Integer(2) > Integer(1) 115 | assert Integer(1) == Integer(1) 116 | assert not Integer(1) == Integer(2) 117 | assert Integer(1) != Integer(2) 118 | assert not Integer(1) != Integer(1) 119 | assert String("a") < String("b") 120 | assert String("a") == String("a") 121 | assert String("b") > String("a") 122 | assert Float(1) < Float(2) 123 | assert Float(2) > Float(1) 124 | assert Float(1) == Float(1) 125 | assert not Float(1) == Float(1.1) 126 | assert Float(1) != Float(1.1) 127 | assert not Float(1) != Float(1) 128 | assert Float(1.1) > Float(1) 129 | 130 | # all types < 131 | for typ1 in (Float, String, Integer): 132 | for typ2 in (Float, String, Integer): 133 | if typ1 != typ2: 134 | assert typ1(1) < typ2(1) 135 | assert typ1(1) <= typ2(1) 136 | assert not typ1(1) > typ2(1) 137 | assert not typ1(1) >= typ2(1) 138 | 139 | 140 | def test_hash(): 141 | map = { 142 | Integer(1): 'foo', 143 | String("bar"): 'baz', 144 | Float('{{herp}}'): 'derp', 145 | Boolean('true'): 'slerp' 146 | } 147 | assert Integer(1) in map 148 | assert String("bar") in map 149 | assert Float('{{herp}}') in map 150 | assert Float('{{derp}}') not in map 151 | assert Integer(2) not in map 152 | assert String("baz") not in map 153 | assert Boolean('false') not in map 154 | assert Boolean('true') in map 155 | 156 | 157 | def test_N_part_enum_constructors(): 158 | EmptyEnum = Enum() 159 | EmptyEnum('{{should_work}}') 160 | with pytest.raises(ValueError): 161 | repr(EmptyEnum('Anything')) 162 | 163 | OneEnum = Enum('One') 164 | OneEnum('One') 165 | with pytest.raises(ValueError): 166 | OneEnum('Two') 167 | 168 | TwoEnum = Enum('One', 'Two') 169 | for value in ('One', 'Two', '{{anything}}'): 170 | TwoEnum(value) 171 | for value in ('', 1, None, 'Three'): 172 | with pytest.raises(ValueError): 173 | TwoEnum(value) 174 | 175 | assert TwoEnum('One').check().ok() 176 | assert TwoEnum('Two').check().ok() 177 | assert TwoEnum('{{anything}}').bind(anything='One').check().ok() 178 | assert not TwoEnum('{{anything}}').check().ok() 179 | assert not TwoEnum('{{anything}}').bind(anything='Three').check().ok() 180 | 181 | 182 | def test_two_part_enum_constructors(): 183 | Numbers = Enum('Numbers', ('One', 'Two', 'Three')) 184 | Dogs = Enum('Dogs', ('Pug', 'Pit bull')) 185 | 186 | assert not Dogs('Pit {{what}}').check().ok() 187 | assert not Dogs('Pit {{what}}').bind(what='frank').check().ok() 188 | assert Dogs('Pit {{what}}').bind(what='bull').check().ok() 189 | -------------------------------------------------------------------------------- /tests/test_bigger_examples.py: -------------------------------------------------------------------------------- 1 | from pystachio import * 2 | 3 | 4 | class CommandLine(Struct): 5 | binary = Required(String) 6 | args = List(String) 7 | params = Map(String, String) 8 | 9 | class Resources(Struct): 10 | cpu = Required(Float) 11 | ram = Required(Integer) 12 | disk = Default(Integer, 2 * 2**30) 13 | 14 | class Process(Struct): 15 | name = Required(String) 16 | resources = Required(Resources) 17 | cmdline = String 18 | command = CommandLine 19 | max_failures = Default(Integer, 1) 20 | 21 | class Task(Struct): 22 | name = Required(String) 23 | processes = Required(List(Process)) 24 | max_failures = Default(Integer, 1) 25 | 26 | def test_simple_task(): 27 | command = "echo I am {{process.name}} in {{mesos.datacenter}}." 28 | process_template = Process( 29 | name = '{{process.name}}', 30 | resources = Resources(cpu = 1.0, ram = 2**24), 31 | cmdline = command) 32 | basic = Task(name = "basic")( 33 | processes = [ 34 | process_template.bind(process = {'name': 'process_1'}), 35 | process_template.bind(process = {'name': 'process_2'}), 36 | process_template.bind(process = {'name': 'process_3'}), 37 | ]) 38 | 39 | bi, unbound = basic.interpolate() 40 | assert unbound == [Ref.from_address('mesos.datacenter')] 41 | 42 | bi, unbound = (basic % {'mesos': {'datacenter': 'california'}}).interpolate() 43 | assert unbound == [] 44 | assert bi.check().ok() 45 | 46 | def test_type_type_type(): 47 | assert Map(String,Integer) != Map(String,Integer), "Types are no longer memoized." 48 | assert isinstance(Map(String,Integer)({}), Map(String,Integer)) 49 | assert isinstance(Map(Map(String,Integer),Integer)({}), Map(Map(String,Integer),Integer)) 50 | 51 | fake_ages = Map(String,Integer)({ 52 | 'brian': 28, 53 | 'robey': 5000, 54 | 'ian': 15 55 | }) 56 | 57 | real_ages = Map(String,Integer)({ 58 | 'brian': 30, 59 | 'robey': 37, 60 | 'ian': 21 61 | }) 62 | 63 | believability = Map(Map(String, Integer), Integer)({ 64 | fake_ages: 0, 65 | real_ages: 1 66 | }) 67 | 68 | assert Map(Map(String, Integer), Integer)(believability.get()) == believability 69 | 70 | 71 | def test_recursive_unwrapping(): 72 | task = { 73 | 'name': 'basic', 74 | 'processes': [ 75 | { 76 | 'name': 'process1', 77 | 'resources': { 78 | 'cpu': 1.0, 79 | 'ram': 100 80 | }, 81 | 'cmdline': 'echo hello world' 82 | } 83 | ] 84 | } 85 | assert Task(**task).check().ok() 86 | assert Task(task).check().ok() 87 | assert Task(task, **task).check().ok() 88 | assert Task(task) == Task(Task(task).get()) 89 | 90 | task['processes'][0].pop('name') 91 | assert not Task(task).check().ok() 92 | assert not Task(**task).check().ok() 93 | assert not Task(task, **task).check().ok() 94 | assert Task(task) == Task(Task(task).get()) 95 | -------------------------------------------------------------------------------- /tests/test_choice.py: -------------------------------------------------------------------------------- 1 | import json 2 | 3 | from pystachio import Choice, Default, Enum, Float, Integer, List, Ref, Required, String, Struct 4 | from pystachio.naming import frozendict 5 | 6 | 7 | def test_choice_type(): 8 | IntStr = Choice("IntStrFloat", (Integer, String)) 9 | one = IntStr(123) 10 | two = IntStr("123") 11 | three = IntStr("abc") 12 | 13 | assert one == IntStr(123) 14 | assert two == IntStr("123") 15 | assert three == IntStr("abc") 16 | 17 | assert one == two 18 | assert not one == three 19 | assert one != three 20 | assert not one != two 21 | 22 | assert one.unwrap() == Integer(123) 23 | assert two.unwrap() == Integer(123) 24 | assert three.unwrap() == String("abc") 25 | 26 | 27 | def test_choice_error(): 28 | IntFloat = Choice((Integer, Float)) 29 | one = IntFloat(123) 30 | two = IntFloat(123.456) 31 | three = IntFloat("123.abc") 32 | assert one.check().ok() 33 | assert two.check().ok() 34 | assert not three.check().ok() 35 | 36 | 37 | def test_choice_triple(): 38 | Triple = Choice((Integer, Float, String)) 39 | one = Triple(123) 40 | two = Triple(123.456) 41 | three = Triple("123.abc") 42 | assert one.check().ok() 43 | assert two.check().ok() 44 | assert three.check().ok() 45 | 46 | 47 | def test_choice_interpolation(): 48 | IntFloat = Choice((Integer, Float)) 49 | one = IntFloat('{{abc}}') 50 | two = IntFloat('{{a}}{{b}}') 51 | one_int = one.bind(abc=34) 52 | assert isinstance(one_int.interpolate()[0], Integer) 53 | assert one_int.check().ok() 54 | one_fl = one.bind(abc=123.354) 55 | assert isinstance(one_fl.interpolate()[0], Float) 56 | assert one_fl.check().ok() 57 | one_str = one.bind(abc="def") 58 | assert not one_str.check().ok() 59 | assert two.interpolate()[1] == [Ref.from_address('a'), Ref.from_address('b')] 60 | two_one = two.bind(a=12, b=23) 61 | assert two_one.check().ok() 62 | assert two_one.unwrap() == Integer(1223) 63 | two_two = two.bind(a=12, b=".34") 64 | assert two_two.check().ok() 65 | assert two_two.unwrap() == Float(12.34) 66 | 67 | 68 | def test_choice_in_struct(): 69 | class SOne(Struct): 70 | a = Choice((Integer, Float)) 71 | b = String 72 | 73 | one = SOne(a=12, b="abc") 74 | assert one.check().ok() 75 | assert one.interpolate()[0].a().unwrap() == Integer(12) 76 | 77 | two = SOne(a="1{{q}}2", b="hi there") 78 | assert not two.check().ok() 79 | refs = two.interpolate()[1] 80 | assert refs == [Ref.from_address('q')] 81 | 82 | two_int = two.bind(q="34") 83 | assert two_int.check().ok() 84 | assert two_int.a().unwrap() == Integer(1342) 85 | 86 | two_fl = two.bind(q="3.4") 87 | assert two_fl.check().ok() 88 | assert two_fl.a().unwrap() == Float(13.42) 89 | 90 | two_str = two.bind(q="abc") 91 | assert not two_str.check().ok() 92 | 93 | 94 | def test_struct_in_choice_in_struct(): 95 | class Foo(Struct): 96 | a = String 97 | b = Integer 98 | class Yuck(Struct): 99 | one = Choice([Foo, Integer]) 100 | two = String 101 | 102 | y = Yuck(one=3, two="abc") 103 | assert y.check().ok() 104 | 105 | z = Yuck(one=Foo(a="1", b=2), two="hello") 106 | assert z.check().ok() 107 | 108 | 109 | def test_json_choice(): 110 | """Make sure that serializing to JSON works for structs with choices.""" 111 | class Foo(Struct): 112 | a = String 113 | b = Integer 114 | class Yuck(Struct): 115 | one = Choice([Foo, Integer]) 116 | two = String 117 | 118 | z = Yuck(one=Foo(a="1", b=2), two="hello") 119 | assert z.check().ok() 120 | 121 | d = json.loads(z.json_dumps()) 122 | assert d == {"two": "hello", "one": {"a": "1", "b": 2}} 123 | 124 | 125 | def test_choice_string_enum(): 126 | TestEnum = Enum("TestEnum", ("A", "B", "C")) 127 | TestChoice = Choice("TestChoice", (TestEnum, String)) 128 | v = TestChoice("A") 129 | assert isinstance(v.interpolate()[0], TestEnum) 130 | assert isinstance(TestChoice("Q").interpolate()[0], String) 131 | 132 | 133 | def test_choice_default(): 134 | """Ensure that choices with a default work correctly.""" 135 | class Dumb(Struct): 136 | one = String 137 | 138 | class ChoiceDefaultStruct(Struct): 139 | a = Default(Choice("IntOrDumb", [Dumb, Integer]), 28) 140 | b = Integer 141 | 142 | class OtherStruct(Struct): 143 | first = ChoiceDefaultStruct 144 | second = String 145 | 146 | v = OtherStruct(second="hello") 147 | assert v.check() 148 | assert json.loads(v.json_dumps()) == {"second": "hello"} 149 | w = v(first=ChoiceDefaultStruct()) 150 | assert w.check() 151 | assert json.loads(w.json_dumps()) == {'first': {'a': 28}, 'second': 'hello'} 152 | x = v(first=ChoiceDefaultStruct(a=296, b=36)) 153 | assert x.check() 154 | assert json.loads(x.json_dumps()) == {'first': {'a': 296, 'b': 36}, 155 | 'second': 'hello'} 156 | y = v(first=ChoiceDefaultStruct(a=Dumb(one="Oops"), b=37)) 157 | assert y.check() 158 | assert json.loads(y.json_dumps()) == {'first': {'a': {'one': 'Oops'}, 'b': 37}, 159 | 'second': 'hello'} 160 | 161 | 162 | def test_choice_primlist(): 163 | """Test that choices with a list value work correctly.""" 164 | C = Choice([String, List(Integer)]) 165 | c = C([1, 2, 3]) 166 | assert c == C([1, 2, 3]) 167 | assert c.check().ok() 168 | c = C("hello") 169 | assert c.check().ok() 170 | c = C([1, 2, "{{x}}"]) 171 | assert not c.check().ok() 172 | assert c.bind(x=3).check().ok() 173 | 174 | 175 | def test_repr(): 176 | class Dumb(Struct): 177 | one = String 178 | 179 | class ChoiceDefaultStruct(Struct): 180 | a = Default(Choice("IntOrDumb", [Dumb, Integer]), 28) 181 | b = Integer 182 | 183 | class OtherStruct(Struct): 184 | first = ChoiceDefaultStruct 185 | second = String 186 | 187 | C = Choice([String, List(Integer)]) 188 | 189 | testvalone = C("hello") 190 | testvaltwo = C([1, 2, 3]) 191 | assert repr(testvalone) == "Choice_String_IntegerList('hello')" 192 | assert repr(testvaltwo) == "Choice_String_IntegerList([1, 2, 3])" 193 | 194 | 195 | def test_choice_in_stache(): 196 | class Foo(Struct): 197 | x = Integer 198 | 199 | class Bar(Struct): 200 | a = Choice("StringOrFoo", [Integer, String, Foo]) 201 | b = String 202 | 203 | stringbar = Bar(a="hello", b="{{a}} world!") 204 | assert stringbar.check().ok() 205 | assert json.loads(stringbar.json_dumps()) == {'a': 'hello', 'b': 'hello world!'} 206 | 207 | intbar = Bar(a=4, b="{{a}} world!") 208 | assert intbar.check().ok() 209 | assert json.loads(intbar.json_dumps()) == {'a': 4, 'b': '4 world!'} 210 | 211 | foobar = Bar(a=Foo(x=5), b='{{a}} world!') 212 | assert foobar.check().ok() 213 | assert json.loads(foobar.json_dumps()) == {'a': {'x': 5}, 'b': 'Foo(x=5) world!'} 214 | 215 | 216 | def test_get_choice_in_struct(): 217 | class Foo(Struct): 218 | foo = Required(String) 219 | 220 | class Bar(Struct): 221 | bar = Required(String) 222 | 223 | Item = Choice("Item", (Foo, Bar)) 224 | 225 | class Qux(Struct): 226 | item = Choice([String, List(Item)]) 227 | 228 | b = Qux(item=[Foo(foo="fubar")]) 229 | assert b.get() == frozendict({'item': (frozendict({'foo': u'fubar'}),)}) 230 | 231 | 232 | def test_hashing(): 233 | IntStr = Choice("IntStrFloat", (Integer, String)) 234 | 235 | map = { 236 | IntStr(123): 'foo', 237 | IntStr("123"): 'bar', 238 | IntStr("abc"): 'baz' 239 | } 240 | assert IntStr(123) in map 241 | assert IntStr("123") in map 242 | assert IntStr("abc") in map 243 | assert IntStr(456) not in map 244 | assert IntStr("456") not in map 245 | assert IntStr("def") not in map 246 | -------------------------------------------------------------------------------- /tests/test_composite_types.py: -------------------------------------------------------------------------------- 1 | import pytest 2 | 3 | from pystachio.basic import * 4 | from pystachio.composite import * 5 | from pystachio.container import List, Map 6 | from pystachio.naming import Ref 7 | 8 | 9 | def ref(address): 10 | return Ref.from_address(address) 11 | 12 | 13 | def test_basic_types(): 14 | class Resources(Struct): 15 | cpu = Float 16 | ram = Integer 17 | assert Resources().check().ok() 18 | assert Resources(cpu = 1.0).check().ok() 19 | assert Resources(cpu = 1.0, ram = 100).check().ok() 20 | assert Resources(cpu = 1, ram = 100).check().ok() 21 | assert Resources(cpu = '1.0', ram = 100).check().ok() 22 | 23 | 24 | def test_bad_inputs(): 25 | class Resources(Struct): 26 | cpu = Float 27 | ram = Required(Integer) 28 | with pytest.raises(AttributeError): 29 | Resources(herp = "derp") 30 | with pytest.raises(AttributeError): 31 | Resources({'foo': 'bar'}) 32 | with pytest.raises(ValueError): 33 | Resources(None) 34 | 35 | 36 | def test_nested_composites(): 37 | class Resources(Struct): 38 | cpu = Float 39 | ram = Integer 40 | class Process(Struct): 41 | name = String 42 | resources = Resources 43 | assert Process().check().ok() 44 | assert Process(name = "hello_world").check().ok() 45 | assert Process(resources = Resources()).check().ok() 46 | assert Process(resources = Resources(cpu = 1.0)).check().ok() 47 | assert Process(resources = Resources(cpu = 1)).check().ok() 48 | assert Process(name = 15)(resources = Resources(cpu = 1.0)).check().ok() 49 | repr(Process(name = 15)(resources = Resources(cpu = 1.0))) 50 | repr(Process.TYPEMAP) 51 | 52 | 53 | def test_typesig(): 54 | class Process1(Struct): 55 | name = String 56 | class Process2(Struct): 57 | name = Required(String) 58 | class Process3(Struct): 59 | name = Default(String, "foo") 60 | class Process4(Struct): 61 | name = String 62 | assert Process1.TYPEMAP['name'] == Process4.TYPEMAP['name'] 63 | assert Process1.TYPEMAP['name'] != Process2.TYPEMAP['name'] 64 | assert Process1.TYPEMAP['name'] != Process3.TYPEMAP['name'] 65 | assert Process2.TYPEMAP['name'] != Process3.TYPEMAP['name'] 66 | repr(Process1.TYPEMAP['name']) 67 | 68 | 69 | def test_defaults(): 70 | class Resources(Struct): 71 | cpu = Default(Float, 1.0) 72 | ram = Integer 73 | assert Resources() == Resources(cpu = 1.0) 74 | assert not Resources() == Resources(cpu = 2.0) 75 | assert Resources() != Resources(cpu = 2.0) 76 | assert not Resources() != Resources(cpu = 1.0) 77 | assert Resources(cpu = 2.0)._schema_data['cpu'] == Float(2.0) 78 | 79 | class Process(Struct): 80 | name = String 81 | resources = Default(Resources, Resources(ram = 10)) 82 | 83 | assert Process().check().ok() 84 | assert Process() == Process(resources = Resources(cpu = 1.0, ram = 10)) 85 | assert Process() != Process(resources = Resources()) 86 | assert Process()(resources = Empty).check().ok() 87 | 88 | 89 | def test_composite_interpolation(): 90 | class Resources(Struct): 91 | cpu = Required(Float) 92 | ram = Integer 93 | disk = Integer 94 | 95 | class Process(Struct): 96 | name = Required(String) 97 | resources = Map(String, Resources) 98 | 99 | p = Process(name = "hello") 100 | assert p(resources = {'foo': Resources()}) == \ 101 | p(resources = {'{{whee}}': Resources()}).bind(whee='foo') 102 | assert p(resources = {'{{whee}}': Resources(cpu='{{whee}}')}).bind(whee=1.0) == \ 103 | p(resources = {'1.0': Resources(cpu=1.0)}) 104 | 105 | 106 | def test_internal_interpolate(): 107 | class Process(Struct): 108 | name = Required(String) 109 | cmdline = Required(String) 110 | 111 | class Task(Struct): 112 | name = Default(String, 'task-{{processes[0].name}}') 113 | processes = Required(List(Process)) 114 | 115 | class Job(Struct): 116 | name = Default(String, '{{task.name}}') 117 | task = Required(Task) 118 | 119 | assert Task().name() == String('task-{{processes[0].name}}') 120 | assert Task(processes=[Process(name='hello_world', cmdline='echo hello_world')]).name() == \ 121 | String('task-hello_world') 122 | assert Task(processes=[Process(name='hello_world', cmdline='echo hello_world'), 123 | Process(name='hello_world2', cmdline='echo hello world')]).name() == \ 124 | String('task-hello_world') 125 | assert Job(task=Task(processes=[Process(name="hello_world")])).name() == \ 126 | String('task-hello_world') 127 | 128 | 129 | def test_find(): 130 | class Resources(Struct): 131 | cpu = Required(Float) 132 | ram = Integer 133 | disks = List(String) 134 | 135 | class Process(Struct): 136 | name = Required(String) 137 | resources = Map(String, Resources) 138 | 139 | res0 = Resources(cpu = 0.0, ram = 0) 140 | res1 = Resources(cpu = 1.0, ram = 1, disks = ['hda3']) 141 | res2 = Resources(cpu = 2.0, ram = 2, disks = ['hda3', 'hdb3']) 142 | proc = Process(name = "hello", resources = { 143 | 'res0': res0, 144 | 'res1': res1, 145 | 'res2': res2 146 | }) 147 | 148 | with pytest.raises(Namable.NotFound): 149 | proc.find(ref('herp')) 150 | 151 | assert proc.find(ref('name')) == String('hello') 152 | assert proc.find(ref('resources[res0].cpu')) == Float(0.0) 153 | assert proc.find(ref('resources[res0].ram')) == Integer(0) 154 | with pytest.raises(Namable.NotFound): 155 | proc.find(ref('resources[res0].disks')) 156 | with pytest.raises(Namable.NamingError): 157 | proc.find(ref('resources.res0.disks')) 158 | with pytest.raises(Namable.NamingError): 159 | proc.find(ref('resources[res0][disks]')) 160 | with pytest.raises(Namable.Unnamable): 161 | proc.find(ref('name.herp')) 162 | with pytest.raises(Namable.Unnamable): 163 | proc.find(ref('name[herp]')) 164 | assert proc.find(ref('resources[res1].ram')) == Integer(1) 165 | assert proc.find(ref('resources[res1].disks[0]')) == String('hda3') 166 | assert proc.find(ref('resources[res2].disks[0]')) == String('hda3') 167 | assert proc.find(ref('resources[res2].disks[1]')) == String('hdb3') 168 | 169 | 170 | def test_getattr_functions(): 171 | class Resources(Struct): 172 | cpu = Required(Float) 173 | ram = Integer 174 | disk = Integer 175 | 176 | class Process(Struct): 177 | name = Required(String) 178 | resources = Map(String, Resources) 179 | 180 | # Basic getattr + hasattr 181 | assert Process(name = "hello").name() == String('hello') 182 | assert Process().has_name() is False 183 | assert Process(name = "hello").has_name() is True 184 | 185 | 186 | p = Process(name = "hello") 187 | p1 = p(resources = {'foo': Resources()}) 188 | p2 = p(resources = {'{{whee}}': Resources()}).bind(whee='foo') 189 | 190 | assert p1.has_resources() 191 | assert p2.has_resources() 192 | assert String('foo') in p1.resources() 193 | assert String('foo') in p2.resources() 194 | 195 | 196 | def test_getattr_bad_cases(): 197 | # Technically speaking if we had 198 | # class Tricky(Struct): 199 | # stuff = Integer 200 | # has_stuff = Integer 201 | # would be ~= undefined. 202 | 203 | class Tricky(Struct): 204 | has_stuff = Integer 205 | t = Tricky() 206 | assert t.has_has_stuff() is False 207 | assert t.has_stuff() is Empty 208 | 209 | with pytest.raises(AttributeError): 210 | t.this_should_properly_raise 211 | 212 | 213 | def test_self_super(): 214 | class Child(Struct): 215 | value = Integer 216 | 217 | class Parent(Struct): 218 | child = Child 219 | value = Integer 220 | 221 | class Grandparent(Struct): 222 | parent = Parent 223 | value = Integer 224 | 225 | parent = Parent(child=Child(value='{{super.value}}'), value=23) 226 | parent, _ = parent.interpolate() 227 | assert parent.child().value().get() == 23 228 | 229 | grandparent = Grandparent(parent=Parent(child=Child(value='{{super.super.value}}')), value=23) 230 | grandparent, _ = grandparent.interpolate() 231 | assert grandparent.parent().child().value().get() == 23 232 | 233 | parent = Parent(child=Child(value=23), value='{{child.value}}') 234 | parent, _ = parent.interpolate() 235 | assert parent.child().value().get() == 23 236 | 237 | parent = Parent(child=Child(value=23), value='{{self.child.value}}') 238 | parent, _ = parent.interpolate() 239 | assert parent.child().value().get() == 23 240 | 241 | 242 | def test_hashing(): 243 | class Resources(Struct): 244 | cpu = Float 245 | ram = Integer 246 | class Process(Struct): 247 | name = String 248 | resources = Resources 249 | 250 | map = { 251 | Resources(): 'foo', 252 | Process(): 'bar', 253 | Resources(cpu=1.1): 'baz', 254 | Process(resources=Resources(cpu=1.1)): 'derp' 255 | } 256 | assert Resources() in map 257 | assert Process() in map 258 | assert Resources(cpu=1.1) in map 259 | assert Resources(cpu=2.2) not in map 260 | assert Process(resources=Resources(cpu=1.1)) in map 261 | assert Process(resources=Resources(cpu=2.2)) not in map 262 | 263 | def test_super(): 264 | class Monitor(Struct): 265 | 266 | def json_dumps(self): 267 | return super(Monitor, self).json_dumps() 268 | -------------------------------------------------------------------------------- /tests/test_config.py: -------------------------------------------------------------------------------- 1 | import contextlib 2 | import json 3 | import os 4 | import shutil 5 | import tempfile 6 | import textwrap 7 | from io import BytesIO 8 | 9 | import pytest 10 | 11 | from pystachio.config import Config, ConfigContext 12 | 13 | 14 | @contextlib.contextmanager 15 | def make_layout(layout): 16 | tempdir = tempfile.mkdtemp() 17 | 18 | for filename, file_content in layout.items(): 19 | real_path = os.path.join(tempdir, filename) 20 | try: 21 | os.makedirs(os.path.dirname(real_path)) 22 | except OSError: 23 | # assume EEXIST 24 | pass 25 | with open(real_path, 'w') as fp: 26 | fp.write(textwrap.dedent(file_content)) 27 | 28 | try: 29 | yield tempdir 30 | finally: 31 | shutil.rmtree(tempdir) 32 | 33 | 34 | @contextlib.contextmanager 35 | def pushd(dirname): 36 | cwd = os.getcwd() 37 | os.chdir(dirname) 38 | try: 39 | yield 40 | finally: 41 | os.chdir(cwd) 42 | 43 | 44 | def test_includes(): 45 | layout = { 46 | 'dir1/dir2/a.config': 47 | """ 48 | include("../b.config") 49 | a = b 50 | """, 51 | 52 | 'dir1/b.config': 53 | """ 54 | include("../c.config") 55 | b = "Hello" 56 | """, 57 | 58 | 'c.config': 59 | """ 60 | c = "Goodbye" 61 | """ 62 | } 63 | 64 | def k(a, b): 65 | return ConfigContext.key(a, b) 66 | 67 | with make_layout(layout) as td: 68 | with pushd(td): 69 | config = Config('dir1/dir2/a.config') 70 | assert config.environment['a'] == 'Hello' 71 | assert config.environment['b'] == 'Hello' 72 | assert config.environment['c'] == 'Goodbye' 73 | assert config.loadables[k('', 'dir1/dir2/a.config')] == textwrap.dedent( 74 | layout['dir1/dir2/a.config']) 75 | assert config.loadables[k('dir1/dir2/a.config', '../b.config')] == ( 76 | textwrap.dedent(layout['dir1/b.config'])) 77 | 78 | config2 = Config(config.loadables) 79 | assert config.environment['a'] == config2.environment['a'] 80 | assert config.environment['b'] == config2.environment['b'] 81 | assert config2.environment['c'] == 'Goodbye' 82 | assert config.loadables == config2.loadables 83 | 84 | config3 = Config(json.loads(json.dumps(config.loadables))) 85 | assert config.environment['a'] == config3.environment['a'] 86 | assert config.environment['b'] == config3.environment['b'] 87 | assert config3.environment['c'] == 'Goodbye' 88 | assert config.loadables == config3.loadables 89 | 90 | with make_layout(layout) as td: 91 | config = Config(os.path.join(td, 'dir1/dir2/a.config')) 92 | config2 = Config(config.loadables) 93 | assert config.environment['a'] == config2.environment['a'] 94 | assert config.environment['b'] == config2.environment['b'] 95 | assert config.loadables == config2.loadables 96 | 97 | 98 | def test_filelike_config(): 99 | foo = b"a = 'Hello'" 100 | config = Config(BytesIO(foo)) 101 | assert config.environment['a'] == 'Hello' 102 | 103 | config2 = Config(config.loadables) 104 | assert config2.environment['a'] == 'Hello' 105 | 106 | foo = b"include('derp')\na = 'Hello'" 107 | with pytest.raises(Config.InvalidConfigError): 108 | config = Config(BytesIO(foo)) 109 | -------------------------------------------------------------------------------- /tests/test_container_types.py: -------------------------------------------------------------------------------- 1 | import os 2 | import tempfile 3 | 4 | import pytest 5 | 6 | from pystachio.basic import * 7 | from pystachio.composite import Default, Struct 8 | from pystachio.container import List, Map 9 | from pystachio.naming import Namable, Ref 10 | 11 | 12 | def ref(address): 13 | return Ref.from_address(address) 14 | 15 | def test_basic_lists(): 16 | assert List(Integer)([]).check().ok() 17 | assert List(Integer)([1]).check().ok() 18 | assert List(Integer)((1,)).check().ok() 19 | assert List(Integer)(["1",]).check().ok() 20 | assert not List(Integer)([1, "{{two}}"]).check().ok() 21 | assert (List(Integer)([1, "{{two}}"]) % {'two': 2}).check().ok() 22 | with pytest.raises(ValueError): 23 | List(Integer)({'not': 'a', 'list': 'type'}) 24 | repr(List(Integer)([1, '{{two}}'])) 25 | 26 | def test_basic_scoping(): 27 | i = Integer('{{intvalue}}') 28 | lst = List(Integer)([i.bind(intvalue = 1), i.bind(intvalue = 2), i]) 29 | lsti, _ = lst.bind(intvalue = 3).interpolate() 30 | assert lsti == List(Integer)([Integer(1), Integer(2), Integer(3)]) 31 | lsti, _ = lst.in_scope(intvalue = 3).interpolate() 32 | assert lsti == List(Integer)([Integer(1), Integer(2), Integer(3)]) 33 | one = ref('[0]') 34 | two = ref('[1]') 35 | three = ref('[2]') 36 | assert lst.find(one) == Integer(1) 37 | assert lst.find(two) == Integer(2) 38 | assert lst.find(three) == Integer('{{intvalue}}') 39 | assert lst.in_scope(intvalue = 3).find(one) == Integer(1) 40 | assert lst.in_scope(intvalue = 3).find(two) == Integer(2) 41 | assert lst.in_scope(intvalue = 3).find(three) == Integer(3) 42 | 43 | def test_iteration(): 44 | li = List(Integer)([1,2,3]) 45 | liter = iter(li) 46 | assert next(liter) == Integer(1) 47 | assert next(liter) == Integer(2) 48 | assert next(liter) == Integer(3) 49 | with pytest.raises(StopIteration): 50 | next(liter) 51 | li = List(Integer)([]) 52 | with pytest.raises(StopIteration): 53 | next(iter(li)) 54 | 55 | def test_indexing(): 56 | li = List(Integer)([1,2,3]) 57 | for bad in ['a', None, type, Integer]: 58 | with pytest.raises(TypeError): 59 | li[bad] 60 | 61 | # Indexing should also support slices 62 | li = List(Integer)(range(10)) 63 | assert li[0] == Integer(0) 64 | assert li[-1] == Integer(9) 65 | assert li[::2] == (Integer(0), Integer(2), Integer(4), Integer(6), Integer(8)) 66 | assert li[8:] == (Integer(8), Integer(9)) 67 | assert li[2:0:-1] == (Integer(2), Integer(1)) 68 | with pytest.raises(IndexError): 69 | li[10] 70 | 71 | def test_list_scoping(): 72 | assert List(Integer)([1, "{{wut}}"]).interpolate() == ( 73 | List(Integer)([Integer(1), Integer('{{wut}}')]), [ref('wut')]) 74 | assert List(Integer)([1, "{{wut}}"]).bind(wut = 23).interpolate() == ( 75 | List(Integer)([Integer(1), Integer(23)]), []) 76 | assert List(Integer)([1, Integer("{{wut}}").bind(wut = 24)]).bind(wut = 23).interpolate() == ( 77 | List(Integer)([Integer(1), Integer(24)]), []) 78 | 79 | def test_list_find(): 80 | ls = List(String)(['a', 'b', 'c']) 81 | assert ls.find(ref('[0]')) == String('a') 82 | with pytest.raises(Namable.NamingError): 83 | ls.find(ref('.a')) 84 | with pytest.raises(Namable.NamingError): 85 | ls.find(ref('[a]')) 86 | with pytest.raises(Namable.NotFound): 87 | ls.find(ref('[4]')) 88 | with pytest.raises(Namable.Unnamable): 89 | ls.find(ref('[1].foo')) 90 | 91 | def test_equals(): 92 | assert List(Integer)([1, "{{wut}}"]).bind(wut=23) == List(Integer)([1, 23]) 93 | assert not List(Integer)([1, 2]) == List(Integer)([1, 3]) 94 | assert List(Integer)([1, 2]) != List(Integer)([1, 3]) 95 | assert not List(Integer)([1, "{{wut}}"]).bind(wut=23) != List(Integer)([1, 23]) 96 | 97 | def test_basic_maps(): 98 | assert Map(String,Integer)({}).check().ok() 99 | assert Map(String,Integer)({'a':1}).check().ok() 100 | assert Map(String,Integer)(('a', 1)).check().ok() 101 | assert Map(String,Integer)(('a', 1), ('b', 2)).check().ok() 102 | assert not Map(String,Integer)({'a':'{{foo}}'}).check().ok() 103 | assert not Map(Integer,String)({'{{foo}}':'a'}).check().ok() 104 | assert Map(String,Integer)({'a':'{{foo}}'}).bind(foo = 5).check().ok() 105 | with pytest.raises(TypeError): 106 | Map(String,Integer)(a = 1) 107 | with pytest.raises(ValueError): 108 | Map(String,Integer)({'a': 1}, {'b': 2}) 109 | for value in [None, type, 123, 'a']: 110 | with pytest.raises(ValueError): 111 | Map(String,Integer)(value) 112 | repr(Map(String,Integer)(('a', 1), ('b', 2))) 113 | 114 | def test_map_find(): 115 | msi = Map(String,Integer)({'a':1}) 116 | assert msi.find(ref('[a]')) == Integer(1) 117 | with pytest.raises(Namable.NamingError): 118 | msi.find(ref('.a')) 119 | with pytest.raises(Namable.NotFound): 120 | msi.find(ref('[b]')) 121 | with pytest.raises(Namable.Unnamable): 122 | msi.find(ref('[a].foo')) 123 | 124 | mii = Map(Integer,String)({3: 'foo', '5': 'bar'}) 125 | assert mii.find(ref('[3]')) == String('foo') 126 | assert mii.find(ref('[5]')) == String('bar') 127 | 128 | def test_map_iteration(): 129 | mi = Map(String,Integer)({'a': 1, 'b': 2}) 130 | miter = iter(mi) 131 | assert next(miter) in (String('a'), String('b')) 132 | assert next(miter) in (String('a'), String('b')) 133 | with pytest.raises(StopIteration): 134 | next(miter) 135 | 136 | mi = Map(String,Integer)({}) 137 | with pytest.raises(StopIteration): 138 | next(iter(mi)) 139 | 140 | def test_map_idioms(): 141 | mi = Map(String,Integer)({'a': 1, 'b': 2}) 142 | for key in ['a', String('a')]: 143 | assert mi[key] == Integer(1) 144 | assert key in mi 145 | for key in ['b', String('b')]: 146 | assert mi[key] == Integer(2) 147 | assert key in mi 148 | for key in [1, 'c', String('c')]: 149 | with pytest.raises(KeyError): 150 | mi[key] 151 | assert key not in mi 152 | 153 | @pytest.mark.xfail(reason="Pre-coercion checks need to be improved.") 154 | def test_map_keys_that_should_improve(): 155 | mi = Map(String, Integer)() 156 | for key in [{2: "hello"}, String, Integer, type]: 157 | with pytest.raises(KeyError): 158 | mi[key] 159 | assert key not in mi 160 | 161 | def test_hashing(): 162 | map = { 163 | List(Integer)([1,2,3]): 'foo', 164 | Map(String,Integer)({'a': 1, 'b': 2}): 'bar' 165 | } 166 | assert List(Integer)([1,2,3]) in map 167 | assert Map(String,Integer)({'a': 1, 'b': 2}) in map 168 | assert List(Integer)([3,2,1]) not in map 169 | assert Map(String,Integer)({'a': 2, 'b': 1}) not in map 170 | 171 | def test_load_json(): 172 | class Child(Struct): 173 | child_name = String 174 | 175 | class Process(Struct): 176 | name = Default(String, 'hello') 177 | cmdline = String 178 | child = Child 179 | 180 | GOOD_JSON = [ 181 | '{}', 182 | '{"name": "hello world"}', 183 | '{"cmdline": "bitchin"}', 184 | '{"name": "hello world", "cmdline": "bitchin"}', 185 | '{"name": "hello world", "cmdline": "bitchin", "child": {}}', 186 | '{"name": "hello world", "cmdline": "bitchin", "child": {"child_name": "wow"}}', 187 | ] 188 | 189 | FAILSTRICT_JSON = [ 190 | '{"name": "hello world", "cmdline": "bitchin", "extra_schema_arg": "yay"}', 191 | '{"name": "hello world", "cmdline": "bitchin", "child": {"extra_arg": "yayer"}}', 192 | ] 193 | 194 | FAIL = [ 195 | '{"name": [1,2], "cmdline": "bitchin"}', 196 | '{"name": [1,2], "cmdline": "bitchin", "extra_schema": "foo"}', 197 | '{"name": "hello world", "cmdline": "bitchin", "child": {"child_name": [1, 2]}}', 198 | ] 199 | 200 | for js in GOOD_JSON + FAILSTRICT_JSON: 201 | assert Process.json_loads(js, strict=False).check().ok() 202 | 203 | for js in GOOD_JSON: 204 | assert Process.json_loads(js, strict=True).check().ok() 205 | 206 | for js in FAILSTRICT_JSON: 207 | with pytest.raises(AttributeError): 208 | Process.json_loads(js, strict=True) 209 | 210 | for js in FAIL: 211 | assert not Process.json_loads(js, strict=False).check().ok() 212 | 213 | with pytest.raises(AttributeError): 214 | Process.json_loads('{"name": [1,2], "cmdline": "bitchin", "extra_schema": "foo"}', strict=True) 215 | 216 | 217 | def test_load_json_fp(): 218 | class Process(Struct): 219 | name = Default(String, 'hello') 220 | cmdline = String 221 | 222 | GOOD_JSON = [ 223 | '{}', 224 | '{"name": "hello world"}', 225 | '{"cmdline": "bitchin"}', 226 | '{"name": "hello world", "cmdline": "bitchin"}', 227 | ] 228 | 229 | for js in GOOD_JSON: 230 | p = Process.json_loads(js) 231 | assert p == Process.json_loads(p.json_dumps()) 232 | try: 233 | fd, fn = tempfile.mkstemp() 234 | os.close(fd) 235 | with open(fn, 'w') as fp: 236 | p.json_dump(fp) 237 | with open(fn, 'r') as fp: 238 | assert p == Process.json_load(fp) 239 | finally: 240 | os.unlink(fn) 241 | -------------------------------------------------------------------------------- /tests/test_matching.py: -------------------------------------------------------------------------------- 1 | from pystachio import * 2 | from pystachio.matcher import Any, Matcher 3 | 4 | 5 | def test_matcher(): 6 | matcher = Matcher('hello') 7 | assert list(matcher.match(String(''))) == [] 8 | assert list(matcher.match(String('hello'))) == [] 9 | assert list(matcher.match(String('{{hello}}'))) == [('hello',)] 10 | 11 | matcher = Matcher('packer')[Any][Any][Any] 12 | matches = list(matcher.match(String('{{packer[foo][bar][baz].bak}}'))) 13 | assert len(matches) == 1 14 | assert matches[0] == ('packer', 'foo', 'bar', 'baz') 15 | 16 | matcher = Matcher('derp').Any[r'\d+'] 17 | matches = list(matcher.match(String('{{derp.a[23]}}'))) 18 | assert len(matches) == 1 19 | assert matches[0] == ('derp', 'a', '23') 20 | 21 | matcher = Matcher('herp').derp 22 | assert list(matcher.match(String('{{herp.derp}}'))) == [('herp', 'derp')] 23 | 24 | matcher = Matcher('herp')._('.*') 25 | assert list(matcher.match(String('{{herp.derp}}'))) == [('herp', 'derp')] 26 | 27 | 28 | def test_negative_matches(): 29 | matcher = Matcher('hello') 30 | assert list(matcher.match(String('{{not_hello}}'))) == [] 31 | 32 | matcher = Matcher('a').b.c 33 | assert list(matcher.match(String('{{a.b.d}}'))) == [] 34 | 35 | 36 | def test_binder(): 37 | class Packer(Struct): 38 | target = Required(String) 39 | 40 | packer_matcher = Matcher('packer')[Any][Any][Any] 41 | 42 | def packer_binder(_, role, env, name): 43 | return Packer(target = '{{role}}/{{env}}/{{name}}').bind(role=role, env=env, name=name) 44 | 45 | assert str(packer_matcher.apply(packer_binder, String('{{packer[foo][bar][baz].target}}'))) == ( 46 | 'foo/bar/baz') 47 | -------------------------------------------------------------------------------- /tests/test_naming.py: -------------------------------------------------------------------------------- 1 | from copy import deepcopy 2 | 3 | import pytest 4 | 5 | from pystachio.basic import * 6 | from pystachio.composite import * 7 | from pystachio.container import * 8 | from pystachio.naming import Ref 9 | 10 | 11 | def ref(address): 12 | return Ref.from_address(address) 13 | 14 | 15 | def test_ref_parsing(): 16 | for input in ['', None, type, 1, 3.0, 'hork bork']: 17 | with pytest.raises(Ref.InvalidRefError): 18 | ref(input) 19 | 20 | ref('a').components() == [Ref.Dereference('a')] 21 | ref('.a').components() == [Ref.Dereference('a')] 22 | ref('a.b').components() == [Ref.Dereference('a'), Ref.Dereference('b')] 23 | ref('[a]').components() == [Ref.Index('a')] 24 | ref('[a-]').components() == [Ref.Index('a-')] 25 | ref('[0].a').components() == [Ref.Index('0'), Ref.Dereference('a')] 26 | ref('[0][a]').components() == [Ref.Index('0'), Ref.Index('a')] 27 | ref('[.]').components() == [Ref.Index('.')] 28 | ref('[/]').components() == [Ref.Index('/')] 29 | ref('[_]').components() == [Ref.Index('_')] 30 | ref('[-]').components() == [Ref.Index('-')] 31 | ref('[a.b]').components() == [Ref.Index('a.b')] 32 | ref('[a/b]').components() == [Ref.Index('a/b')] 33 | ref('[a_b]').components() == [Ref.Index('a_b')] 34 | ref('[a-b]').components() == [Ref.Index('a-b')] 35 | ref('[a/b/c/d]').components() == [Ref.Index('a/b/c/d')] 36 | ref('[2.0.a_c/d-e]').components() == [Ref.Index('2.0.a_c/d-e')] 37 | for refstr in ('[a]b', '[]', '[[a]', 'b[[[', 'a.1', '1.a', '.[a]', '0'): 38 | with pytest.raises(Ref.InvalidRefError): 39 | ref(refstr) 40 | for refstr in ('a-b', '-b', 'a-'): 41 | with pytest.raises(Ref.InvalidRefError): 42 | ref(refstr) 43 | 44 | 45 | def test_ref_lookup(): 46 | oe = Environment(a = 1) 47 | assert oe.find(ref("a")) == '1' 48 | 49 | oe = Environment(a = {'b': 1}) 50 | assert oe.find(ref("a.b")) == '1' 51 | 52 | oe = Environment(a = {'b': {'c': 1}, 'c': Environment(d = 2)}) 53 | assert oe.find(ref('a.b.c')) == '1' 54 | assert oe.find(ref('a.c.d')) == '2' 55 | 56 | for address in ["a", "a.b", "a.c"]: 57 | with pytest.raises(Namable.NotFound): 58 | oe.find(ref(address)) 59 | 60 | oe = List(String)(["a", "b", "c"]) 61 | assert oe.find(ref('[0]')) == String('a') 62 | with pytest.raises(Namable.NotFound): 63 | oe.find(ref('[3]')) 64 | 65 | oe = List(Map(String,Integer))([{'a': 27}]) 66 | oe.find(ref('[0][a]')) == Integer(27) 67 | Environment(foo = oe).find(ref('foo[0][a]')) == Integer(27) 68 | 69 | 70 | def test_complex_lookup(): 71 | class Employee(Struct): 72 | first = String 73 | last = String 74 | 75 | class Employer(Struct): 76 | name = String 77 | employees = List(Employee) 78 | 79 | twttr = Employer( 80 | name = 'Twitter', 81 | employees = [ 82 | Employee(first = 'brian', last = 'wickman'), 83 | Employee(first = 'marius'), 84 | Employee(last = '{{default.last}}') 85 | ]) 86 | 87 | 88 | assert Environment(twttr = twttr).find(ref('twttr.employees[1].first')) == String('marius') 89 | assert Map(String,Employer)({'twttr': twttr}).find(ref('[twttr].employees[1].first')) == \ 90 | String('marius') 91 | assert List(Employer)([twttr]).find(ref('[0].employees[0].last')) == String('wickman') 92 | assert List(Employer)([twttr]).find(ref('[0].employees[2].last')) == String('{{default.last}}') 93 | 94 | 95 | def test_scope_lookup(): 96 | refs = [ref('mesos.ports[health]'), ref('mesos.config'), ref('logrotate.filename'), 97 | ref('mesos.ports.http')] 98 | scoped_refs = [m for m in map(ref('mesos.ports').scoped_to, refs) if m] 99 | assert scoped_refs == [ref('[health]'), ref('http')] 100 | 101 | refs = [ref('[a]'), ref('[a][b]'), ref('[a].b')] 102 | scoped_refs = [m for m in map(ref('[a]').scoped_to, refs) if m] 103 | assert scoped_refs == [ref('[b]'), ref('b')] 104 | 105 | 106 | def test_scope_override(): 107 | class MesosConfig(Struct): 108 | ports = Map(String, Integer) 109 | config = MesosConfig(ports = {'http': 80, 'health': 8888}) 110 | env = Environment({ref('config.ports[http]'): 5000}, config = config) 111 | assert env.find(ref('config.ports[http]')) == '5000' 112 | assert env.find(ref('config.ports[health]')) == Integer(8888) 113 | 114 | 115 | def test_inherited_scope(): 116 | class PhoneBookEntry(Struct): 117 | name = Required(String) 118 | number = Required(Integer) 119 | 120 | class PhoneBook(Struct): 121 | city = Required(String) 122 | people = List(PhoneBookEntry) 123 | 124 | sf = PhoneBook(city = "San Francisco").bind(areacode = 415) 125 | sj = PhoneBook(city = "San Jose").bind(areacode = 408) 126 | jenny = PhoneBookEntry(name = "Jenny", number = "{{areacode}}8675309") 127 | brian = PhoneBookEntry(name = "Brian", number = "{{areacode}}5551234") 128 | brian = brian.bind(areacode = 402) 129 | sfpb = sf(people = [jenny, brian]) 130 | assert sfpb.find(ref('people[0].number')) == Integer(4158675309) 131 | assert sfpb.bind(areacode = 999).find(ref('people[0].number')) == Integer(9998675309) 132 | assert sfpb.find(ref('people[1].number')) == Integer(4025551234) 133 | assert sfpb.bind(areacode = 999).find(ref('people[1].number')) == Integer(4025551234) 134 | 135 | 136 | def test_deepcopy_preserves_bindings(): 137 | class PhoneBookEntry(Struct): 138 | name = Required(String) 139 | number = Required(Integer) 140 | brian = PhoneBookEntry(number = "{{areacode}}5551234") 141 | brian = brian.bind(areacode = 402) 142 | briancopy = deepcopy(brian) 143 | assert brian.find(ref('number')) == Integer(4025551234) 144 | assert briancopy.find(ref('number')) == Integer(4025551234) 145 | -------------------------------------------------------------------------------- /tests/test_parsing.py: -------------------------------------------------------------------------------- 1 | import pytest 2 | 3 | from pystachio.base import Environment 4 | from pystachio.basic import String 5 | from pystachio.composite import Default, Required, Struct 6 | from pystachio.container import List 7 | from pystachio.naming import Ref 8 | from pystachio.parsing import MustacheParser 9 | 10 | 11 | def ref(address): 12 | return Ref.from_address(address) 13 | 14 | 15 | def test_mustache_re(): 16 | assert MustacheParser.split("{{foo}}") == [ref("foo")] 17 | assert MustacheParser.split("{{_}}") == [ref("_")] 18 | with pytest.raises(Ref.InvalidRefError): 19 | MustacheParser.split("{{4}}") 20 | def chrange(a,b): 21 | return ''.join(map(lambda ch: str(chr(ch)), range(ord(a), ord(b)+1))) 22 | slash_w = chrange('a','z') + chrange('A','Z') + chrange('0','9') + '_' 23 | assert MustacheParser.split("{{%s}}" % slash_w) == [ref(slash_w)] 24 | 25 | # bracketing 26 | assert MustacheParser.split("{{{foo}}") == ['{', ref('foo')] 27 | assert MustacheParser.split("{{foo}}}") == [ref('foo'), '}'] 28 | assert MustacheParser.split("{{{foo}}}") == ['{', ref('foo'), '}'] 29 | assert MustacheParser.split("{{}}") == ['{{}}'] 30 | assert MustacheParser.split("{{{}}}") == ['{{{}}}'] 31 | assert MustacheParser.split("{{{{foo}}}}") == ['{{', ref("foo"), '}}'] 32 | 33 | invalid_refs = ['!@', '-', '$', ':'] 34 | for val in invalid_refs: 35 | with pytest.raises(Ref.InvalidRefError): 36 | MustacheParser.split("{{%s}}" % val) 37 | 38 | 39 | def test_mustache_splitting(): 40 | assert MustacheParser.split("{{foo}}") == [ref("foo")] 41 | assert MustacheParser.split("{{&foo}}") == ["{{foo}}"] 42 | splits = MustacheParser.split('blech {{foo}} {{bar}} bonk {{&baz}} bling') 43 | assert splits == ['blech ', ref("foo"), ' ', ref('bar'), ' bonk ', '{{baz}}', ' bling'] 44 | 45 | 46 | def test_mustache_joining(): 47 | oe = Environment(foo = "foo herp", 48 | bar = "bar derp", 49 | baz = "baz blerp") 50 | 51 | joined, unbound = MustacheParser.join(MustacheParser.split("{{foo}}"), oe) 52 | assert joined == "foo herp" 53 | assert unbound == [] 54 | 55 | splits = MustacheParser.split('blech {{foo}} {{bar}} bonk {{&baz}} bling') 56 | joined, unbound = MustacheParser.join(splits, oe) 57 | assert joined == 'blech foo herp bar derp bonk {{baz}} bling' 58 | assert unbound == [] 59 | 60 | splits = MustacheParser.split('{{foo}} {{bar}} {{unbound}}') 61 | joined, unbound = MustacheParser.join(splits, oe) 62 | assert joined == 'foo herp bar derp {{unbound}}' 63 | assert unbound == [Ref.from_address('unbound')] 64 | 65 | 66 | def test_nested_mustache_resolution(): 67 | # straight 68 | oe = Environment(foo = '{{bar}}', bar = '{{baz}}', baz = 'hello') 69 | for pattern in ('{{foo}}', '{{bar}}', '{{baz}}', 'hello'): 70 | resolved, unbound = MustacheParser.resolve('%s world' % pattern, oe) 71 | assert resolved == 'hello world' 72 | assert unbound == [] 73 | 74 | # in structs 75 | class Process(Struct): 76 | name = Required(String) 77 | cmdline = String 78 | 79 | class Task(Struct): 80 | name = Default(String, '{{processes[0].name}}') 81 | processes = List(Process) 82 | 83 | task = Task(processes = [Process(name="hello"), Process(name="world")]) 84 | assert task.name().get() == 'hello' 85 | 86 | # iterably 87 | resolve_string = '{{foo[{{bar}}]}}' 88 | resolve_list = List(String)(["hello", "world"]) 89 | resolved, unbound = MustacheParser.resolve(resolve_string, Environment(foo=resolve_list, bar=0)) 90 | assert resolved == 'hello' 91 | assert unbound == [] 92 | resolved, unbound = MustacheParser.resolve(resolve_string, Environment(foo=resolve_list, bar="0")) 93 | assert resolved == 'hello' 94 | assert unbound == [] 95 | resolved, _ = MustacheParser.resolve(resolve_string, Environment(foo=resolve_list, bar=1)) 96 | assert resolved == 'world' 97 | resolved, unbound = MustacheParser.resolve(resolve_string, Environment(foo=resolve_list, bar=2)) 98 | assert resolved == '{{foo[2]}}' 99 | assert unbound == [ref('foo[2]')] 100 | 101 | 102 | def test_mustache_resolve_cycles(): 103 | with pytest.raises(MustacheParser.Uninterpolatable): 104 | MustacheParser.resolve('{{foo[{{bar}}]}} {{baz}}', 105 | Environment(foo = List(String)(["{{foo[{{bar}}]}}", "world"])), Environment(bar = 0)) 106 | -------------------------------------------------------------------------------- /tests/test_typing.py: -------------------------------------------------------------------------------- 1 | import pytest 2 | 3 | from pystachio import * 4 | 5 | 6 | def ref(address): 7 | return Ref.from_address(address) 8 | 9 | 10 | def test_basic_schemas(): 11 | BASIC_TYPES = (Integer, Float, String, Boolean) 12 | 13 | for typ in BASIC_TYPES: 14 | assert TypeFactory.new({}, *typ.serialize_type()) == typ 15 | 16 | for typ in BASIC_TYPES: 17 | assert isinstance(TypeFactory.new({}, *List(typ).serialize_type())([]), List(typ)) 18 | 19 | for typ1 in BASIC_TYPES: 20 | for typ2 in BASIC_TYPES: 21 | assert isinstance(TypeFactory.new({}, *Map(typ1, typ2).serialize_type())({}), Map(typ1, typ2)) 22 | 23 | 24 | def test_complex_schemas(): 25 | BASIC_TYPES = (Integer, Float, String, Boolean) 26 | LIST_TYPES = [List(bt) for bt in BASIC_TYPES] 27 | MAP_TYPES = [] 28 | ENUM_TYPES = [Enum(), Enum('Red', 'Green', 'Blue'), Enum('Animals', ('Dog', 'Cat'))] 29 | for typ1 in BASIC_TYPES: 30 | for typ2 in BASIC_TYPES: 31 | MAP_TYPES.append(Map(typ1,typ2)) 32 | for mt1 in (BASIC_TYPES, LIST_TYPES, MAP_TYPES, ENUM_TYPES): 33 | for mt2 in (BASIC_TYPES, LIST_TYPES, MAP_TYPES, ENUM_TYPES): 34 | for typ1 in mt1: 35 | for typ2 in mt2: 36 | assert isinstance(TypeFactory.new({}, *Map(typ1, typ2).serialize_type())({}), Map(typ1, typ2)) 37 | 38 | 39 | def test_composite_schemas_are_not_lossy(): 40 | class C1(Struct): 41 | required_attribute = Required(Integer) 42 | optional_attribute = Float 43 | default_attribute = Default(String, "Hello") 44 | required_list = Required(List(Boolean)) 45 | optional_list = List(Integer) 46 | default_list = Default(List(Float), [1.0, Float(2.0)]) 47 | required_map = Required(Map(String, Integer)) 48 | optional_map = Map(Integer, Float) 49 | default_map = Default(Map(Boolean, Integer), {True: 2, False: 3}) 50 | 51 | class M1(Struct): 52 | required_attribute = Required(Integer) 53 | optional_attribute = Float 54 | default_attribute = Default(String, "Hello") 55 | required_composite = Required(C1) 56 | optional_composite = C1 57 | default_composite = Default(C1, C1(required_attribute = 1, required_list = ["a", "b"])) 58 | 59 | BASIC_TYPES = [Integer, Float, String, Boolean, C1, M1] 60 | LIST_TYPES = [List(bt) for bt in BASIC_TYPES] 61 | MAP_TYPES = [] 62 | for typ1 in (Integer, C1, M1): 63 | for typ2 in (String, C1, M1): 64 | MAP_TYPES.append(Map(typ1,typ2)) 65 | for mt1 in BASIC_TYPES + LIST_TYPES + MAP_TYPES: 66 | for mt2 in BASIC_TYPES + LIST_TYPES + MAP_TYPES: 67 | t = Map(mt1, mt2) 68 | ser = t.serialize_type() 69 | serdes = TypeFactory.new({}, *ser) 70 | serdesser = serdes.serialize_type() 71 | assert ser == serdesser, 'Multiple ser/der cycles should not affect types.' 72 | default = Map(typ1, typ2)({}) 73 | assert Map(typ1, typ2)(default.get()) == default, ( 74 | 'Unwrapping/rewrapping should leave values intact: %s vs %s' % (typ1, typ2)) 75 | 76 | 77 | def test_recursive_unwrapping(): 78 | class Employee(Struct): 79 | name = Required(String) 80 | location = Default(String, "San Francisco") 81 | age = Integer 82 | 83 | class Employer(Struct): 84 | name = Required(String) 85 | employees = Default(List(Employee), [Employee(name = 'Bob')]) 86 | 87 | new_employer = TypeFactory.new({}, *Employer.serialize_type()) 88 | assert new_employer.serialize_type() == Employer.serialize_type() 89 | assert isinstance(new_employer(), Employer) 90 | nontwttr = Employer() 91 | assert not nontwttr.check().ok() 92 | repr(nontwttr.check()) 93 | twttr = Employer(name = 'Twitter') 94 | assert twttr.check().ok() 95 | repr(twttr.check()) 96 | 97 | 98 | def test_json(): 99 | import os, tempfile 100 | 101 | class Employee(Struct): 102 | name = Required(String) 103 | location = Default(String, "San Francisco") 104 | age = Integer 105 | 106 | class Employer(Struct): 107 | name = Required(String) 108 | employees = Default(List(Employee), [Employee(name = 'Bob')]) 109 | 110 | try: 111 | fd, fn = tempfile.mkstemp() 112 | os.close(fd) 113 | 114 | with open(fn, 'w') as fp: 115 | Employer.dump(fp) 116 | 117 | namespace = TypeFactory.load_file(fn) 118 | assert isinstance(namespace['Employer'](), Employer) 119 | 120 | finally: 121 | os.unlink(fn) 122 | 123 | 124 | def test_load(): 125 | class Employee(Struct): 126 | name = Required(String) 127 | location = Default(String, "San Francisco") 128 | age = Integer 129 | 130 | class Employer(Struct): 131 | name = Required(String) 132 | employees = Default(List(Employee), [Employee(name = 'Bob')]) 133 | 134 | employer_type = Employer.serialize_type() 135 | # It would be swell if I could 'del Employee' here. 136 | 137 | deposit = {} 138 | TypeFactory.load(employer_type, into=deposit) 139 | assert 'Employee' in deposit 140 | assert 'Employer' in deposit 141 | 142 | twttr = deposit['Employer'](name = 'Twitter') 143 | assert twttr.find(ref('employees[0].name')) == String('Bob') 144 | assert twttr.find(ref('employees[0].location')) == String('San Francisco') 145 | 146 | 147 | def test_unimplementeds(): 148 | # coverage-whore 149 | with pytest.raises(NotImplementedError): 150 | Type.type_factory() 151 | with pytest.raises(NotImplementedError): 152 | Type.type_parameters() 153 | with pytest.raises(NotImplementedError): 154 | Type().check() 155 | with pytest.raises(NotImplementedError): 156 | Type.serialize_type() 157 | -------------------------------------------------------------------------------- /tox.ini: -------------------------------------------------------------------------------- 1 | # Tox (http://tox.readthedocs.org) is a tool for running tests 2 | # in multiple virtualenvs. This configuration file will run the tornado 3 | # test suite on all supported python versions. To use it, "pip install tox" 4 | # and then run "tox" from this directory. 5 | 6 | [tox] 7 | envlist = 8 | # Basic configurations: Run the tests in both minimal installations 9 | # and with all optional dependencies. 10 | py36, 11 | py38, 12 | py39, 13 | py310, 14 | pypy3, 15 | isort-check 16 | 17 | [testenv] 18 | commands = py.test --basetemp={envtmpdir} -n 4 {posargs:} 19 | 20 | deps = 21 | pytest 22 | pytest-cov 23 | pytest-xdist 24 | 25 | # python will import relative to the current working directory by default, 26 | # so cd into the tox working directory to avoid picking up the working 27 | # copy of the files (especially important for the speedups module). 28 | changedir = tests 29 | 30 | # tox 1.6 passes --pre to pip by default, which currently has problems 31 | # installing pycurl and monotime (https://github.com/pypa/pip/issues/1405). 32 | # Remove it (it's not a part of {opts}) to only install real releases. 33 | install_command = pip install {opts} {packages} 34 | 35 | [testenv:isort-run] 36 | basepython = python3.8 37 | deps = isort 38 | commands = isort -ns __init__.py -rc {toxinidir}/setup.py {toxinidir}/pystachio {toxinidir}/tests 39 | 40 | [testenv:isort-check] 41 | basepython = python3.8 42 | deps = isort 43 | commands = isort -ns __init__.py -rc -c {toxinidir}/setup.py {toxinidir}/pystachio {toxinidir}/tests 44 | 45 | [testenv:coverage] 46 | basepython = python3.8 47 | commands = py.test \ 48 | --basetemp={envtmpdir} \ 49 | -n 4 \ 50 | --cov=pystachio --cov-report=term-missing --cov-report=html \ 51 | {posargs:} 52 | 53 | [testenv:py36] 54 | basepython = python3.6 55 | 56 | [testenv:py38] 57 | basepython = python3.8 58 | 59 | [testenv:py39] 60 | basepython = python3.9 61 | 62 | [testenv:py310] 63 | basepython = python3.10 64 | 65 | [testenv:pypy3] 66 | basepython = pypy3 67 | 68 | [testenv:jython] 69 | basepython = jython 70 | --------------------------------------------------------------------------------