├── AUTHORS
├── CONTRIBUTING.md
├── FAQ.md
├── LICENSE
├── README.md
├── pycolab
    ├── __init__.py
    ├── ascii_art.py
    ├── cropping.py
    ├── engine.py
    ├── examples
    │   ├── __init__.py
    │   ├── aperture.py
    │   ├── apprehend.py
    │   ├── better_scrolly_maze.py
    │   ├── classics
    │   │   ├── __init__.py
    │   │   ├── chain_walk.py
    │   │   ├── cliff_walk.py
    │   │   └── four_rooms.py
    │   ├── extraterrestrial_marauders.py
    │   ├── fluvial_natation.py
    │   ├── hello_world.py
    │   ├── ordeal.py
    │   ├── research
    │   │   ├── README.md
    │   │   ├── box_world
    │   │   │   ├── README.md
    │   │   │   └── box_world.py
    │   │   └── lp-rnn
    │   │   │   ├── README.md
    │   │   │   ├── cued_catch.py
    │   │   │   ├── sequence_recall.py
    │   │   │   └── t_maze.py
    │   ├── scrolly_maze.py
    │   ├── shockwave.py
    │   ├── tennnnnnnnnnnnnnnnnnnnnnnnis.py
    │   └── warehouse_manager.py
    ├── human_ui.py
    ├── plot.py
    ├── prefab_parts
    │   ├── __init__.py
    │   ├── drapes.py
    │   └── sprites.py
    ├── protocols
    │   ├── __init__.py
    │   ├── logging.py
    │   └── scrolling.py
    ├── rendering.py
    ├── storytelling.py
    ├── tests
    │   ├── __init__.py
    │   ├── ascii_art_test.py
    │   ├── cropping_test.py
    │   ├── engine_test.py
    │   ├── maze_walker_test.py
    │   ├── scrolling_test.py
    │   ├── story_test.py
    │   └── test_things.py
    └── things.py
└── setup.py


/AUTHORS:
--------------------------------------------------------------------------------
1 | # This is the official list of pycolab authors for copyright purposes.
2 | 
3 | # Names should be added to this file as:
4 | # Name or Organization <email address>
5 | # The email address is not required for organizations.
6 | 
7 | DeepMind Technologies
8 | 


--------------------------------------------------------------------------------
/CONTRIBUTING.md:
--------------------------------------------------------------------------------
 1 | # Contributing guidelines
 2 | 
 3 | ## How to become a contributor and submit your own code
 4 | 
 5 | ### Contributor License Agreements
 6 | 
 7 | Before we can accept your patches to pycolab, we have to jump a couple of legal
 8 | hurdles.
 9 | 
10 | Please fill out either the individual or corporate Contributor License Agreement
11 | (CLA).
12 | 
13 | *   If you are an individual writing original source code and you're sure you
14 |     own the intellectual property, then you'll need to sign an [individual
15 |     CLA](http://code.google.com/legal/individual-cla-v1.0.html).
16 | *   If you work for a company that wants to allow you to contribute your work,
17 |     then you'll need to sign a [corporate
18 |     CLA](http://code.google.com/legal/corporate-cla-v1.0.html).
19 | 
20 | Follow either of the two links above to access the appropriate CLA and
21 | instructions for how to sign and return it. Once we receive it, we'll be able to
22 | accept your pull requests.
23 | 
24 | ***NOTE***: Only original source code from you and other people that have signed
25 | the CLA can be accepted into the main repository.
26 | 
27 | ### Contributing code
28 | 
29 | We welcome useful contributions to pycolab!
30 | 
31 | Because pycolab aims to be a stable platform for describing various
32 | reinforcement learning tasks, API changes will need to be pretty important to be
33 | accepted into this "official" distribution. For some changes, the best option
34 | may be to fork pycolab and apply modifications to the fork.
35 | 
36 | New components are desirable, particularly new Sprites and Drapes (and protocols
37 | to support them). (A laser beam Drape would be especially handy...)
38 | 
39 | Internal speedups that don't make it harder to use or modify pycolab would be
40 | great, too!
41 | 


--------------------------------------------------------------------------------
/FAQ.md:
--------------------------------------------------------------------------------
 1 | # Some frequently-asked questions about pycolab
 2 | 
 3 | ### 1. How can a game return more data beyond observation/reward/discount?
 4 | 
 5 | Some pycolab applications want to return more data to the caller of
 6 | `Engine.play()` than what this method returns---for example, auxiliary rewards
 7 | or statistics for monitoring. The canonical way to do this is to place the extra
 8 | data in the game's `Plot` object; the caller can then retrieve the `Plot` and
 9 | the data inside via the `Engine.the_plot` attribute.
10 | 
11 | ### 2. How do I disable occlusion in the `layers` argument to `update()`?
12 | 
13 | There are several options.
14 | 
15 | Background: by default, the `layers` dict supplied to the `update()` methods of
16 | Sprites and Drapes show where different characters appear on the game board. For
17 | example, if you have two Drapes on the game board, A and B, and they look like
18 | this:
19 | 
20 | ```
21 |    A: AAAAAA     B: BBB...
22 |       AAAAAA        BBB...
23 |       ......        BBB...
24 |       ......        BBB...
25 | ```
26 | 
27 | and then if B comes after A in the `Engine`'s z-order, then the game board will
28 | show B occluding A:
29 | 
30 | ```
31 |    BBBAAA
32 |    BBBAAA
33 |    BBB...
34 |    BBB...
35 | ```
36 | 
37 | This means the 'A' and 'B' entries in the `layers` dict will look like this:
38 | 
39 | ```
40 |    A: ...XXX     B: XXX...
41 |       ...XXX        XXX...
42 |       ......        XXX...
43 |       ......        XXX...
44 | ```
45 | 
46 | The easiest approach is to disable occlusion in layers when constructing your
47 | pycolab `Engine`. See the `occlusion_in_layers` argument to the `Engine`
48 | constructor and to `ascii_art.ascii_art_to_game()`.
49 | 
50 | Besides changing the behaviour of `layers` arguments to `update()` methods, this
51 | approach also disables occlusion in the `layers` field of `Observation`
52 | namedtuples returned by `engine.play()`. If this is unacceptable, there is an
53 | easy second workaround. If you'd like have a look at all of A, don't look in
54 | `layers`. Instead, look at A's curtain directly through the `things` argument to
55 | `update`: `things['A'].curtain`. Note that this will only work for Drapes;
56 | Sprites don't express a curtain, only a (row, column) screen position under the
57 | `position` attribute. Hopefully this will be useful for locating an occluded
58 | sprite.
59 | 
60 | ### 3. What are the ways to get a game with top-down scrolling?
61 | 
62 | There are two "official" ways to do top-down, egocentric scrolling (where the
63 | world appears to move around the player as they walk through it).
64 | 
65 | If you have a world with a fixed-size, finite map, the easiest way is to program
66 | a pycolab game that renders the entire world as one giant observation, then to
67 | "crop" out a smaller observation centered on the player from each observation
68 | returned by `play()`. As the player moves, the cropped observation will follow
69 | along, giving the impression of scrolling. The `cropping` module provides some
70 | utility classes for this approach; the `better_scrolly_maze.py` and
71 | `tennnnnnnnnnnnnnnnnnnnnnnnis.py` examples show these classes in use.
72 | 
73 | If you have a world with an "infinite-ish" map (for example, a game where the
74 | scenery generates itself just-in-time as the player moves through it), or any
75 | world where it would be impractical to render the entire world into one big
76 | observation, one alternative approach has the game entities communicate with
77 | each other about scrolling (e.g. "the game should scroll one row downward now")
78 | and then update their own appearances to match. This approach is much more
79 | complicated than the cropping technique, but also more flexible. The scrolling
80 | protocol module (`protocols/scrolling.py`) provides utilities for game entities
81 | to talk about scrolling through the Plot object; the `MazeWalker` Sprite and
82 | `Scrolly` Drape in the `prefab_parts/` modules work with the scrolling protocol
83 | out of the box; and the `scrolly_maze.py` example demonstrates this approach.
84 | 


--------------------------------------------------------------------------------
/LICENSE:
--------------------------------------------------------------------------------
  1 | 
  2 |                                  Apache License
  3 |                            Version 2.0, January 2004
  4 |                         http://www.apache.org/licenses/
  5 | 
  6 |    TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
  7 | 
  8 |    1. Definitions.
  9 | 
 10 |       "License" shall mean the terms and conditions for use, reproduction,
 11 |       and distribution as defined by Sections 1 through 9 of this document.
 12 | 
 13 |       "Licensor" shall mean the copyright owner or entity authorized by
 14 |       the copyright owner that is granting the License.
 15 | 
 16 |       "Legal Entity" shall mean the union of the acting entity and all
 17 |       other entities that control, are controlled by, or are under common
 18 |       control with that entity. For the purposes of this definition,
 19 |       "control" means (i) the power, direct or indirect, to cause the
 20 |       direction or management of such entity, whether by contract or
 21 |       otherwise, or (ii) ownership of fifty percent (50%) or more of the
 22 |       outstanding shares, or (iii) beneficial ownership of such entity.
 23 | 
 24 |       "You" (or "Your") shall mean an individual or Legal Entity
 25 |       exercising permissions granted by this License.
 26 | 
 27 |       "Source" form shall mean the preferred form for making modifications,
 28 |       including but not limited to software source code, documentation
 29 |       source, and configuration files.
 30 | 
 31 |       "Object" form shall mean any form resulting from mechanical
 32 |       transformation or translation of a Source form, including but
 33 |       not limited to compiled object code, generated documentation,
 34 |       and conversions to other media types.
 35 | 
 36 |       "Work" shall mean the work of authorship, whether in Source or
 37 |       Object form, made available under the License, as indicated by a
 38 |       copyright notice that is included in or attached to the work
 39 |       (an example is provided in the Appendix below).
 40 | 
 41 |       "Derivative Works" shall mean any work, whether in Source or Object
 42 |       form, that is based on (or derived from) the Work and for which the
 43 |       editorial revisions, annotations, elaborations, or other modifications
 44 |       represent, as a whole, an original work of authorship. For the purposes
 45 |       of this License, Derivative Works shall not include works that remain
 46 |       separable from, or merely link (or bind by name) to the interfaces of,
 47 |       the Work and Derivative Works thereof.
 48 | 
 49 |       "Contribution" shall mean any work of authorship, including
 50 |       the original version of the Work and any modifications or additions
 51 |       to that Work or Derivative Works thereof, that is intentionally
 52 |       submitted to Licensor for inclusion in the Work by the copyright owner
 53 |       or by an individual or Legal Entity authorized to submit on behalf of
 54 |       the copyright owner. For the purposes of this definition, "submitted"
 55 |       means any form of electronic, verbal, or written communication sent
 56 |       to the Licensor or its representatives, including but not limited to
 57 |       communication on electronic mailing lists, source code control systems,
 58 |       and issue tracking systems that are managed by, or on behalf of, the
 59 |       Licensor for the purpose of discussing and improving the Work, but
 60 |       excluding communication that is conspicuously marked or otherwise
 61 |       designated in writing by the copyright owner as "Not a Contribution."
 62 | 
 63 |       "Contributor" shall mean Licensor and any individual or Legal Entity
 64 |       on behalf of whom a Contribution has been received by Licensor and
 65 |       subsequently incorporated within the Work.
 66 | 
 67 |    2. Grant of Copyright License. Subject to the terms and conditions of
 68 |       this License, each Contributor hereby grants to You a perpetual,
 69 |       worldwide, non-exclusive, no-charge, royalty-free, irrevocable
 70 |       copyright license to reproduce, prepare Derivative Works of,
 71 |       publicly display, publicly perform, sublicense, and distribute the
 72 |       Work and such Derivative Works in Source or Object form.
 73 | 
 74 |    3. Grant of Patent License. Subject to the terms and conditions of
 75 |       this License, each Contributor hereby grants to You a perpetual,
 76 |       worldwide, non-exclusive, no-charge, royalty-free, irrevocable
 77 |       (except as stated in this section) patent license to make, have made,
 78 |       use, offer to sell, sell, import, and otherwise transfer the Work,
 79 |       where such license applies only to those patent claims licensable
 80 |       by such Contributor that are necessarily infringed by their
 81 |       Contribution(s) alone or by combination of their Contribution(s)
 82 |       with the Work to which such Contribution(s) was submitted. If You
 83 |       institute patent litigation against any entity (including a
 84 |       cross-claim or counterclaim in a lawsuit) alleging that the Work
 85 |       or a Contribution incorporated within the Work constitutes direct
 86 |       or contributory patent infringement, then any patent licenses
 87 |       granted to You under this License for that Work shall terminate
 88 |       as of the date such litigation is filed.
 89 | 
 90 |    4. Redistribution. You may reproduce and distribute copies of the
 91 |       Work or Derivative Works thereof in any medium, with or without
 92 |       modifications, and in Source or Object form, provided that You
 93 |       meet the following conditions:
 94 | 
 95 |       (a) You must give any other recipients of the Work or
 96 |           Derivative Works a copy of this License; and
 97 | 
 98 |       (b) You must cause any modified files to carry prominent notices
 99 |           stating that You changed the files; and
100 | 
101 |       (c) You must retain, in the Source form of any Derivative Works
102 |           that You distribute, all copyright, patent, trademark, and
103 |           attribution notices from the Source form of the Work,
104 |           excluding those notices that do not pertain to any part of
105 |           the Derivative Works; and
106 | 
107 |       (d) If the Work includes a "NOTICE" text file as part of its
108 |           distribution, then any Derivative Works that You distribute must
109 |           include a readable copy of the attribution notices contained
110 |           within such NOTICE file, excluding those notices that do not
111 |           pertain to any part of the Derivative Works, in at least one
112 |           of the following places: within a NOTICE text file distributed
113 |           as part of the Derivative Works; within the Source form or
114 |           documentation, if provided along with the Derivative Works; or,
115 |           within a display generated by the Derivative Works, if and
116 |           wherever such third-party notices normally appear. The contents
117 |           of the NOTICE file are for informational purposes only and
118 |           do not modify the License. You may add Your own attribution
119 |           notices within Derivative Works that You distribute, alongside
120 |           or as an addendum to the NOTICE text from the Work, provided
121 |           that such additional attribution notices cannot be construed
122 |           as modifying the License.
123 | 
124 |       You may add Your own copyright statement to Your modifications and
125 |       may provide additional or different license terms and conditions
126 |       for use, reproduction, or distribution of Your modifications, or
127 |       for any such Derivative Works as a whole, provided Your use,
128 |       reproduction, and distribution of the Work otherwise complies with
129 |       the conditions stated in this License.
130 | 
131 |    5. Submission of Contributions. Unless You explicitly state otherwise,
132 |       any Contribution intentionally submitted for inclusion in the Work
133 |       by You to the Licensor shall be under the terms and conditions of
134 |       this License, without any additional terms or conditions.
135 |       Notwithstanding the above, nothing herein shall supersede or modify
136 |       the terms of any separate license agreement you may have executed
137 |       with Licensor regarding such Contributions.
138 | 
139 |    6. Trademarks. This License does not grant permission to use the trade
140 |       names, trademarks, service marks, or product names of the Licensor,
141 |       except as required for reasonable and customary use in describing the
142 |       origin of the Work and reproducing the content of the NOTICE file.
143 | 
144 |    7. Disclaimer of Warranty. Unless required by applicable law or
145 |       agreed to in writing, Licensor provides the Work (and each
146 |       Contributor provides its Contributions) on an "AS IS" BASIS,
147 |       WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
148 |       implied, including, without limitation, any warranties or conditions
149 |       of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
150 |       PARTICULAR PURPOSE. You are solely responsible for determining the
151 |       appropriateness of using or redistributing the Work and assume any
152 |       risks associated with Your exercise of permissions under this License.
153 | 
154 |    8. Limitation of Liability. In no event and under no legal theory,
155 |       whether in tort (including negligence), contract, or otherwise,
156 |       unless required by applicable law (such as deliberate and grossly
157 |       negligent acts) or agreed to in writing, shall any Contributor be
158 |       liable to You for damages, including any direct, indirect, special,
159 |       incidental, or consequential damages of any character arising as a
160 |       result of this License or out of the use or inability to use the
161 |       Work (including but not limited to damages for loss of goodwill,
162 |       work stoppage, computer failure or malfunction, or any and all
163 |       other commercial damages or losses), even if such Contributor
164 |       has been advised of the possibility of such damages.
165 | 
166 |    9. Accepting Warranty or Additional Liability. While redistributing
167 |       the Work or Derivative Works thereof, You may choose to offer,
168 |       and charge a fee for, acceptance of support, warranty, indemnity,
169 |       or other liability obligations and/or rights consistent with this
170 |       License. However, in accepting such obligations, You may act only
171 |       on Your own behalf and on Your sole responsibility, not on behalf
172 |       of any other Contributor, and only if You agree to indemnify,
173 |       defend, and hold each Contributor harmless for any liability
174 |       incurred by, or claims asserted against, such Contributor by reason
175 |       of your accepting any such warranty or additional liability.
176 | 
177 |    END OF TERMS AND CONDITIONS
178 | 
179 |    APPENDIX: How to apply the Apache License to your work.
180 | 
181 |       To apply the Apache License to your work, attach the following
182 |       boilerplate notice, with the fields enclosed by brackets "[]"
183 |       replaced with your own identifying information. (Don't include
184 |       the brackets!)  The text should be enclosed in the appropriate
185 |       comment syntax for the file format. We also recommend that a
186 |       file or class name and description of purpose be included on the
187 |       same "printed page" as the copyright notice for easier
188 |       identification within third-party archives.
189 | 
190 |    Copyright [yyyy] [name of copyright owner]
191 | 
192 |    Licensed under the Apache License, Version 2.0 (the "License");
193 |    you may not use this file except in compliance with the License.
194 |    You may obtain a copy of the License at
195 | 
196 |        http://www.apache.org/licenses/LICENSE-2.0
197 | 
198 |    Unless required by applicable law or agreed to in writing, software
199 |    distributed under the License is distributed on an "AS IS" BASIS,
200 |    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
201 |    See the License for the specific language governing permissions and
202 |    limitations under the License.
203 | 


--------------------------------------------------------------------------------
/README.md:
--------------------------------------------------------------------------------
 1 | # The `pycolab` game engine.
 2 | 
 3 | A highly-customisable gridworld game engine with some batteries included.
 4 | Make your own gridworld games to test reinforcement learning agents!
 5 | 
 6 | ## Play some games!
 7 | 
 8 | If you're new, why not try playing some games first? For the full colour
 9 | experience on most UNIX-compatible systems:
10 | 
11 |   1. crack open a nice, new, modern terminal (iterm2 on Mac, gnome-terminal or
12 |      xterm on linux). (Avoid screen/tmux for now---just the terminal, please.)
13 |   2. set the terminal type to `xterm-256color` (usually, you do this by typing
14 |      `export TERM=xterm-256color` at the command prompt).
15 |   3. run the example games! One easy way is to cd to just above the `pycolab/`
16 |      library directory (that is, cd to the root directory of the git repository
17 |      or the distribution tarball, if you're using either of those) and run
18 |      python with the appropriate `PYTHONPATH` environment variable. Example
19 |      command line for `bash`-like shells:
20 |      `PYTHONPATH=. python -B pycolab/examples/scrolly_maze.py`.
21 | 
22 | ## Okay, install some dependencies first.
23 | 
24 | If that didn't work, you may need to obtain the following software packages that
25 | pycolab depends on:
26 | 
27 |   1. Python 2.7, or Python 3.4 and up. We've had success with 2.7.6, 3.4.3, and
28 |      3.6.3; other versions may work.
29 |   2. Numpy. Our version is 1.13.3, but 1.9 seems to have the necessary features.
30 |   3. Scipy, but only for running one of the examples. We have 0.13.3.
31 | 
32 | ## Overview
33 | 
34 | pycolab is extensively documented and commented, so the best ways to understand
35 | how to use it are:
36 | 
37 |   - check out examples in the `examples/` subdirectory,
38 |   - read docstrings in the `.py` files.
39 | 
40 | For docstring reading, the best order is probably this one---stopping whenever
41 | you like (the docs aren't going anywhere...):
42 | 
43 |   1. the docstring for the `Engine` class in `engine.py`
44 |   2. the docstrings for the classes in `things.py`
45 | 
46 | Those two are probably the only bits of "required" reading in order to get an
47 | idea of what's going on in `examples/`. From there, the following reading may be
48 | of interest:
49 | 
50 |   3. `plot.py`: how do game components talk to one another---and how do I
51 |      give the agent rewards and terminate episodes?
52 |   4. `human_ui.py`: how can I try my game out myself?
53 |   5. `prefab_parts/sprites.py`: useful `Sprite` subclasses, including
54 |      `MazeWalker`, a pixel that can walk around but not through walls and
55 |      obstacles.
56 |   6. `cropping.py`: how can I generate the illusion of top-down scrolling by
57 |      cleverly cropping an observation around a particular moving game element?
58 |      (This is a common way to build partial observability into a game.)
59 | 
60 | Don't forget that you can *always read the tests*, too. These can help
61 | demonstrate by example what all the various components do.
62 | 
63 | ## Disclaimer
64 | 
65 | This is not an official Google product.
66 | 
67 | We just thought you should know that.
68 | 


--------------------------------------------------------------------------------
/pycolab/__init__.py:
--------------------------------------------------------------------------------
 1 | # Copyright 2017 the pycolab Authors
 2 | #
 3 | # Licensed under the Apache License, Version 2.0 (the "License");
 4 | # you may not use this file except in compliance with the License.
 5 | # You may obtain a copy of the License at
 6 | #
 7 | #     https://www.apache.org/licenses/LICENSE-2.0
 8 | #
 9 | # Unless required by applicable law or agreed to in writing, software
10 | # distributed under the License is distributed on an "AS IS" BASIS,
11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12 | # See the License for the specific language governing permissions and
13 | # limitations under the License.
14 | 


--------------------------------------------------------------------------------
/pycolab/ascii_art.py:
--------------------------------------------------------------------------------
  1 | # Copyright 2017 the pycolab Authors
  2 | #
  3 | # Licensed under the Apache License, Version 2.0 (the "License");
  4 | # you may not use this file except in compliance with the License.
  5 | # You may obtain a copy of the License at
  6 | #
  7 | #     https://www.apache.org/licenses/LICENSE-2.0
  8 | #
  9 | # Unless required by applicable law or agreed to in writing, software
 10 | # distributed under the License is distributed on an "AS IS" BASIS,
 11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 12 | # See the License for the specific language governing permissions and
 13 | # limitations under the License.
 14 | 
 15 | """Utilities to build a pycolab game from ASCII art diagrams."""
 16 | 
 17 | from __future__ import absolute_import
 18 | from __future__ import division
 19 | from __future__ import print_function
 20 | 
 21 | import itertools
 22 | 
 23 | import numpy as np
 24 | 
 25 | from pycolab import engine
 26 | from pycolab import things
 27 | 
 28 | import six
 29 | 
 30 | 
 31 | def ascii_art_to_game(art,
 32 |                       what_lies_beneath,
 33 |                       sprites=None, drapes=None, backdrop=things.Backdrop,
 34 |                       update_schedule=None,
 35 |                       z_order=None,
 36 |                       occlusion_in_layers=True):
 37 |   """Construct a pycolab game from an ASCII art diagram.
 38 | 
 39 |   This function helps to turn ASCII art diagrams like the following
 40 |   (which is a Sokoban-like puzzle):
 41 | 
 42 |       [' @@@@@@ ',
 43 |        ' @  . @ ',        # '@' means "wall"
 44 |        '@@ab @@ ',        # 'P' means "player"
 45 |        '@  .c @ ',        # '.' means "box storage location"
 46 |        '@.  dP@ ',        # 'a'-'g' are all for separate boxes
 47 |        '@.@@@@@@',        # ' ' means "open, traversable space"
 48 |        '@ @ @@ @',
 49 |        '@ e  . @',
 50 |        '@@@@@@@@',]
 51 | 
 52 |   into pycolab games. The basic idea is that you supply the diagram, along with
 53 |   hints about which characters correspond to `Sprite`s and `Drape`s and the
 54 |   classes that implement those `Sprite`s and `Drape`s. This function then
 55 |   returns an initialised `Engine` object, all ready for you to call the
 56 |   `its_showtime` method and start the game.
 57 | 
 58 |   Several of this function's arguments require you to supply subclasses of the
 59 |   classes found in `things.py`. If your subclass constructors take the same
 60 |   number of arguments as their `things.py` superclasses, then they can be
 61 |   listed directly. Otherwise, you will need to pack the subclasses and their
 62 |   additional `args` and `kwargs` into a `Partial` object. So, for example, if
 63 |   you have a `Sprite` subclass with a constructor like this:
 64 | 
 65 |       class MySprite(Sprite):
 66 |         def __init__(self, corner, position, character, mood, drink_quantity):
 67 |           ...
 68 | 
 69 |   you could package `MySprite` and the "extra" arguments in any of the following
 70 |   ways (among others):
 71 | 
 72 |       Partial(MySprite, 'drowsy', 'two pints')
 73 |       Partial(MySprite, 'yawning', drink_quantity='three pints')
 74 |       Partial(MySprite, mood='asleep', drink_quantity='four pints')
 75 | 
 76 |   Args:
 77 |     art: An ASCII art diagram depicting a game board. This should be a list or
 78 |         tuple whose values are all strings containing the same number of ASCII
 79 |         characters.
 80 |     what_lies_beneath: a single-character ASCII string that will be substituted
 81 |         into the `art` diagram at all places where a character that keys
 82 |         `sprites` or `drapes` is found; *or*, this can also be an entire second
 83 |         ASCII art diagram whose values will be substituted into `art` at (only)
 84 |         those locations. In either case, the resulting diagram will be used to
 85 |         initialise the game's `Backdrop`.
 86 |     sprites: a dict mapping single-character ASCII strings to `Sprite` classes
 87 |         (not objects); or to `Partial` objects that hold the classes and "extra"
 88 |         `args`es and `kwargs`es to use during their construction. It's fine if a
 89 |         character used as a key doesn't appear in the `art` diagram: in this
 90 |         case, we assume that the corresponding `Sprite` will be located at
 91 |         `0, 0`. (If you intend your `Sprite` to be invisible, the `Sprite` will
 92 |         have to take care of that on its own after it is built.) (Optional;
 93 |         omit if your game has no sprites.)
 94 |     drapes: a dict mapping single-character ASCII strings to `Drape` classes
 95 |         (not objects); or to `Partial` objects that hold the classes and "extra"
 96 |         `args`es and `kwargs`es to use during their construction. It's fine if
 97 |         a character used as a key doesn't appear in the `art` diagram: in this
 98 |         case, we assume that the `Drape`'s curtain (i.e. its mask) is completely
 99 |         empty (i.e. False). (Optional; omit if your game has no drapes.)
100 |     backdrop: a `Backdrop` class (not an object); or a `Partial` object that
101 |         holds the class and "extra" `args` and `kwargs` to use during its
102 |         construction. (Optional; if unset, `Backdrop` is used directly, which
103 |         is fine for a game where the background scenery never changes and
104 |         contains no game logic.)
105 |     update_schedule: A list of single-character ASCII strings indicating the
106 |         order in which the `Sprite`s and `Drape`s should be consulted by the
107 |         `Engine` for updates; or, a list of lists that imposes an ordering as
108 |         well, but that groups the entities in each list into separate
109 |         update groups (refer to `Engine` documentation). (Optional; if
110 |         unspecified, the ordering will be arbitrary---be mindful of this if your
111 |         game uses advanced features like scrolling, where update order is pretty
112 |         important.)
113 |     z_order: A list of single-character ASCII strings indicating the depth
114 |         ordering of the `Sprite`s and `Drape`s (from back to front). (Optional;
115 |         if unspecified, the ordering will be the same as what's used for
116 |         `update_schedule`).
117 |     occlusion_in_layers: If `True` (the default), game entities or `Backdrop`
118 |         characters that occupy the same position on the game board will be
119 |         rendered into the `layers` member of `rendering.Observation`s with
120 |         "occlusion": only the entity that appears latest in the game's Z-order
121 |         will have its `layers` entry at that position set to `True`. If
122 |         `False`, all entities and `Backdrop` characters at that position will
123 |         have `True` in their `layers` entries there.
124 | 
125 |         This flag does not change the rendering of the "flat" `board` member
126 |         of `Observation`, which always paints game entities on top of each
127 |         other as dictated by the Z-order.
128 | 
129 |         **NOTE: This flag also determines the occlusion behavior in `layers`
130 |         arguments to all game entities' `update` methods; see docstrings in
131 |         [things.py] for details.**
132 | 
133 |   Returns:
134 |     An initialised `Engine` object as described.
135 | 
136 |   Raises:
137 |     TypeError: when `update_schedule` is neither a "flat" list of characters
138 |         nor a list of lists of characters.
139 |     ValueError: numerous causes, nearly always instances of the user not heeding
140 |         the requirements stipulated in Args:. The exception messages should make
141 |         most errors fairly easy to debug.
142 |   """
143 |   ### 1. Set default arguments, normalise arguments, derive various things ###
144 | 
145 |   # Convert sprites and drapes to be dicts of Partials only. "Bare" Sprite
146 |   # and Drape classes become Partials with no args or kwargs.
147 |   if sprites is None: sprites = {}
148 |   if drapes is None: drapes = {}
149 |   sprites = {char: sprite if isinstance(sprite, Partial) else Partial(sprite)
150 |              for char, sprite in six.iteritems(sprites)}
151 |   drapes = {char: drape if isinstance(drape, Partial) else Partial(drape)
152 |             for char, drape in six.iteritems(drapes)}
153 | 
154 |   # Likewise, turn a bare Backdrop class into an argument-free Partial.
155 |   if not isinstance(backdrop, Partial): backdrop = Partial(backdrop)
156 | 
157 |   # Compile characters corresponding to all Sprites and Drapes.
158 |   non_backdrop_characters = set()
159 |   non_backdrop_characters.update(sprites.keys())
160 |   non_backdrop_characters.update(drapes.keys())
161 |   if update_schedule is None: update_schedule = list(non_backdrop_characters)
162 | 
163 |   # If update_schedule is a string (someone wasn't reading the docs!),
164 |   # gracefully convert it to a list of single-character strings.
165 |   if isinstance(update_schedule, str): update_schedule = list(update_schedule)
166 | 
167 |   # If update_schedule is not a list-of-lists already, convert it to be one.
168 |   if all(isinstance(item, str) for item in update_schedule):
169 |     update_schedule = [update_schedule]
170 | 
171 |   ### 2. Argument checking and derivation of more... things ###
172 | 
173 |   # The update schedule (flattened) is the basis for the default z-order.
174 |   try:
175 |     flat_update_schedule = list(itertools.chain.from_iterable(update_schedule))
176 |   except TypeError:
177 |     raise TypeError('if any element in update_schedule is an iterable (like a '
178 |                     'list), all elements in update_schedule must be')
179 |   if set(flat_update_schedule) != non_backdrop_characters:
180 |     raise ValueError('if specified, update_schedule must list each sprite and '
181 |                      'drape exactly once.')
182 | 
183 |   # The default z-order is derived from there.
184 |   if z_order is None: z_order = flat_update_schedule
185 |   if set(z_order) != non_backdrop_characters:
186 |     raise ValueError('if specified, z_order must list each sprite and drape '
187 |                      'exactly once.')
188 | 
189 |   # All this checking is rather strict, but as this function is likely to be
190 |   # popular with new users, it will help to fail with a helpful error message
191 |   # now rather than an incomprehensible stack trace later.
192 |   if isinstance(what_lies_beneath, str) and len(what_lies_beneath) != 1:
193 |     raise ValueError(
194 |         'what_lies_beneath may either be a single-character ASCII string or '
195 |         'a list of ASCII-character strings')
196 | 
197 |   # Note that the what_lies_beneath check works for characters and lists both.
198 |   try:
199 |     _ = [ord(character) for character in ''.join(what_lies_beneath)]
200 |     _ = [ord(character) for character in non_backdrop_characters]
201 |     _ = [ord(character) for character in z_order]
202 |     _ = [ord(character) for character in flat_update_schedule]
203 |   except TypeError:
204 |     raise ValueError(
205 |         'keys of sprites, keys of drapes, what_lies_beneath (or its entries), '
206 |         'values in z_order, and (possibly nested) values in update_schedule '
207 |         'must all be single-character ASCII strings.')
208 | 
209 |   if non_backdrop_characters.intersection(''.join(what_lies_beneath)):
210 |     raise ValueError(
211 |         'any character specified in what_lies_beneath must not be one of the '
212 |         'characters used as keys in the sprites or drapes arguments.')
213 | 
214 |   ### 3. Convert all ASCII art to numpy arrays ###
215 | 
216 |   # Now convert the ASCII art array to a numpy array of uint8s.
217 |   art = ascii_art_to_uint8_nparray(art)
218 | 
219 |   # In preparation for masking out sprites and drapes from the ASCII art array
220 |   # (to make the background), do similar for what_lies_beneath.
221 |   if isinstance(what_lies_beneath, str):
222 |     what_lies_beneath = np.full_like(art, ord(what_lies_beneath))
223 |   else:
224 |     what_lies_beneath = ascii_art_to_uint8_nparray(what_lies_beneath)
225 |     if art.shape != what_lies_beneath.shape:
226 |       raise ValueError(
227 |           'if not a single ASCII character, what_lies_beneath must be ASCII '
228 |           'art whose shape is the same as that of the ASCII art in art.')
229 | 
230 |   ### 4. Other miscellaneous preparation ###
231 | 
232 |   # This dict maps the characters associated with Sprites and Drapes to an
233 |   # identifier for the update group to which they belong. The sorted order of
234 |   # the identifiers matches the group ordering in update_schedule, but is
235 |   # otherwise generic.
236 |   update_group_for = {}
237 |   for i, update_group in enumerate(update_schedule):
238 |     group_id = '{:05d}'.format(i)
239 |     update_group_for.update({character: group_id for character in update_group})
240 | 
241 |   ### 5. Construct engine; populate with Sprites and Drapes ###
242 | 
243 |   game = engine.Engine(*art.shape, occlusion_in_layers=occlusion_in_layers)
244 | 
245 |   # Sprites and Drapes are added according to the depth-first traversal of the
246 |   # update schedule.
247 |   for character in flat_update_schedule:
248 |     # Switch to this character's update group.
249 |     game.update_group(update_group_for[character])
250 |     # Find locations where this character appears in the ASCII art.
251 |     mask = art == ord(character)
252 | 
253 |     if character in drapes:
254 |       # Add the drape to the Engine.
255 |       partial = drapes[character]
256 |       game.add_prefilled_drape(character, mask,
257 |                                partial.pycolab_thing,
258 |                                *partial.args, **partial.kwargs)
259 | 
260 |     if character in sprites:
261 |       # Get the location of the sprite in the ASCII art, if there was one.
262 |       row, col = np.where(mask)
263 |       if len(row) > 1:
264 |         raise ValueError('sprite character {} can appear in at most one place '
265 |                          'in art.'.format(character))
266 |       # If there was a location, convert it to integer values; otherwise, 0,0.
267 |       # gpylint doesn't know how implicit bools work with numpy arrays...
268 |       row, col = (int(row), int(col)) if len(row) > 0 else (0, 0)  # pylint: disable=g-explicit-length-test
269 | 
270 |       # Add the sprite to the Engine.
271 |       partial = sprites[character]
272 |       game.add_sprite(character, (row, col),
273 |                       partial.pycolab_thing,
274 |                       *partial.args, **partial.kwargs)
275 | 
276 |     # Clear out the newly-added Sprite or Drape from the ASCII art.
277 |     art[mask] = what_lies_beneath[mask]
278 | 
279 |   ### 6. Impose specified Z-order ###
280 | 
281 |   game.set_z_order(z_order)
282 | 
283 |   ### 7. Add the Backdrop to the engine ###
284 | 
285 |   game.set_prefilled_backdrop(
286 |       characters=''.join(chr(c) for c in np.unique(art)),
287 |       prefill=art.view(np.uint8),
288 |       backdrop_class=backdrop.pycolab_thing,
289 |       *backdrop.args, **backdrop.kwargs)
290 | 
291 |   # That's all, folks!
292 |   return game
293 | 
294 | 
295 | def ascii_art_to_uint8_nparray(art):
296 |   """Construct a numpy array of dtype `uint8` from an ASCII art diagram.
297 | 
298 |   This function takes ASCII art diagrams (expressed as lists or tuples of
299 |   equal-length strings) and derives 2-D numpy arrays with dtype `uint8`.
300 | 
301 |   Args:
302 |     art: An ASCII art diagram; this should be a list or tuple whose values are
303 |         all strings containing the same number of ASCII characters.
304 | 
305 |   Returns:
306 |     A 2-D numpy array as described.
307 | 
308 |   Raises:
309 |     ValueError: `art` wasn't an ASCII art diagram, as described; this could be
310 |       because the strings it is made of contain non-ASCII characters, or do not
311 |       have constant length.
312 |     TypeError: `art` was not a list of strings.
313 |   """
314 |   error_text = (
315 |       'the argument to ascii_art_to_uint8_nparray must be a list (or tuple) '
316 |       'of strings containing the same number of strictly-ASCII characters.')
317 |   try:
318 |     art = np.vstack([np.frombuffer(line.encode('ascii'), dtype=np.uint8)
319 |                      for line in art])
320 |   except AttributeError as e:
321 |     if isinstance(art, (list, tuple)) and all(
322 |         isinstance(row, (list, tuple)) for row in art):
323 |       error_text += ' Did you pass a list of list of single characters?'
324 |     raise TypeError('{} (original error: {})'.format(error_text, e))
325 |   except ValueError as e:
326 |     raise ValueError('{} (original error from numpy: {})'.format(error_text, e))
327 |   if np.any(art > 127): raise ValueError(error_text)
328 |   return art
329 | 
330 | 
331 | class Partial(object):
332 |   """Holds a pycolab "thing" and its extra constructor arguments.
333 | 
334 |   In a spirit similar to `functools.partial`, a `Partial` object holds a
335 |   subclass of one of the pycolab game entities described in `things.py`, along
336 |   with any "extra" arguments required for its constructor (i.e. those besides
337 |   the constructor arguments specified by the `things.py` base class
338 |   constructors).
339 | 
340 |   `Partial` instances can be used to pass `Sprite`, `Drape` and `Backdrop`
341 |   subclasses *and* their necessary "extra" constructor arguments to
342 |   `ascii_art_to_game`.
343 |   """
344 | 
345 |   def __init__(self, pycolab_thing, *args, **kwargs):
346 |     """Construct a new Partial object.
347 | 
348 |     Args:
349 |       pycolab_thing: a `Backdrop`, `Sprite`, or `Drape` subclass (note: not an
350 |           object, the class itself).
351 |       *args: "Extra" positional arguments for the `pycolab_thing` constructor.
352 |       **kwargs: "Extra" keyword arguments for the `pycolab_thing` constructor.
353 | 
354 |     Raises:
355 |       TypeError: `pycolab_thing` was not a `Backdrop`, a `Sprite`, or a `Drape`.
356 |     """
357 |     if not issubclass(pycolab_thing,
358 |                       (things.Backdrop, things.Sprite, things.Drape)):
359 |       raise TypeError('the pycolab_thing argument to ascii_art.Partial must be '
360 |                       'a Backdrop, Sprite, or Drape subclass.')
361 | 
362 |     self.pycolab_thing = pycolab_thing
363 |     self.args = args
364 |     self.kwargs = kwargs
365 | 


--------------------------------------------------------------------------------
/pycolab/examples/__init__.py:
--------------------------------------------------------------------------------
 1 | # Copyright 2017 the pycolab Authors
 2 | #
 3 | # Licensed under the Apache License, Version 2.0 (the "License");
 4 | # you may not use this file except in compliance with the License.
 5 | # You may obtain a copy of the License at
 6 | #
 7 | #     https://www.apache.org/licenses/LICENSE-2.0
 8 | #
 9 | # Unless required by applicable law or agreed to in writing, software
10 | # distributed under the License is distributed on an "AS IS" BASIS,
11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12 | # See the License for the specific language governing permissions and
13 | # limitations under the License.
14 | 


--------------------------------------------------------------------------------
/pycolab/examples/aperture.py:
--------------------------------------------------------------------------------
  1 | # Copyright 2017 the pycolab Authors
  2 | #
  3 | # Licensed under the Apache License, Version 2.0 (the "License");
  4 | # you may not use this file except in compliance with the License.
  5 | # You may obtain a copy of the License at
  6 | #
  7 | #     https://www.apache.org/licenses/LICENSE-2.0
  8 | #
  9 | # Unless required by applicable law or agreed to in writing, software
 10 | # distributed under the License is distributed on an "AS IS" BASIS,
 11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 12 | # See the License for the specific language governing permissions and
 13 | # limitations under the License.
 14 | 
 15 | """A game that has absolutely nothing to do with Portal.
 16 | 
 17 | In Aperture, your goal is to reach the cranachan (`C`). Use your Aperture
 18 | Blaster to convert special walls (dark blue) into 'apertures' (blue). You can
 19 | create up to two apertures at a time, and walking into one aperture will
 20 | transport you instantly to the other! You'll need to use you Blaster to get
 21 | around this world of platforms surrounded by deadly green ooze.
 22 | 
 23 | Command-line usage: `aperture.py <level>`, where `<level>` is an optional
 24 | integer argument selecting Aperture levels 0, 1, or 2.
 25 | 
 26 | Keys:
 27 |   up, down, left, right - move.
 28 |   w, a, s, d - shoot blaster up, left, down, right.
 29 |   q - quit.
 30 | """
 31 | 
 32 | from __future__ import absolute_import
 33 | from __future__ import division
 34 | from __future__ import print_function
 35 | 
 36 | import curses
 37 | import sys
 38 | 
 39 | from pycolab import ascii_art
 40 | from pycolab import human_ui
 41 | from pycolab import things as plab_things
 42 | from pycolab.prefab_parts import sprites as prefab_sprites
 43 | 
 44 | from six.moves import xrange  # pylint: disable=redefined-builtin
 45 | 
 46 | 
 47 | LEVELS = [
 48 |     # Level 0: Entranceway.
 49 |     [
 50 |         '##############',
 51 |         '## A ...    @#',
 52 |         '##   ...    @#',
 53 |         '##@@@...    @#',
 54 |         '##......    @#',
 55 |         '##......    @#',
 56 |         '#@   ...    @#',
 57 |         '#@   ...    @#',
 58 |         '##   .......##',
 59 |         '## C .......##',
 60 |         '##############'
 61 |     ],
 62 |     # Level 1: Alien Containment Zone.
 63 |     [
 64 |         '#####################',
 65 |         '##A#@###########C#@##',
 66 |         '## # #         # # ##',
 67 |         '##   #  ZZ ZZ  #   ##',
 68 |         '## ### Z  Z  Z ### ##',
 69 |         '##.#    ZZ ZZ    ..##',
 70 |         '##.#    ZZZZZ    ..##',
 71 |         '##.#   Z Z Z Z   ..##',
 72 |         '##.#  Z  Z Z  Z  # ##',
 73 |         '## #  Z Z   Z Z  # ##',
 74 |         '## #             # ##',
 75 |         '## ............... @#',
 76 |         '##@##################',
 77 |         '#####################',
 78 |     ],
 79 |     # Level 2: Turbine Room.
 80 |     [
 81 |         '####################',
 82 |         '#########@@@########',
 83 |         '##C         ########',
 84 |         '########## ##@######',
 85 |         '#A #.........     ##',
 86 |         '## #.....   ..... @#',
 87 |         '## #..... @ ..... @#',
 88 |         '## #......#......###',
 89 |         '##  ..  ..#..  ..@##',
 90 |         '##  .. @##Z##@ .. ##',
 91 |         '##  ..  ..#..  .. ##',
 92 |         '##@@......#...... ##',
 93 |         '####..... @ ..... ##',
 94 |         '##@ .....   ..... ##',
 95 |         '##@ ............. @#',
 96 |         '####...     .....###',
 97 |         '#######@@@@@########',
 98 |         '####################'
 99 |     ]
100 | ]
101 | 
102 | FG_COLOURS = {
103 |     'A': (999, 500, 0),    # Player wears an orange jumpsuit.
104 |     'X': (200, 200, 999),  # Apertures are blue.
105 |     '#': (700, 700, 700),  # Normal wall, bright grey.
106 |     '@': (400, 400, 600),  # Special wall, grey-blue.
107 |     '.': (100, 300, 100),  # Green ooze.
108 |     'C': (999, 0, 0),      # Cranachan.
109 |     ' ': (200, 200, 200),  # Floor.
110 |     'Z': (0, 999, 0)       # Alien skin.
111 | }
112 | 
113 | BG_COLOURS = {
114 |     'A': (200, 200, 200),
115 |     'X': (200, 200, 999),
116 |     '#': (800, 800, 800),
117 |     '@': (400, 400, 600),
118 |     '.': (100, 300, 100),
119 |     'C': (999, 800, 800),
120 |     ' ': (200, 200, 200)
121 | }
122 | 
123 | 
124 | class PlayerSprite(prefab_sprites.MazeWalker):
125 |   """The player.
126 | 
127 |   Parent class handles basic movement and collision detection. The special
128 |   aperture drape handles the blaster. This class handles quitting the game and
129 |   cranachan-consumption detection.
130 |   """
131 | 
132 |   def __init__(self, corner, position, character):
133 |     super(PlayerSprite, self).__init__(
134 |         corner, position, character, impassable='#.@')
135 | 
136 |   def update(self, actions, board, layers, backdrop, things, the_plot):
137 |     del backdrop  # Unused.
138 | 
139 |     # Handles basic movement, but not movement through apertures.
140 |     if actions == 0:  # go upward?
141 |       self._north(board, the_plot)
142 |     elif actions == 1:  # go downward?
143 |       self._south(board, the_plot)
144 |     elif actions == 2:  # go leftward?
145 |       self._west(board, the_plot)
146 |     elif actions == 3:  # go rightward?
147 |       self._east(board, the_plot)
148 |     elif actions == 9:  # quit?
149 |       the_plot.terminate_episode()
150 | 
151 |     # Did we walk onto exit? If so, we win!
152 |     if layers['C'][self.position]:
153 |       the_plot.add_reward(1)
154 |       the_plot.terminate_episode()
155 | 
156 |     # Did we walk onto an aperture? If so, then teleport!
157 |     if layers['X'][self.position]:
158 |       destinations = [p for p in things['X'].apertures if p != self.position]
159 |       if destinations: self._teleport(destinations[0])
160 | 
161 | 
162 | class ApertureDrape(plab_things.Drape):
163 |   """Drape for all apertures.
164 | 
165 |   Tracks aperture locations, creation of new apertures using blaster, and will
166 |   teleport the player if necessary.
167 |   """
168 | 
169 |   def __init__(self, curtain, character):
170 |     super(ApertureDrape, self).__init__(curtain, character)
171 |     self._apertures = [None, None]
172 | 
173 |   def update(self, actions, board, layers, backdrop, things, the_plot):
174 |     ply_y, ply_x = things['A'].position
175 | 
176 |     if actions == 5:  # w - shoot up?
177 |       dx, dy = 0, -1
178 |     elif actions == 6:  # a - shoot left?
179 |       dx, dy = -1, 0
180 |     elif actions == 7:  # s - shoot down?
181 |       dx, dy = 0, 1
182 |     elif actions == 8:  # d - shoot right?
183 |       dx, dy = 1, 0
184 |     else:
185 |       return
186 | 
187 |     # Walk from the player along direction of blaster shot.
188 |     height, width = layers['A'].shape
189 |     for step in xrange(1, max(height, width)):
190 |       cur_x = ply_x + dx * step
191 |       cur_y = ply_y + dy * step
192 |       if cur_x < 0 or cur_x >= width or cur_y < 0 or cur_y >= height:
193 |         # Out of bounds, beam went nowhere.
194 |         break
195 |       elif layers['#'][cur_y, cur_x]:
196 |         # Hit normal wall before reaching a special wall.
197 |         break
198 |       elif layers['X'][cur_y, cur_x]:
199 |         # Hit an existing aperture.
200 |         break
201 |       if layers['@'][cur_y, cur_x]:
202 |         # Hit special wall, create an aperture.
203 |         self._apertures = self._apertures[1:] + [(cur_y, cur_x)]
204 |         self.curtain.fill(False)
205 |         for aperture in self.apertures:  # note use of None-filtered set.
206 |           self.curtain[aperture] = True
207 |         break
208 | 
209 |   @property
210 |   def apertures(self):
211 |     """Returns locations of all apertures in the map."""
212 |     return tuple(a for a in self._apertures if a is not None)
213 | 
214 | 
215 | def make_game(level_idx):
216 |   return ascii_art.ascii_art_to_game(
217 |       art=LEVELS[level_idx],
218 |       what_lies_beneath=' ',
219 |       sprites={'A': PlayerSprite},
220 |       drapes={'X': ApertureDrape},
221 |       update_schedule=[['A'], ['X']],  # Move player, then check apertures.
222 |       z_order=['X', 'A'])  # Draw player on top of aperture.
223 | 
224 | 
225 | def main(argv=()):
226 |   game = make_game(int(argv[1]) if len(argv) > 1 else 0)
227 | 
228 |   ui = human_ui.CursesUi(
229 |       keys_to_actions={
230 |           # Basic movement.
231 |           curses.KEY_UP: 0,
232 |           curses.KEY_DOWN: 1,
233 |           curses.KEY_LEFT: 2,
234 |           curses.KEY_RIGHT: 3,
235 |           -1: 4,  # Do nothing.
236 |           # Shoot aperture gun.
237 |           'w': 5,
238 |           'a': 6,
239 |           's': 7,
240 |           'd': 8,
241 |           # Quit game.
242 |           'q': 9,
243 |           'Q': 9,
244 |       },
245 |       delay=50,
246 |       colour_fg=FG_COLOURS,
247 |       colour_bg=BG_COLOURS)
248 | 
249 |   ui.play(game)
250 | 
251 | 
252 | if __name__ == '__main__':
253 |   main(sys.argv)
254 | 


--------------------------------------------------------------------------------
/pycolab/examples/apprehend.py:
--------------------------------------------------------------------------------
  1 | # Copyright 2017 the pycolab Authors
  2 | #
  3 | # Licensed under the Apache License, Version 2.0 (the "License");
  4 | # you may not use this file except in compliance with the License.
  5 | # You may obtain a copy of the License at
  6 | #
  7 | #     https://www.apache.org/licenses/LICENSE-2.0
  8 | #
  9 | # Unless required by applicable law or agreed to in writing, software
 10 | # distributed under the License is distributed on an "AS IS" BASIS,
 11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 12 | # See the License for the specific language governing permissions and
 13 | # limitations under the License.
 14 | 
 15 | """A game faintly reminiscent of Catch.
 16 | 
 17 | Keys: left, right - move the "catcher".
 18 | """
 19 | 
 20 | from __future__ import absolute_import
 21 | from __future__ import division
 22 | from __future__ import print_function
 23 | 
 24 | import curses
 25 | import random
 26 | import sys
 27 | 
 28 | from pycolab import ascii_art
 29 | from pycolab import human_ui
 30 | from pycolab import rendering
 31 | from pycolab.prefab_parts import sprites as prefab_sprites
 32 | 
 33 | 
 34 | GAME_ART = ['   b   ',
 35 |             '       ',
 36 |             '       ',
 37 |             '       ',
 38 |             '       ',
 39 |             '       ',
 40 |             '       ',
 41 |             '       ',
 42 |             '       ',
 43 |             '   P   ']
 44 | 
 45 | 
 46 | # In Catch, both the ball and the player look identical.
 47 | REPAINT_MAPPING = {'b': 'X', 'P': 'X'}
 48 | 
 49 | 
 50 | # These "colours" are only for humans to see in the CursesUi.
 51 | COLOURS = {' ': (0, 0, 0),        # The game board is black.
 52 |            'X': (999, 999, 999)}  # The sprites are white.
 53 | 
 54 | 
 55 | def make_game():
 56 |   """Builds and returns an Apprehend game."""
 57 |   return ascii_art.ascii_art_to_game(
 58 |       GAME_ART, what_lies_beneath=' ',
 59 |       sprites={'P': PlayerSprite, 'b': BallSprite},
 60 |       update_schedule=['b', 'P'])
 61 | 
 62 | 
 63 | class PlayerSprite(prefab_sprites.MazeWalker):
 64 |   """A `Sprite` for our player.
 65 | 
 66 |   This `Sprite` ties actions to going left and right. After every move, it
 67 |   checks whether its position is the same as the ball's position. If so, the
 68 |   game is won.
 69 |   """
 70 | 
 71 |   def __init__(self, corner, position, character):
 72 |     """Simply indicates to the superclass that we can't walk off the board."""
 73 |     super(PlayerSprite, self).__init__(
 74 |         corner, position, character, impassable='', confined_to_board=True)
 75 | 
 76 |   def update(self, actions, board, layers, backdrop, things, the_plot):
 77 |     del layers, backdrop  # Unused.
 78 | 
 79 |     if actions == 0:    # go leftward?
 80 |       self._west(board, the_plot)
 81 |     elif actions == 1:  # go rightward?
 82 |       self._east(board, the_plot)
 83 | 
 84 |     if self.virtual_position == things['b'].virtual_position:
 85 |       the_plot.add_reward(1)
 86 |       the_plot.terminate_episode()
 87 | 
 88 | 
 89 | class BallSprite(prefab_sprites.MazeWalker):
 90 |   """A `Sprite` for the falling ball.
 91 | 
 92 |   This `Sprite` marches linearly toward one of the positions in the bottom row
 93 |   of the game board, selected at random. If it is able to go beyond this
 94 |   position, the game is lost.
 95 |   """
 96 | 
 97 |   def __init__(self, corner, position, character):
 98 |     """Inform superclass that we can go anywhere; initialise falling maths."""
 99 |     super(BallSprite, self).__init__(
100 |         corner, position, character, impassable='')
101 |     # Choose one of the positions in the bottom row of the game board, and
102 |     # compute the per-row X motion (fractional) we'd need to fall there.
103 |     self._dx = random.uniform(-2.499, 2.499) / (corner[0] - 1.0)
104 |     # At each game iteration, we add _dx to this accumulator. If the accumulator
105 |     # exceeds 0.5, we move one position right; if it goes below -0.5, we move
106 |     # one position left. We then bump the accumulator by -1 and 1 respectively.
107 |     self._x_accumulator = 0.0
108 | 
109 |   def update(self, actions, board, layers, backdrop, things, the_plot):
110 |     del actions, layers, backdrop, things   # Unused.
111 | 
112 |     # The ball is always falling downward.
113 |     self._south(board, the_plot)
114 |     self._x_accumulator += self._dx
115 | 
116 |     # Sometimes the ball shifts left or right.
117 |     if self._x_accumulator < -0.5:
118 |       self._west(board, the_plot)
119 |       self._x_accumulator += 1.0
120 |     elif self._x_accumulator > 0.5:
121 |       self._east(board, the_plot)
122 |       self._x_accumulator -= 1.0
123 | 
124 |     # Log the motion information for review in e.g. game consoles.
125 |     the_plot.log('Falling with horizontal velocity {}.\n'
126 |                  '  New location: {}.'.format(self._dx, self.virtual_position))
127 | 
128 |     # If we've left the board altogether, then the game is lost.
129 |     if self.virtual_position[0] >= board.shape[0]:
130 |       the_plot.add_reward(-1)
131 |       the_plot.terminate_episode()
132 | 
133 | 
134 | def main(argv=()):
135 |   del argv  # Unused.
136 | 
137 |   # Build an Apprehend game.
138 |   game = make_game()
139 | 
140 |   # Build an ObservationCharacterRepainter that will make the player and the
141 |   # ball look identical.
142 |   repainter = rendering.ObservationCharacterRepainter(REPAINT_MAPPING)
143 | 
144 |   # Make a CursesUi to play it with.
145 |   ui = human_ui.CursesUi(
146 |       keys_to_actions={curses.KEY_LEFT: 0, curses.KEY_RIGHT: 1, -1: 2},
147 |       repainter=repainter, delay=500,
148 |       colour_fg=COLOURS)
149 | 
150 |   # Let the game begin!
151 |   ui.play(game)
152 | 
153 | 
154 | if __name__ == '__main__':
155 |   main(sys.argv)
156 | 


--------------------------------------------------------------------------------
/pycolab/examples/classics/__init__.py:
--------------------------------------------------------------------------------
 1 | # Copyright 2017 the pycolab Authors
 2 | #
 3 | # Licensed under the Apache License, Version 2.0 (the "License");
 4 | # you may not use this file except in compliance with the License.
 5 | # You may obtain a copy of the License at
 6 | #
 7 | #     https://www.apache.org/licenses/LICENSE-2.0
 8 | #
 9 | # Unless required by applicable law or agreed to in writing, software
10 | # distributed under the License is distributed on an "AS IS" BASIS,
11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12 | # See the License for the specific language governing permissions and
13 | # limitations under the License.
14 | 


--------------------------------------------------------------------------------
/pycolab/examples/classics/chain_walk.py:
--------------------------------------------------------------------------------
 1 | # Copyright 2017 the pycolab Authors
 2 | #
 3 | # Licensed under the Apache License, Version 2.0 (the "License");
 4 | # you may not use this file except in compliance with the License.
 5 | # You may obtain a copy of the License at
 6 | #
 7 | #     https://www.apache.org/licenses/LICENSE-2.0
 8 | #
 9 | # Unless required by applicable law or agreed to in writing, software
10 | # distributed under the License is distributed on an "AS IS" BASIS,
11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12 | # See the License for the specific language governing permissions and
13 | # limitations under the License.
14 | 
15 | """An example implementation of the classic chain-walk problem."""
16 | 
17 | from __future__ import absolute_import
18 | from __future__ import division
19 | from __future__ import print_function
20 | 
21 | import curses
22 | import sys
23 | 
24 | from pycolab import ascii_art
25 | from pycolab import human_ui
26 | from pycolab.prefab_parts import sprites as prefab_sprites
27 | 
28 | 
29 | GAME_ART = ['..P...................']
30 | 
31 | 
32 | def make_game():
33 |   """Builds and returns a chain-walk game."""
34 |   return ascii_art.ascii_art_to_game(
35 |       GAME_ART, what_lies_beneath='.',
36 |       sprites={'P': PlayerSprite})
37 | 
38 | 
39 | class PlayerSprite(prefab_sprites.MazeWalker):
40 |   """A `Sprite` for our player.
41 | 
42 |   This sprite ties actions to going left and right. If it reaches the leftmost
43 |   extreme of the board, it receives a small reward; if it reaches the rightmost
44 |   extreme it receives a large reward. The game terminates in either case.
45 |   """
46 | 
47 |   def __init__(self, corner, position, character):
48 |     """Inform superclass that we can go anywhere."""
49 |     super(PlayerSprite, self).__init__(
50 |         corner, position, character, impassable='')
51 | 
52 |   def update(self, actions, board, layers, backdrop, things, the_plot):
53 |     del layers, backdrop, things   # Unused.
54 | 
55 |     # Apply motion commands.
56 |     if actions == 0:    # walk leftward?
57 |       self._west(board, the_plot)
58 |     elif actions == 1:  # walk rightward?
59 |       self._east(board, the_plot)
60 | 
61 |     # See if the game is over.
62 |     if self.position[1] == 0:
63 |       the_plot.add_reward(1.0)
64 |       the_plot.terminate_episode()
65 |     elif self.position[1] == (self.corner[1] - 1):
66 |       the_plot.add_reward(100.0)
67 |       the_plot.terminate_episode()
68 | 
69 | 
70 | def main(argv=()):
71 |   del argv  # Unused.
72 | 
73 |   # Build a chain-walk game.
74 |   game = make_game()
75 | 
76 |   # Make a CursesUi to play it with.
77 |   ui = human_ui.CursesUi(
78 |       keys_to_actions={curses.KEY_LEFT: 0, curses.KEY_RIGHT: 1, -1: 2},
79 |       delay=200)
80 | 
81 |   # Let the game begin!
82 |   ui.play(game)
83 | 
84 | 
85 | if __name__ == '__main__':
86 |   main(sys.argv)
87 | 


--------------------------------------------------------------------------------
/pycolab/examples/classics/cliff_walk.py:
--------------------------------------------------------------------------------
  1 | # Copyright 2017 the pycolab Authors
  2 | #
  3 | # Licensed under the Apache License, Version 2.0 (the "License");
  4 | # you may not use this file except in compliance with the License.
  5 | # You may obtain a copy of the License at
  6 | #
  7 | #     https://www.apache.org/licenses/LICENSE-2.0
  8 | #
  9 | # Unless required by applicable law or agreed to in writing, software
 10 | # distributed under the License is distributed on an "AS IS" BASIS,
 11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 12 | # See the License for the specific language governing permissions and
 13 | # limitations under the License.
 14 | 
 15 | """An example implementation of the classic cliff-walk problem."""
 16 | 
 17 | from __future__ import absolute_import
 18 | from __future__ import division
 19 | from __future__ import print_function
 20 | 
 21 | import curses
 22 | import sys
 23 | 
 24 | from pycolab import ascii_art
 25 | from pycolab import human_ui
 26 | from pycolab.prefab_parts import sprites as prefab_sprites
 27 | 
 28 | 
 29 | GAME_ART = ['............',
 30 |             '............',
 31 |             '............',
 32 |             'P...........']
 33 | 
 34 | 
 35 | def make_game():
 36 |   """Builds and returns a cliff-walk game."""
 37 |   return ascii_art.ascii_art_to_game(
 38 |       GAME_ART, what_lies_beneath='.',
 39 |       sprites={'P': PlayerSprite})
 40 | 
 41 | 
 42 | class PlayerSprite(prefab_sprites.MazeWalker):
 43 |   """A `Sprite` for our player.
 44 | 
 45 |   This `Sprite` ties actions to going in the four cardinal directions. If it
 46 |   walks into all but the first and last columns of the bottom row, it receives a
 47 |   reward of -100 and the episode terminates. Moving to any other cell yields a
 48 |   reward of -1; moving into the bottom right cell terminates the episode.
 49 |   """
 50 | 
 51 |   def __init__(self, corner, position, character):
 52 |     """Inform superclass that we can go anywhere, but not off the board."""
 53 |     super(PlayerSprite, self).__init__(
 54 |         corner, position, character, impassable='', confined_to_board=True)
 55 | 
 56 |   def update(self, actions, board, layers, backdrop, things, the_plot):
 57 |     del layers, backdrop, things   # Unused.
 58 | 
 59 |     # Apply motion commands.
 60 |     if actions == 0:    # walk upward?
 61 |       self._north(board, the_plot)
 62 |     elif actions == 1:  # walk downward?
 63 |       self._south(board, the_plot)
 64 |     elif actions == 2:  # walk leftward?
 65 |       self._west(board, the_plot)
 66 |     elif actions == 3:  # walk rightward?
 67 |       self._east(board, the_plot)
 68 |     else:
 69 |       # All other actions are ignored. Although humans using the CursesUi can
 70 |       # issue action 4 (no-op), agents should only have access to actions 0-3.
 71 |       # Otherwise staying put is going to look like a terrific strategy.
 72 |       return
 73 | 
 74 |     # See what reward we get for moving where we moved.
 75 |     if (self.position[0] == (self.corner[0] - 1) and
 76 |         0 < self.position[1] < (self.corner[1] - 2)):
 77 |       the_plot.add_reward(-100.0)  # Fell off the cliff.
 78 |     else:
 79 |       the_plot.add_reward(-1.0)
 80 | 
 81 |     # See if the game is over.
 82 |     if self.position[0] == (self.corner[0] - 1) and 0 < self.position[1]:
 83 |       the_plot.terminate_episode()
 84 | 
 85 | 
 86 | def main(argv=()):
 87 |   del argv  # Unused.
 88 | 
 89 |   # Build a cliff-walk game.
 90 |   game = make_game()
 91 | 
 92 |   # Make a CursesUi to play it with.
 93 |   ui = human_ui.CursesUi(
 94 |       keys_to_actions={curses.KEY_UP: 0, curses.KEY_DOWN: 1,
 95 |                        curses.KEY_LEFT: 2, curses.KEY_RIGHT: 3,
 96 |                        -1: 4},
 97 |       delay=200)
 98 | 
 99 |   # Let the game begin!
100 |   ui.play(game)
101 | 
102 | 
103 | if __name__ == '__main__':
104 |   main(sys.argv)
105 | 


--------------------------------------------------------------------------------
/pycolab/examples/classics/four_rooms.py:
--------------------------------------------------------------------------------
  1 | # Copyright 2017 the pycolab Authors
  2 | #
  3 | # Licensed under the Apache License, Version 2.0 (the "License");
  4 | # you may not use this file except in compliance with the License.
  5 | # You may obtain a copy of the License at
  6 | #
  7 | #     https://www.apache.org/licenses/LICENSE-2.0
  8 | #
  9 | # Unless required by applicable law or agreed to in writing, software
 10 | # distributed under the License is distributed on an "AS IS" BASIS,
 11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 12 | # See the License for the specific language governing permissions and
 13 | # limitations under the License.
 14 | 
 15 | """An example implementation of the classic four-rooms scenario."""
 16 | 
 17 | from __future__ import absolute_import
 18 | from __future__ import division
 19 | from __future__ import print_function
 20 | 
 21 | import curses
 22 | import sys
 23 | 
 24 | from pycolab import ascii_art
 25 | from pycolab import human_ui
 26 | from pycolab.prefab_parts import sprites as prefab_sprites
 27 | 
 28 | 
 29 | GAME_ART = ['#############',
 30 |             '#     #     #',
 31 |             '#     #     #',
 32 |             '#     #     #',
 33 |             '#           #',
 34 |             '#     #     #',
 35 |             '#### ###### #',
 36 |             '#     #     #',
 37 |             '#     #     #',
 38 |             '#           #',
 39 |             '#     #     #',
 40 |             '# P   #     #',
 41 |             '#############']
 42 | 
 43 | 
 44 | def make_game():
 45 |   """Builds and returns a four-rooms game."""
 46 |   return ascii_art.ascii_art_to_game(
 47 |       GAME_ART, what_lies_beneath=' ',
 48 |       sprites={'P': PlayerSprite})
 49 | 
 50 | 
 51 | class PlayerSprite(prefab_sprites.MazeWalker):
 52 |   """A `Sprite` for our player.
 53 | 
 54 |   This `Sprite` ties actions to going in the four cardinal directions. If we
 55 |   reach a magical location (in this example, (4, 3)), the agent receives a
 56 |   reward of 1 and the episode terminates.
 57 |   """
 58 | 
 59 |   def __init__(self, corner, position, character):
 60 |     """Inform superclass that we can't walk through walls."""
 61 |     super(PlayerSprite, self).__init__(
 62 |         corner, position, character, impassable='#')
 63 | 
 64 |   def update(self, actions, board, layers, backdrop, things, the_plot):
 65 |     del layers, backdrop, things   # Unused.
 66 | 
 67 |     # Apply motion commands.
 68 |     if actions == 0:    # walk upward?
 69 |       self._north(board, the_plot)
 70 |     elif actions == 1:  # walk downward?
 71 |       self._south(board, the_plot)
 72 |     elif actions == 2:  # walk leftward?
 73 |       self._west(board, the_plot)
 74 |     elif actions == 3:  # walk rightward?
 75 |       self._east(board, the_plot)
 76 | 
 77 |     # See if we've found the mystery spot.
 78 |     if self.position == (4, 3):
 79 |       the_plot.add_reward(1.0)
 80 |       the_plot.terminate_episode()
 81 | 
 82 | 
 83 | def main(argv=()):
 84 |   del argv  # Unused.
 85 | 
 86 |   # Build a four-rooms game.
 87 |   game = make_game()
 88 | 
 89 |   # Make a CursesUi to play it with.
 90 |   ui = human_ui.CursesUi(
 91 |       keys_to_actions={curses.KEY_UP: 0, curses.KEY_DOWN: 1,
 92 |                        curses.KEY_LEFT: 2, curses.KEY_RIGHT: 3,
 93 |                        -1: 4},
 94 |       delay=200)
 95 | 
 96 |   # Let the game begin!
 97 |   ui.play(game)
 98 | 
 99 | 
100 | if __name__ == '__main__':
101 |   main(sys.argv)
102 | 


--------------------------------------------------------------------------------
/pycolab/examples/extraterrestrial_marauders.py:
--------------------------------------------------------------------------------
  1 | # Copyright 2017 the pycolab Authors
  2 | #
  3 | # Licensed under the Apache License, Version 2.0 (the "License");
  4 | # you may not use this file except in compliance with the License.
  5 | # You may obtain a copy of the License at
  6 | #
  7 | #     https://www.apache.org/licenses/LICENSE-2.0
  8 | #
  9 | # Unless required by applicable law or agreed to in writing, software
 10 | # distributed under the License is distributed on an "AS IS" BASIS,
 11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 12 | # See the License for the specific language governing permissions and
 13 | # limitations under the License.
 14 | 
 15 | """Defeat marauders from somewhere exterior to this planet.
 16 | 
 17 | Keys: left, right - move. space - fire. q - quit.
 18 | """
 19 | 
 20 | from __future__ import absolute_import
 21 | from __future__ import division
 22 | from __future__ import print_function
 23 | 
 24 | import curses
 25 | 
 26 | import numpy as np
 27 | 
 28 | import sys
 29 | 
 30 | from pycolab import ascii_art
 31 | from pycolab import human_ui
 32 | from pycolab import rendering
 33 | from pycolab import things as plab_things
 34 | from pycolab.prefab_parts import sprites as prefab_sprites
 35 | 
 36 | 
 37 | # Not shown in this ASCII art diagram are the Sprites we use for laser blasts,
 38 | # which control the characters listed in UPWARD_BOLT_CHARS and
 39 | # DOWNWARD_BOLT_CHARS below.
 40 | GAME_ART = ['    X   X   X   X   X   X   X   X      ',  # Row 0
 41 |             '     X   X   X   X   X   X   X   X     ',
 42 |             '    X   X   X   X   X   X   X   X      ',
 43 |             '     X   X   X   X   X   X   X   X     ',
 44 |             '    X   X   X   X   X   X   X   X      ',
 45 |             '                                       ',  # Row 5
 46 |             '                                       ',
 47 |             '                                       ',
 48 |             '                                       ',
 49 |             '                                       ',
 50 |             '                                       ',  # Row 10. If a Marauder
 51 |             '    BBBB     BBBB     BBBB     BBBB    ',  # makes it to row 10,
 52 |             '    BBBB     BBBB     BBBB     BBBB    ',  # the game is over.
 53 |             '    BBBB     BBBB     BBBB     BBBB    ',
 54 |             '                                       ',
 55 |             '  P                                    ']
 56 | 
 57 | 
 58 | # Characters listed in UPWARD_BOLT_CHARS are used for Sprites that represent
 59 | # laser bolts that the player shoots toward Marauders. Add more characters if
 60 | # you want to be able to have more than two of these bolts in the "air" at once.
 61 | UPWARD_BOLT_CHARS = 'abcd'
 62 | # Characters listed in DOWNWARD_BOLT_CHARS are used for Sprites that represent
 63 | # laser bolts that Marauders shoot toward the player. Add more charcters if you
 64 | # want more shooting from the Marauders.
 65 | DOWNWARD_BOLT_CHARS = 'yz'
 66 | # Shorthand for various points in the program:
 67 | _ALL_BOLT_CHARS = UPWARD_BOLT_CHARS + DOWNWARD_BOLT_CHARS
 68 | 
 69 | 
 70 | # To make life a bit easier for the player (and avoid the need for frame
 71 | # stacking), we use different characters to indicate the directions that the
 72 | # bolts go. If you'd like to make this game harder, you might try mapping both
 73 | # kinds of bolts to the same character.
 74 | LASER_REPAINT_MAPPING = dict(
 75 |     [(b, '^') for b in UPWARD_BOLT_CHARS] +
 76 |     [(b, '|') for b in DOWNWARD_BOLT_CHARS])
 77 | 
 78 | 
 79 | # These colours are only for humans to see in the CursesUi.
 80 | COLOURS_FG = {' ': (0, 0, 0),        # Space, inky blackness of.
 81 |               'X': (999, 999, 999),  # The Marauders.
 82 |               'B': (400, 50, 30),    # The bunkers.
 83 |               'P': (0, 999, 0),      # The player.
 84 |               '^': (0, 999, 999),    # Bolts from player to aliens.
 85 |               '|': (0, 999, 999)}    # Bolts from aliens to player.
 86 | 
 87 | COLOURS_BG = {'^': (0, 0, 0),        # Bolts from player to aliens.
 88 |               '|': (0, 0, 0)}        # Bolts from aliens to player.
 89 | 
 90 | 
 91 | def make_game():
 92 |   """Builds and returns an Extraterrestrial Marauders game."""
 93 |   return ascii_art.ascii_art_to_game(
 94 |       GAME_ART, what_lies_beneath=' ',
 95 |       sprites=dict(
 96 |           [('P', PlayerSprite)] +
 97 |           [(c, UpwardLaserBoltSprite) for c in UPWARD_BOLT_CHARS] +
 98 |           [(c, DownwardLaserBoltSprite) for c in DOWNWARD_BOLT_CHARS]),
 99 |       drapes=dict(X=MarauderDrape,
100 |                   B=BunkerDrape),
101 |       update_schedule=['P', 'B', 'X'] + list(_ALL_BOLT_CHARS))
102 | 
103 | 
104 | class BunkerDrape(plab_things.Drape):
105 |   """A `Drape` for the bunkers at the bottom of the screen.
106 | 
107 |   Bunkers are gradually eroded by laser bolts, for which the user loses one
108 |   point. Other than that, they don't really do much. If a laser bolt hits a
109 |   bunker, this Drape leaves a note about it in the Plot---the bolt's Sprite
110 |   checks this and removes itself from the board if it's present.
111 |   """
112 | 
113 |   def update(self, actions, board, layers, backdrop, things, the_plot):
114 |     # Where are the laser bolts? Bolts from players or marauders do damage.
115 |     bolts = np.logical_or.reduce([layers[c] for c in _ALL_BOLT_CHARS], axis=0)
116 |     hits = bolts & self.curtain                       # Any hits to a bunker?
117 |     np.logical_xor(self.curtain, hits, self.curtain)  # If so, erode the bunker...
118 |     the_plot.add_reward(-np.sum(hits))                # ...and impose a penalty.
119 |     # Save the identities of bunker-striking bolts in the Plot.
120 |     the_plot['bunker_hitters'] = [chr(c) for c in board[hits]]
121 | 
122 | 
123 | class MarauderDrape(plab_things.Drape):
124 |   """A `Drape` for the marauders descending downward toward the player.
125 | 
126 |   The Marauders all move in lockstep, which makes them an ideal application of
127 |   a Drape. Bits of the Drape get eroded by laser bolts from the player; each
128 |   hit earns ten points. If the Drape goes completely empty, or if any Marauder
129 |   makes it down to row 10, the game terminates.
130 | 
131 |   As with `BunkerDrape`, if a laser bolt hits a Marauder, this Drape leaves a
132 |   note about it in the Plot; the bolt's Sprite checks this and removes itself
133 |   from the board if present.
134 |   """
135 | 
136 |   def __init__(self, curtain, character):
137 |     # The constructor just sets the Marauder's initial horizontal direction.
138 |     super(MarauderDrape, self).__init__(curtain, character)
139 |     self._dx = -1
140 | 
141 |   def update(self, actions, board, layers, backdrop, things, the_plot):
142 |     # Where are the laser bolts? Only bolts from the player kill a Marauder.
143 |     bolts = np.logical_or.reduce([layers[c] for c in UPWARD_BOLT_CHARS], axis=0)
144 |     hits = bolts & self.curtain                       # Any hits to Marauders?
145 |     np.logical_xor(self.curtain, hits, self.curtain)  # If so, zap the marauder...
146 |     the_plot.add_reward(np.sum(hits)*10)              # ...and supply a reward.
147 |     # Save the identities of marauder-striking bolts in the Plot.
148 |     the_plot['marauder_hitters'] = [chr(c) for c in board[hits]]
149 | 
150 |     # If no Marauders are left, or if any are sitting on row 10, end the game.
151 |     if (not self.curtain.any()) or self.curtain[10, :].any():
152 |       return the_plot.terminate_episode()  # i.e. return None.
153 | 
154 |     # We move faster if there are fewer Marauders. The odd divisor causes speed
155 |     # jumps to align on the high sides of multiples of 8; so, speed increases as
156 |     # the number of Marauders decreases to 32 (or 24 etc.), not 31 (or 23 etc.).
157 |     if the_plot.frame % max(1, np.sum(self.curtain)//8.0000001): return
158 |     # If any Marauder reaches either side of the screen, reverse horizontal
159 |     # motion and advance vertically one row.
160 |     if np.any(self.curtain[:, 0] | self.curtain[:, -1]):
161 |       self._dx = -self._dx
162 |       self.curtain[:] = np.roll(self.curtain, shift=1, axis=0)
163 |     self.curtain[:] = np.roll(self.curtain, shift=self._dx, axis=1)
164 | 
165 | 
166 | class PlayerSprite(prefab_sprites.MazeWalker):
167 |   """A `Sprite` for our player.
168 | 
169 |   This `Sprite` simply ties actions to going left and right. In interactive
170 |   settings, the user can also quit.
171 |   """
172 | 
173 |   def __init__(self, corner, position, character):
174 |     """Simply indicates to the superclass that we can't walk off the board."""
175 |     super(PlayerSprite, self).__init__(
176 |         corner, position, character, impassable='', confined_to_board=True)
177 | 
178 |   def update(self, actions, board, layers, backdrop, things, the_plot):
179 |     del layers, backdrop, things  # Unused.
180 | 
181 |     if actions == 0:    # go leftward?
182 |       self._west(board, the_plot)
183 |     elif actions == 1:  # go rightward?
184 |       self._east(board, the_plot)
185 |     elif actions == 4:  # quit?
186 |       the_plot.terminate_episode()
187 | 
188 | 
189 | class UpwardLaserBoltSprite(prefab_sprites.MazeWalker):
190 |   """Laser bolts shot from the player toward Marauders."""
191 | 
192 |   def __init__(self, corner, position, character):
193 |     """Starts the Sprite in a hidden position off of the board."""
194 |     super(UpwardLaserBoltSprite, self).__init__(
195 |         corner, position, character, impassable='')
196 |     self._teleport((-1, -1))
197 | 
198 |   def update(self, actions, board, layers, backdrop, things, the_plot):
199 |     if self.visible:
200 |       self._fly(board, layers, things, the_plot)
201 |     elif actions == 2:
202 |       self._fire(layers, things, the_plot)
203 | 
204 |   def _fly(self, board, layers, things, the_plot):
205 |     """Handles the behaviour of visible bolts flying toward Marauders."""
206 |     # Disappear if we've hit a Marauder or a bunker.
207 |     if (self.character in the_plot['bunker_hitters'] or
208 |         self.character in the_plot['marauder_hitters']):
209 |       return self._teleport((-1, -1))
210 |     # Otherwise, northward!
211 |     self._north(board, the_plot)
212 | 
213 |   def _fire(self, layers, things, the_plot):
214 |     """Launches a new bolt from the player."""
215 |     # We don't fire if the player fired another bolt just now.
216 |     if the_plot.get('last_player_shot') == the_plot.frame: return
217 |     the_plot['last_player_shot'] = the_plot.frame
218 |     # We start just above the player.
219 |     row, col = things['P'].position
220 |     self._teleport((row-1, col))
221 | 
222 | 
223 | class DownwardLaserBoltSprite(prefab_sprites.MazeWalker):
224 |   """Laser bolts shot from Marauders toward the player."""
225 | 
226 |   def __init__(self, corner, position, character):
227 |     """Starts the Sprite in a hidden position off of the board."""
228 |     super(DownwardLaserBoltSprite, self).__init__(
229 |         corner, position, character, impassable='')
230 |     self._teleport((-1, -1))
231 | 
232 |   def update(self, actions, board, layers, backdrop, things, the_plot):
233 |     if self.visible:
234 |       self._fly(board, layers, things, the_plot)
235 |     else:
236 |       self._fire(layers, the_plot)
237 | 
238 |   def _fly(self, board, layers, things, the_plot):
239 |     """Handles the behaviour of visible bolts flying toward the player."""
240 |     # Disappear if we've hit a bunker.
241 |     if self.character in the_plot['bunker_hitters']:
242 |       return self._teleport((-1, -1))
243 |     # End the game if we've hit the player.
244 |     if self.position == things['P'].position: the_plot.terminate_episode()
245 |     self._south(board, the_plot)
246 | 
247 |   def _fire(self, layers, the_plot):
248 |     """Launches a new bolt from a random Marauder."""
249 |     # We don't fire if another Marauder fired a bolt just now.
250 |     if the_plot.get('last_marauder_shot') == the_plot.frame: return
251 |     the_plot['last_marauder_shot'] = the_plot.frame
252 |     # Which Marauder should fire the laser bolt?
253 |     col = np.random.choice(np.nonzero(layers['X'].sum(axis=0))[0])
254 |     row = np.nonzero(layers['X'][:, col])[0][-1] + 1
255 |     # Move ourselves just below that Marauder.
256 |     self._teleport((row, col))
257 | 
258 | 
259 | def main(argv=()):
260 |   del argv  # Unused.
261 | 
262 |   # Build an Extraterrestrial Marauders game.
263 |   game = make_game()
264 | 
265 |   # Build an ObservationCharacterRepainter that will make laser bolts of the
266 |   # same type all look identical.
267 |   repainter = rendering.ObservationCharacterRepainter(LASER_REPAINT_MAPPING)
268 | 
269 |   # Make a CursesUi to play it with.
270 |   ui = human_ui.CursesUi(
271 |       keys_to_actions={curses.KEY_LEFT: 0, curses.KEY_RIGHT: 1,
272 |                        ' ': 2,   # shoot
273 |                        -1: 3,    # no-op
274 |                        'q': 4},  # quit
275 |       repainter=repainter, delay=300,
276 |       colour_fg=COLOURS_FG, colour_bg=COLOURS_BG)
277 | 
278 |   # Let the game begin!
279 |   ui.play(game)
280 | 
281 | 
282 | if __name__ == '__main__':
283 |   main(sys.argv)
284 | 


--------------------------------------------------------------------------------
/pycolab/examples/fluvial_natation.py:
--------------------------------------------------------------------------------
  1 | # Copyright 2017 the pycolab Authors
  2 | #
  3 | # Licensed under the Apache License, Version 2.0 (the "License");
  4 | # you may not use this file except in compliance with the License.
  5 | # You may obtain a copy of the License at
  6 | #
  7 | #     https://www.apache.org/licenses/LICENSE-2.0
  8 | #
  9 | # Unless required by applicable law or agreed to in writing, software
 10 | # distributed under the License is distributed on an "AS IS" BASIS,
 11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 12 | # See the License for the specific language governing permissions and
 13 | # limitations under the License.
 14 | 
 15 | """Swimming a river, walking a chain, whatever you like to call it.
 16 | 
 17 | Keys: left, right - move the "swimmer". Fight current to get to the right.
 18 | """
 19 | 
 20 | from __future__ import absolute_import
 21 | from __future__ import division
 22 | from __future__ import print_function
 23 | 
 24 | import curses
 25 | 
 26 | import numpy as np
 27 | 
 28 | import sys
 29 | 
 30 | from pycolab import ascii_art
 31 | from pycolab import human_ui
 32 | from pycolab import things as plab_things
 33 | from pycolab.prefab_parts import sprites as prefab_sprites
 34 | 
 35 | 
 36 | GAME_ART = ['===================================================',
 37 |             '     .      :   ,     `     ~          ,    .    ` ',
 38 |             '   ,    ~   P     :     .  `    ,    ,    ~    `   ',
 39 |             '     `   .     ~~   ,     .   :     .   `     `   ~',
 40 |             '===================================================']
 41 | 
 42 | 
 43 | COLOURS_FG = {'P': (0, 999, 0),       # The swimmer
 44 |               '=': (576, 255, 0),     # The river bank
 45 |               ' ': (0, 505, 999),     # Empty water
 46 |               '.': (999, 999, 999),   # Waves on the water
 47 |               ',': (999, 999, 999),
 48 |               '`': (999, 999, 999),
 49 |               ':': (999, 999, 999),
 50 |               '~': (999, 999, 999)}
 51 | 
 52 | 
 53 | COLOURS_BG = {'.': (0, 505, 999),     # Waves on the water have the "water"
 54 |               ',': (0, 505, 999),     # colour as their background colour.
 55 |               '`': (0, 505, 999),
 56 |               ':': (0, 505, 999),
 57 |               '~': (0, 505, 999)}
 58 | 
 59 | 
 60 | def make_game():
 61 |   """Builds and returns a Fluvial Natation game."""
 62 |   return ascii_art.ascii_art_to_game(
 63 |       GAME_ART, what_lies_beneath=' ',
 64 |       sprites={'P': PlayerSprite},
 65 |       backdrop=RiverBackdrop)
 66 | 
 67 | 
 68 | class PlayerSprite(prefab_sprites.MazeWalker):
 69 |   """A `Sprite` for our player.
 70 | 
 71 |   This `Sprite` ties actions to going left and right. On even-numbered game
 72 |   iterations, it moves left in addition to whatever the user action specifies,
 73 |   akin to the current sweeping a swimmer downriver. If the player goes beyond
 74 |   the right edge of the game board, the game is won; if it goes beyond the left
 75 |   edge, the game is lost.
 76 |   """
 77 | 
 78 |   def __init__(self, corner, position, character):
 79 |     """Inform superclass that we can go anywhere."""
 80 |     super(PlayerSprite, self).__init__(
 81 |         corner, position, character, impassable='')
 82 | 
 83 |   def update(self, actions, board, layers, backdrop, things, the_plot):
 84 |     del layers, backdrop, things   # Unused.
 85 | 
 86 |     # Move one square left on even game iterations.
 87 |     if the_plot.frame % 2 == 0:
 88 |       self._west(board, the_plot)
 89 | 
 90 |     # Apply swimming commands.
 91 |     if actions == 0:    # swim leftward?
 92 |       self._west(board, the_plot)
 93 |     elif actions == 1:  # swim rightward?
 94 |       self._east(board, the_plot)
 95 | 
 96 |     # See if the game is won or lost.
 97 |     if self.virtual_position[1] < 0:
 98 |       the_plot.add_reward(-1)
 99 |       the_plot.terminate_episode()
100 |     elif self.virtual_position[1] >= board.shape[1]:
101 |       the_plot.add_reward(1)
102 |       the_plot.terminate_episode()
103 | 
104 | 
105 | class RiverBackdrop(plab_things.Backdrop):
106 |   """A `Backdrop` for the river.
107 | 
108 |   This `Backdrop` rotates the river part of the backdrop leftward on every even
109 |   game iteration, making the river appear to be flowing.
110 |   """
111 | 
112 |   def update(self, actions, board, layers, things, the_plot):
113 |     del actions, board, layers, things   # Unused.
114 | 
115 |     if the_plot.frame % 2 == 0:
116 |       self.curtain[1:4, :] = np.roll(self.curtain[1:4, :], shift=-1, axis=1)
117 | 
118 | 
119 | def main(argv=()):
120 |   del argv  # Unused.
121 | 
122 |   # Build a Fluvial Natation game.
123 |   game = make_game()
124 | 
125 |   # Make a CursesUi to play it with.
126 |   ui = human_ui.CursesUi(
127 |       keys_to_actions={curses.KEY_LEFT: 0, curses.KEY_RIGHT: 1, -1: 2},
128 |       delay=200, colour_fg=COLOURS_FG, colour_bg=COLOURS_BG)
129 | 
130 |   # Let the game begin!
131 |   ui.play(game)
132 | 
133 | 
134 | if __name__ == '__main__':
135 |   main(sys.argv)
136 | 


--------------------------------------------------------------------------------
/pycolab/examples/hello_world.py:
--------------------------------------------------------------------------------
  1 | # Copyright 2017 the pycolab Authors
  2 | #
  3 | # Licensed under the Apache License, Version 2.0 (the "License");
  4 | # you may not use this file except in compliance with the License.
  5 | # You may obtain a copy of the License at
  6 | #
  7 | #     https://www.apache.org/licenses/LICENSE-2.0
  8 | #
  9 | # Unless required by applicable law or agreed to in writing, software
 10 | # distributed under the License is distributed on an "AS IS" BASIS,
 11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 12 | # See the License for the specific language governing permissions and
 13 | # limitations under the License.
 14 | 
 15 | """A "game" whereby you move text around the board.
 16 | 
 17 | Keys: up, down, left, right - move. q - quit.
 18 | """
 19 | 
 20 | from __future__ import absolute_import
 21 | from __future__ import division
 22 | from __future__ import print_function
 23 | 
 24 | import curses
 25 | 
 26 | import numpy as np
 27 | 
 28 | import sys
 29 | 
 30 | from pycolab import ascii_art
 31 | from pycolab import human_ui
 32 | from pycolab import things
 33 | 
 34 | 
 35 | HELLO_ART = ['                                    ',
 36 |              '  #   #  ### #    #     ###         ',
 37 |              '  #   # #    #    #    #   #        ',
 38 |              '  ##### #### #    #    #   #        ',
 39 |              '  #   # #    #    #    #   #        ',
 40 |              '  #   #  ###  ###  ###  ###         ',
 41 |              '                                    ',
 42 |              '     @   @  @@@   @@@  @    @@@@  1 ',
 43 |              '     @   @ @   @ @   @ @    @   @ 2 ',
 44 |              '     @ @ @ @   @ @@@@  @    @   @ 3 ',
 45 |              '     @ @ @ @   @ @   @ @    @   @   ',
 46 |              '      @@@   @@@  @   @  @@@ @@@@  4 ',
 47 |              '                                    ']
 48 | 
 49 | 
 50 | HELLO_COLOURS = {' ': (123, 123, 123),  # Only used in this program by
 51 |                  '#': (595, 791, 928),  # the CursesUi.
 52 |                  '@': (54, 501, 772),
 53 |                  '1': (999, 222, 222),
 54 |                  '2': (222, 999, 222),
 55 |                  '3': (999, 999, 111),
 56 |                  '4': (222, 222, 999)}
 57 | 
 58 | 
 59 | def make_game():
 60 |   """Builds and returns a Hello World game."""
 61 |   return ascii_art.ascii_art_to_game(
 62 |       HELLO_ART,
 63 |       what_lies_beneath=' ',
 64 |       sprites={'1': ascii_art.Partial(SlidingSprite, 0),
 65 |                '2': ascii_art.Partial(SlidingSprite, 1),
 66 |                '3': ascii_art.Partial(SlidingSprite, 2),
 67 |                '4': ascii_art.Partial(SlidingSprite, 3)},
 68 |       drapes={'@': RollingDrape},
 69 |       z_order='12@34')
 70 | 
 71 | 
 72 | class RollingDrape(things.Drape):
 73 |   """A Drape that just `np.roll`s the mask around either axis."""
 74 | 
 75 |   # There are four rolls to choose from: two shifts of size 1 along both axes.
 76 |   _ROLL_AXES = [0, 0, 1, 1]
 77 |   _ROLL_SHIFTS = [-1, 1, -1, 1]
 78 | 
 79 |   def update(self, actions, board, layers, backdrop, all_things, the_plot):
 80 |     del board, layers, backdrop, all_things  # unused
 81 | 
 82 |     if actions is None: return  # No work needed to make the first observation.
 83 |     if actions == 4: the_plot.terminate_episode()  # Action 4 means "quit".
 84 | 
 85 |     # If the player has chosen a motion action, use that action to index into
 86 |     # the set of four rolls.
 87 |     if actions < 4:
 88 |       rolled = np.roll(self.curtain,  # Makes a copy, alas.
 89 |                        self._ROLL_SHIFTS[actions], self._ROLL_AXES[actions])
 90 |       np.copyto(self.curtain, rolled)
 91 |       the_plot.add_reward(1)  # Give ourselves a point for moving.
 92 | 
 93 | 
 94 | class SlidingSprite(things.Sprite):
 95 |   """A Sprite that moves in diagonal directions."""
 96 | 
 97 |   # We have four mappings from actions to motions to choose from. The mappings
 98 |   # are arranged so that given any index i, then across all sets, the motion
 99 |   # that undoes motion i always has the same index j.
100 |   _DX = ([-1, 1, -1, 1], [-1, 1, -1, 1], [1, -1, 1, -1], [1, -1, 1, -1])
101 |   _DY = ([-1, 1, 1, -1], [1, -1, -1, 1], [1, -1, -1, 1], [-1, 1, 1, -1])
102 | 
103 |   def __init__(self, corner, position, character, direction_set):
104 |     """Build a SlidingSprite.
105 | 
106 |     Args:
107 |       corner: required argument for Sprite.
108 |       position: required argument for Sprite.
109 |       character: required argument for Sprite.
110 |       direction_set: an integer in `[0,3]` that selects from any of four
111 |           mappings from actions to (diagonal) motions.
112 |     """
113 |     super(SlidingSprite, self).__init__(corner, position, character)
114 |     self._dx = self._DX[direction_set]
115 |     self._dy = self._DY[direction_set]
116 | 
117 |   def update(self, actions, board, layers, backdrop, all_things, the_plot):
118 |     del board, layers, backdrop, all_things, the_plot  # unused
119 |     # Actions 0-3 are motion actions; the others we ignore.
120 |     if actions is None or actions > 3: return
121 |     new_col = (self._position.col + self._dx[actions]) % self.corner.col
122 |     new_row = (self._position.row + self._dy[actions]) % self.corner.row
123 |     self._position = self.Position(new_row, new_col)
124 | 
125 | 
126 | def main(argv=()):
127 |   del argv  # Unused.
128 | 
129 |   # Build a Hello World game.
130 |   game = make_game()
131 | 
132 |   # Log a message in its Plot object.
133 |   game.the_plot.log('Hello, world!')
134 | 
135 |   # Make a CursesUi to play it with.
136 |   ui = human_ui.CursesUi(
137 |       keys_to_actions={curses.KEY_UP: 0, curses.KEY_DOWN: 1, curses.KEY_LEFT: 2,
138 |                        curses.KEY_RIGHT: 3, 'q': 4, 'Q': 4, -1: 5},
139 |       delay=50, colour_fg=HELLO_COLOURS)
140 | 
141 |   # Let the game begin!
142 |   ui.play(game)
143 | 
144 | 
145 | if __name__ == '__main__':
146 |   main(sys.argv)
147 | 


--------------------------------------------------------------------------------
/pycolab/examples/ordeal.py:
--------------------------------------------------------------------------------
  1 | # Copyright 2018 the pycolab Authors
  2 | #
  3 | # Licensed under the Apache License, Version 2.0 (the "License");
  4 | # you may not use this file except in compliance with the License.
  5 | # You may obtain a copy of the License at
  6 | #
  7 | #     https://www.apache.org/licenses/LICENSE-2.0
  8 | #
  9 | # Unless required by applicable law or agreed to in writing, software
 10 | # distributed under the License is distributed on an "AS IS" BASIS,
 11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 12 | # See the License for the specific language governing permissions and
 13 | # limitations under the License.
 14 | 
 15 | """An heroic undertaking of exploration and derring-do! Slay the dragon/duck!
 16 | 
 17 | Also, a demonstration of `storytelling.Story`.
 18 | 
 19 | Keys: up, down, left, right - move. q - quit.
 20 | """
 21 | 
 22 | from __future__ import absolute_import
 23 | from __future__ import division
 24 | from __future__ import print_function
 25 | 
 26 | import curses
 27 | 
 28 | import sys
 29 | 
 30 | from pycolab import ascii_art
 31 | from pycolab import cropping
 32 | from pycolab import human_ui
 33 | from pycolab import storytelling
 34 | from pycolab import things as plab_things
 35 | from pycolab.prefab_parts import sprites as prefab_sprites
 36 | 
 37 | 
 38 | GAME_ART_CASTLE = ['##  ##   ##  ##',
 39 |                    '###############',
 40 |                    '#             #',
 41 |                    '#      D      #',  # As in "dragon" or "duck".
 42 |                    '#             #',
 43 |                    '#             #',
 44 |                    '#             #',
 45 |                    '###### P ######']  # As in "player".
 46 | 
 47 | 
 48 | GAME_ART_CAVERN = ['@@@@@@@@@@@@@@@',
 49 |                    '@@@@@@     @@@@',
 50 |                    '@@@@@      @@@@',
 51 |                    '@ @@    S    @@',  # As in "sword".
 52 |                    '            @@@',
 53 |                    'P @@@     @@@@@',  # As in "player".
 54 |                    '@@@@@@  @@@@@@@',
 55 |                    '@@@@@@@@@@@@@@@']
 56 | 
 57 | 
 58 | GAME_ART_KANSAS = ['######%%%######wwwwwwwwwwwwwwwwwwwwww@wwwwwww',
 59 |                    'w~~~~~%%%~~~~~~~~~~~~~~~~@~~~wwwww~~~~~~~~~~@',
 60 |                    'ww~~~~%%%~~~~~~~~~@~~~~~~~~~~~~~~~~~~~~~~@@@@',
 61 |                    'ww~~~~~%%%%~~~~~~~~~~~~~~~~~~~~~~~~~~~~~@@@@@',
 62 |                    '@ww~~~~~~%%%%~~~~~~~~~~~~~@~~%%%%%%%%%%%%%%%%',
 63 |                    'ww~~~~~~~~~~%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%',
 64 |                    'w~~~~~~@~~~~~~~~%%%%%%%%%%%%%%~~~~~~~~~~~~@@@',
 65 |                    'ww~~~~~~~~~~P~~~~~~~~~~~~~~~~~~~~~~~~~@~~~@@@',  # "Player".
 66 |                    'wwww~@www~~~~~~~~~wwwwww~~~@~~~~wwwww~~~~~~ww',
 67 |                    'wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww']
 68 | 
 69 | 
 70 | # These colours are only for humans to see in the CursesUi.
 71 | COLOURS = {' ': (0, 0, 0),        # Interior floors.
 72 |            '#': (447, 400, 400),  # Castle walls.
 73 |            '%': (999, 917, 298),  # Road.
 74 |            '~': (270, 776, 286),  # Grass.
 75 |            '@': (368, 333, 388),  # Stones and cave walls.
 76 |            'w': (309, 572, 999),  # Water. (The shores of Kansas.)
 77 |            'P': (999, 364, 0),    # Player.
 78 |            'S': (500, 999, 948),  # Sword.
 79 |            'D': (670, 776, 192)}  # Dragonduck.
 80 | 
 81 | 
 82 | def make_game():
 83 |   """Builds and returns an Ordeal game."""
 84 | 
 85 |   # Factories for building the three subgames of Ordeal.
 86 |   def make_castle():
 87 |     return ascii_art.ascii_art_to_game(
 88 |         GAME_ART_CASTLE, what_lies_beneath=' ',
 89 |         sprites=dict(P=PlayerSprite, D=DragonduckSprite),
 90 |         update_schedule=['P', 'D'], z_order=['D', 'P'])
 91 | 
 92 |   def make_cavern():
 93 |     return ascii_art.ascii_art_to_game(
 94 |         GAME_ART_CAVERN, what_lies_beneath=' ',
 95 |         sprites=dict(P=PlayerSprite), drapes=dict(S=SwordDrape),
 96 |         update_schedule=['P', 'S'])
 97 | 
 98 |   def make_kansas():
 99 |     return ascii_art.ascii_art_to_game(
100 |         GAME_ART_KANSAS, what_lies_beneath='~', sprites=dict(P=PlayerSprite))
101 | 
102 |   # A cropper for cropping the "Kansas" part of the game to the size of the
103 |   # other two games.
104 |   crop_kansas = cropping.ScrollingCropper(
105 |       rows=8, cols=15, to_track='P', scroll_margins=(2, 3))
106 | 
107 |   return storytelling.Story(
108 |       chapters=dict(castle=make_castle, cavern=make_cavern, kansas=make_kansas),
109 |       croppers=dict(castle=None, cavern=None, kansas=crop_kansas),
110 |       first_chapter='kansas')
111 | 
112 | 
113 | class SwordDrape(plab_things.Drape):
114 |   """A `Drape` for the sword.
115 | 
116 |   This `Drape` simply disappears if the player sprite steps on any element where
117 |   its curtain is True, setting the 'has_sword' flag in the Plot as it goes.
118 |   I guess we'll give the player a reward for sword collection, too.
119 |   """
120 | 
121 |   def update(self, actions, board, layers, backdrop, things, the_plot):
122 |     if self.curtain[things['P'].position]:
123 |       the_plot['has_sword'] = True
124 |       the_plot.add_reward(1.0)
125 |     if the_plot.get('has_sword'): self.curtain[:] = False  # Only one sword.
126 | 
127 | 
128 | class DragonduckSprite(prefab_sprites.MazeWalker):
129 |   """A `Sprite` for the castle dragon, or duck. Whatever.
130 | 
131 |   This creature shuffles toward the player. It moves faster than the player
132 |   since it can go diagonally. If the player has the sword, then if the creature
133 |   touches the player, it dies and the player receives a point. Otherwise,
134 |   contact is fatal to the player.
135 |   """
136 | 
137 |   def __init__(self, corner, position, character):
138 |     """Simply registers impassables and board confinement to the superclass."""
139 |     super(DragonduckSprite, self).__init__(
140 |         corner, position, character, impassable='#', confined_to_board=True)
141 | 
142 |   def update(self, actions, board, layers, backdrop, things, the_plot):
143 |     if the_plot.frame == 0: return  # Do nothing on the first frame..
144 | 
145 |     # Where is the player in relation to us?
146 |     player = things['P'].position
147 |     relative_locations = (self.position.row > player.row,  # Player is above us.
148 |                           self.position.col < player.col,  # To the right.
149 |                           self.position.row < player.row,  # Below us.
150 |                           self.position.col > player.col)  # To the left.
151 | 
152 |     # Move toward player! -- (North..East...West..South).
153 |     if relative_locations == (True, False, False, False):
154 |       self._north(board, the_plot)
155 |     elif relative_locations == (True, True, False, False):
156 |       self._northeast(board, the_plot)
157 |     elif relative_locations == (False, True, False, False):
158 |       self._east(board, the_plot)
159 |     elif relative_locations == (False, True, True, False):
160 |       self._southeast(board, the_plot)
161 |     elif relative_locations == (False, False, True, False):
162 |       self._south(board, the_plot)
163 |     elif relative_locations == (False, False, True, True):
164 |       self._southwest(board, the_plot)
165 |     elif relative_locations == (False, False, False, True):
166 |       self._west(board, the_plot)
167 |     elif relative_locations == (True, False, False, True):
168 |       self._northwest(board, the_plot)
169 | 
170 |     # If we're on top of the player, battle! Note that we use the layers
171 |     # argument to determine whether we're on top, which keeps there from being
172 |     # battles that don't look like battles (basically, where the player and the
173 |     # monster both move to the same location, but the screen hasn't updated
174 |     # quite yet).
175 |     if layers['P'][self.position]:
176 |       # This battle causes a termination that will end the game.
177 |       the_plot.next_chapter = None
178 |       the_plot.terminate_episode()
179 |       # But who is the winner?
180 |       if the_plot.get('has_sword'):
181 |         the_plot.add_reward(1.0)   # Player had the sword! Our goose is cooked!
182 |         the_plot.change_z_order(move_this='D', in_front_of_that='P')
183 |       else:
184 |         the_plot.add_reward(-1.0)  # No sword. Chomp chomp chomp!
185 |         the_plot.change_z_order(move_this='P', in_front_of_that='D')
186 | 
187 | 
188 | class PlayerSprite(prefab_sprites.MazeWalker):
189 |   """A `Sprite` for our player.
190 | 
191 |   This `Sprite` mainly ties actions to the arrow keys, although it contains
192 |   some logic that uses the Plot to make sure that when we transition between
193 |   subgames (i.e. when we go from Cavern to Kansas and so forth), the player
194 |   position reflects the idea that we've only moved a single step in the
195 |   "real world".
196 |   """
197 | 
198 |   def __init__(self, corner, position, character):
199 |     """Simply registers impassables and board confinement to the superclass."""
200 |     super(PlayerSprite, self).__init__(
201 |         corner, position, character, impassable='@#w', confined_to_board=True)
202 |     # Like corner, but inclusive of the last row/column.
203 |     self._limits = self.Position(corner.row - 1, corner.col - 1)
204 | 
205 |   def update(self, actions, board, layers, backdrop, things, the_plot):
206 |     limits = self._limits  # Abbreviation.
207 | 
208 |     # This large if statement probably deserves to be abbreviated with a
209 |     # protocol. Anyhow, the first four clauses handle actual agent actions.
210 |     # Note how much of the work amounts to detecting whether we are moving
211 |     # beyond the edge of the board, and if so, which game we should go to next.
212 |     if actions == 0:  # go upward?
213 |       if the_plot.this_chapter == 'kansas' and self.position.row <= 0:
214 |         the_plot.next_chapter = 'castle'
215 |         the_plot.terminate_episode()
216 |       else:
217 |         self._north(board, the_plot)
218 | 
219 |     elif actions == 1:  # go downward?
220 |       if the_plot.this_chapter == 'castle' and self.position.row >= limits.row:
221 |         the_plot.next_chapter = 'kansas'
222 |         the_plot.terminate_episode()
223 |       else:
224 |         self._south(board, the_plot)
225 | 
226 |     elif actions == 2:  # go leftward?
227 |       if the_plot.this_chapter == 'cavern' and self.position.col <= 0:
228 |         the_plot.next_chapter = 'kansas'
229 |         the_plot.terminate_episode()
230 |       else:
231 |         self._west(board, the_plot)
232 | 
233 |     elif actions == 3:  # go rightward?
234 |       if the_plot.this_chapter == 'kansas' and self.position.col >= limits.col:
235 |         the_plot.next_chapter = 'cavern'
236 |         the_plot.terminate_episode()
237 |       else:
238 |         self._east(board, the_plot)
239 | 
240 |     elif actions == 4:  # just quit?
241 |       the_plot.next_chapter = None  # This termination will be final.
242 |       the_plot.terminate_episode()
243 | 
244 |     # This last clause of the big if statement handles the very first action in
245 |     # a game. If we are starting this game just after concluding the previous
246 |     # game in a Story, we teleport to a place that "lines up" with where we were
247 |     # in the last game, so that our motion appears to be smooth.
248 |     elif the_plot.frame == 0:
249 |       if (the_plot.prior_chapter == 'kansas' and
250 |           the_plot.this_chapter == 'castle'):
251 |         self._teleport((limits.row, the_plot['last_position'].col))
252 | 
253 |       elif (the_plot.prior_chapter == 'castle' and
254 |             the_plot.this_chapter == 'kansas'):
255 |         self._teleport((0, the_plot['last_position'].col))
256 | 
257 |       elif (the_plot.prior_chapter == 'kansas' and
258 |             the_plot.this_chapter == 'cavern'):
259 |         self._teleport((the_plot['last_position'].row, 0))
260 | 
261 |       elif (the_plot.prior_chapter == 'cavern' and
262 |             the_plot.this_chapter == 'kansas'):
263 |         self._teleport((the_plot['last_position'].row, limits.col))
264 | 
265 |     # We always save our position to support the teleporting just above.
266 |     the_plot['last_position'] = self.position
267 | 
268 | 
269 | def main(argv=()):
270 |   del argv  # Unused.
271 | 
272 |   # Build an Ordeal game.
273 |   game = make_game()
274 | 
275 |   # Make a CursesUi to play it with.
276 |   ui = human_ui.CursesUi(
277 |       keys_to_actions={curses.KEY_UP: 0, curses.KEY_DOWN: 1,
278 |                        curses.KEY_LEFT: 2, curses.KEY_RIGHT: 3,
279 |                        'q': 4, -1: None},  # quit
280 |       delay=200, colour_fg=COLOURS)
281 | 
282 |   # Let the game begin!
283 |   ui.play(game)
284 | 
285 | 
286 | if __name__ == '__main__':
287 |   main(sys.argv)
288 | 


--------------------------------------------------------------------------------
/pycolab/examples/research/README.md:
--------------------------------------------------------------------------------
 1 | # Environments for RL research
 2 | 
 3 | This subdirectory contains pycolab games that have been created to explore real
 4 | research questions in reinforcement learning and related fields. Developed in
 5 | the frenzy of scientific investigation, they may not be the tidiest
 6 | demonstrations of how to use pycolab (or Python for that matter, or perhaps even
 7 | computers in general). Additionally, the system requirements for the games
 8 | (required libraries, etc.) may differ from the base requirements for pycolab
 9 | itself. Still, they may help illustrate what you can build. Contents include:
10 | 
11 |   * `[lp-rnn]`: Challenging memory tasks for an upcoming arXiv paper release
12 |     (more details pending).
13 |   * `[box_world]`: Perceptually simple but combinatorially complex environment that requires abstract relational reasoning and planning.
14 |   * *your code here?*
15 | 


--------------------------------------------------------------------------------
/pycolab/examples/research/box_world/README.md:
--------------------------------------------------------------------------------
 1 | # Box-World environment
 2 | 
 3 | Box-World is a perceptually simple but combinatorially complex environment that
 4 | requires abstract relational reasoning and planning. It consists of a
 5 | fully-observable pixel grid representing a room with keys and boxes randomly
 6 | scattered. The agent, represented by a single dark gray pixel, is randomly
 7 | placed in this room and can move in four directions: _up_, _down_, _left_,
 8 | _right_. Keys are represented by a single colored pixel. The agent can pick up a
 9 | loose key (i.e., one not adjacent to any other colored pixel) by walking over
10 | it. Boxes are represented by two adjacent colored pixels - the pixel on the
11 | right represents the box's lock and its color indicates which key can be used to
12 | open that lock; the pixel on the left indicates the content of the box which is
13 | inaccessible while the box is locked. To collect the content of a box the agent
14 | must first collect the key that opens the box (the one that matches the lock's
15 | color) and walk over the lock, making the lock disappear. At this point the
16 | content of the box becomes accessible and can be picked up by the agent. Most
17 | boxes contain keys that, if made accessible, can be used to open other boxes.
18 | One of the boxes contains a gem, represented by a single white pixel. The goal
19 | of the agent is to collect the gem. Keys in the agent's possession are depicted
20 | as a pixel in the top-left corner.
21 | 
22 | In each level there is a unique sequence of boxes that need to be opened in
23 | order to reach the gem. Opening one wrong box (a distractor box) leads to a
24 | dead-end where the gem cannot be reached and the level becomes unsolvable. There
25 | are three user-controlled parameters that contribute to the difficulty of the
26 | level: (1) the number of boxes in the path to the goal (solution length); (2)
27 | the number of distractor branches; (3) the length of the distractor branches. In
28 | general, the task is difficult for a few reasons. First, a key can only be used
29 | once, so the agent must be able to reason about whether a particular box is
30 | along a distractor branch or along the solution path. Second, keys and boxes
31 | appear in random locations in the room, emphasising a capacity to reason about
32 | keys and boxes based on their abstract relations, rather than based on their
33 | spatial positions.
34 | 
35 | Each level in Box-World is procedurally generated. We start by generating a
36 | random graph (a tree) that defines the correct path to the goal - i.e., the
37 | sequence of boxes that need to be opened to reach the gem. This graph also
38 | defines multiple distractor branches - boxes that lead to dead-ends. The agent,
39 | keys and boxes, including the one containing the gem, are positioned randomly in
40 | the room, assuring that there is enough space for the agent to navigate between
41 | boxes. There is a total of 20 keys and 20 locks that are randomly sampled to
42 | produce the level. An agent receives a reward of +10 for collecting the gem, +1
43 | for opening a box in the solution path and -1 for opening a distractor box. A
44 | level terminates immediately after the gem is collected, a distractor box is
45 | opened, or a maximum number of steps is reached (default is 120 steps). The
46 | generation process produces a very large number of possible trees, making it
47 | extremely unlikely that the agent will face the same level twice.
48 | 
49 | This environment was first introduced in the following paper:
50 | [https://arxiv.org/abs/1806.01830](https://arxiv.org/abs/1806.01830)
51 | 


--------------------------------------------------------------------------------
/pycolab/examples/research/box_world/box_world.py:
--------------------------------------------------------------------------------
  1 | # Copyright 2019 the pycolab Authors
  2 | #
  3 | # Licensed under the Apache License, Version 2.0 (the "License");
  4 | # you may not use this file except in compliance with the License.
  5 | # You may obtain a copy of the License at
  6 | #
  7 | #     https://www.apache.org/licenses/LICENSE-2.0
  8 | #
  9 | # Unless required by applicable law or agreed to in writing, software
 10 | # distributed under the License is distributed on an "AS IS" BASIS,
 11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 12 | # See the License for the specific language governing permissions and
 13 | # limitations under the License.
 14 | """Box-World game for Pycolab.
 15 | 
 16 | See README.md for more details.
 17 | """
 18 | 
 19 | from __future__ import absolute_import
 20 | from __future__ import division
 21 | from __future__ import print_function
 22 | 
 23 | import argparse
 24 | import itertools
 25 | import string
 26 | import sys
 27 | import numpy as np
 28 | 
 29 | from pycolab import ascii_art
 30 | from pycolab import human_ui
 31 | from pycolab import things as plab_things
 32 | from pycolab.prefab_parts import sprites as prefab_sprites
 33 | 
 34 | if __name__ == '__main__':  # Avoid defining flags when used as a library.
 35 |   parser = argparse.ArgumentParser(description='Play Box-World.')
 36 |   parser.add_argument(
 37 |       '--grid_size', type=int, default=12, help='height and width of the grid.')
 38 |   parser.add_argument(
 39 |       '--solution_length',
 40 |       nargs='+',
 41 |       type=int,
 42 |       default=(1, 2, 3, 4),
 43 |       help='number of boxes in the path to the goal.')
 44 |   parser.add_argument(
 45 |       '--num_forward',
 46 |       nargs='+',
 47 |       type=int,
 48 |       default=(0, 1, 2, 3, 4),
 49 |       help='possible values for num of forward distractors.')
 50 |   parser.add_argument(
 51 |       '--num_backward',
 52 |       nargs='+',
 53 |       type=int,
 54 |       default=(0,),
 55 |       help='possible values for num of backward distractors.')
 56 |   parser.add_argument(
 57 |       '--branch_length',
 58 |       type=int,
 59 |       default=1,
 60 |       help='length of forward distractor branches.')
 61 |   parser.add_argument(
 62 |       '--max_num_steps',
 63 |       type=int,
 64 |       default=120,
 65 |       help='number of steps before the episode is halted.')
 66 |   parser.add_argument(
 67 |       '--random_state',
 68 |       type=int,
 69 |       default=None,
 70 |       help='random number generator state.')
 71 |   FLAGS = parser.parse_args()
 72 | 
 73 | # Module constants
 74 | GEM = '*'
 75 | PLAYER = '.'
 76 | BACKGROUND = ' '
 77 | BORDER = '#'
 78 | 
 79 | COLORS = [(700, 350, 350), (700, 454, 350), (700, 559, 350), (700, 664, 350),
 80 |           (629, 700, 350), (524, 700, 350), (420, 700, 350), (350, 700, 384),
 81 |           (350, 700, 490), (350, 700, 595), (350, 700, 700), (350, 594, 700),
 82 |           (350, 490, 700), (350, 384, 700), (419, 350, 700), (524, 350, 700),
 83 |           (630, 350, 700), (700, 350, 665), (700, 350, 559), (700, 350, 455)]
 84 | 
 85 | MAX_NUM_KEYS = len(COLORS)
 86 | KEYS = list(string.ascii_lowercase[:MAX_NUM_KEYS])
 87 | LOCKS = list(string.ascii_uppercase[:MAX_NUM_KEYS])
 88 | 
 89 | OBJECT_COLORS = {
 90 |     PLAYER: (500, 500, 500),
 91 |     GEM: (999, 999, 999),
 92 |     BACKGROUND: (800, 800, 800),
 93 |     BORDER: (0, 0, 0),
 94 | }
 95 | OBJECT_COLORS.update({k: v for (k, v) in zip(KEYS, COLORS)})
 96 | OBJECT_COLORS.update({k: v for (k, v) in zip(LOCKS, COLORS)})
 97 | 
 98 | REWARD_GOAL = 10.
 99 | REWARD_STEP = 0.
100 | REWARD_OPEN_CORRECT = 1.
101 | REWARD_OPEN_WRONG = -1.
102 | 
103 | WALL_WIDTH = 1
104 | 
105 | MAX_PLACEMENT_TRIES = 200
106 | MAX_GENERATION_TRIES = 200
107 | 
108 | NORTH = (-1, 0)
109 | EAST = (0, 1)
110 | SOUTH = (1, 0)
111 | WEST = (0, -1)
112 | 
113 | ACTION_NORTH = 0
114 | ACTION_SOUTH = 1
115 | ACTION_WEST = 2
116 | ACTION_EAST = 3
117 | ACTION_DELAY = -1
118 | 
119 | ACTION_MAP = {
120 |     ACTION_NORTH: NORTH,
121 |     ACTION_SOUTH: SOUTH,
122 |     ACTION_WEST: WEST,
123 |     ACTION_EAST: EAST,
124 | }
125 | 
126 | 
127 | class PlayerSprite(prefab_sprites.MazeWalker):
128 |   """A `Sprite` for the player.
129 | 
130 |   This `Sprite` simply ties actions to moving.
131 |   """
132 | 
133 |   def __init__(self, corner, position, character, grid_size, x, y, distractors,
134 |                max_num_steps):
135 |     """Initialise a PlayerSprite.
136 | 
137 |     Args:
138 |       corner: standard `Sprite` constructor parameter.
139 |       position: standard `Sprite` constructor parameter.
140 |       character: standard `Sprite` constructor parameter.
141 |       grid_size: int, height and width of the grid.
142 |       x: int, initial player x coordinate on the grid.
143 |       y: int, initial player y coordinate on the grid.
144 |       distractors: <desc>.
145 |       max_num_steps: int, number of steps before the episode is halted.
146 |     """
147 |     # Indicate to the superclass that we can't walk off the board.
148 |     super(PlayerSprite, self).__init__(
149 |         corner, [y, x], character, impassable=BORDER, confined_to_board=True)
150 |     self.distractors = distractors
151 |     self._max_num_steps = max_num_steps
152 |     self._step_counter = 0
153 | 
154 |   def _in_direction(self, direction, board):
155 |     """Report character in the direction of movement and its coordinates."""
156 |     new_position = np.array(self.position) + direction
157 |     char = chr(board[new_position[0], new_position[1]])
158 |     if char == BORDER:
159 |       return None, None
160 |     else:
161 |       return char, new_position.tolist()
162 | 
163 |   def update(self, actions, board, layers, backdrop, things, the_plot):
164 |     del backdrop  # Unused.
165 | 
166 |     # Only actually move if action is one of the 4 directions of movement.
167 |     # Action could be -1, in which case we skip this altogether.
168 |     if actions in range(4):
169 |       # Add penalty per step.
170 |       the_plot.add_reward(REWARD_STEP)
171 | 
172 |       direction = ACTION_MAP[actions]
173 |       target_character, target_position = self._in_direction(direction, board)
174 |       # This will return None if target_character is None
175 |       target_thing = things.get(target_character)
176 |       inventory_item = chr(board[0, 0])
177 | 
178 |       # Moving the player can only occur under 3 conditions:
179 |       # (1) Move if there is nothing in the way.
180 |       if not target_thing:
181 |         self._move(board, the_plot, direction)
182 |       else:
183 |         # (2) If there is a lock in the way, only move if you hold the key that
184 |         # opens the lock.
185 |         is_lock = target_character in LOCKS
186 |         if is_lock and inventory_item == target_thing.key_that_opens:
187 |           self._move(board, the_plot, direction)
188 |         # (3) If there is a gem or key in the way, only move if that thing is
189 |         # not locked.
190 |         thing_is_locked = target_thing.is_locked_at(things, target_position)
191 |         if not is_lock and not thing_is_locked:
192 |           self._move(board, the_plot, direction)
193 | 
194 |       self._step_counter += 1
195 | 
196 |       # Episode terminates if maximum number of steps is reached.
197 |       if self._step_counter > self._max_num_steps:
198 |         the_plot.terminate_episode()
199 | 
200 |       # Inform plot of overlap between player and thing.
201 |       if target_thing:
202 |         the_plot['over_this'] = (target_character, self.position)
203 | 
204 | 
205 | class BoxThing(plab_things.Drape):
206 |   """Base class for locks, keys and gems."""
207 | 
208 |   def __init__(self, curtain, character, x, y):
209 |     super(BoxThing, self).__init__(curtain, character)
210 |     self.curtain[y][x] = True
211 | 
212 |   def is_locked_at(self, things, position):
213 |     """Check if a key or gem is locked at a given position."""
214 |     y, x = position
215 |     # Loop through all possible locks that can be locking this key or gem.
216 |     for lock_chr in things.keys():
217 |       if lock_chr in LOCKS and things[lock_chr].curtain[y][x + 1]:
218 |         return True
219 |     return False
220 | 
221 |   def where_player_over_me(self, the_plot):
222 |     """Check if player is over this thing. If so, returns the coordinates."""
223 |     over_this = the_plot.get('over_this')
224 |     if over_this:
225 |       character, (y, x) = over_this
226 |       if character == self.character and self.curtain[y][x]:
227 |         return y, x
228 |     else:
229 |       return False
230 | 
231 | 
232 | class GemDrape(BoxThing):
233 |   """The gem."""
234 | 
235 |   def update(self, actions, board, layers, backdrop, things, the_plot):
236 |     if self.where_player_over_me(the_plot):
237 |       the_plot.add_reward(REWARD_GOAL)
238 |       the_plot.terminate_episode()
239 | 
240 | 
241 | class KeyDrape(BoxThing):
242 |   """The keys."""
243 | 
244 |   def update(self, actions, board, layers, backdrop, things, the_plot):
245 |     position = self.where_player_over_me(the_plot)
246 |     if position:
247 |       inventory_item = chr(board[0, 0])
248 |       if inventory_item in KEYS:
249 |         things[inventory_item].curtain[0][0] = False
250 |       self.curtain[position[0]][position[1]] = False
251 |       self.curtain[0][0] = True
252 | 
253 | 
254 | class LockDrape(BoxThing):
255 |   """The locks."""
256 | 
257 |   def __init__(self, curtain, character, x, y):
258 |     super(LockDrape, self).__init__(curtain, character, x, y)
259 |     self.key_that_opens = KEYS[LOCKS.index(self.character)]
260 | 
261 |   def update(self, actions, board, layers, backdrop, things, the_plot):
262 |     position = self.where_player_over_me(the_plot)
263 |     if position:
264 |       self.curtain[position[0]][position[1]] = False
265 |       inventory_item = chr(board[0, 0])
266 |       things[inventory_item].curtain[0][0] = False
267 |       if (position[1], position[0]) in things[PLAYER].distractors:
268 |         the_plot.add_reward(REWARD_OPEN_WRONG)
269 |         the_plot.terminate_episode()
270 |       else:
271 |         the_plot.add_reward(REWARD_OPEN_CORRECT)
272 | 
273 | 
274 | def _sample_keys_locks_long(rand,
275 |                             solution_length_range,
276 |                             num_forward_range,
277 |                             num_backward_range,
278 |                             branch_length=1):
279 |   """Randomly sample a new problem."""
280 | 
281 |   solution_length = rand.choice(solution_length_range)
282 |   num_forward = rand.choice(num_forward_range)
283 |   num_backward = rand.choice(num_backward_range)
284 | 
285 |   locks = list(range(solution_length + 1))
286 |   keys = list(range(1, solution_length + 1)) + [-1]
287 | 
288 |   # Forward distractors
289 |   for _ in range(num_forward):
290 |     lock = rand.choice(range(1, solution_length + 1))
291 |     for _ in range(branch_length):
292 |       key = None
293 |       while key is None or key == lock:
294 |         key = rand.choice(range(solution_length + 1, MAX_NUM_KEYS))
295 |       locks.append(lock)
296 |       keys.append(key)
297 |       lock = key
298 | 
299 |   # Backward distractors. Note that branch length is not implemented here.
300 |   for _ in range(num_backward):
301 |     key = rand.choice(range(1, solution_length + 1))
302 |     lock = rand.choice(range(solution_length + 1, MAX_NUM_KEYS))
303 |     locks.append(lock)
304 |     keys.append(key)
305 | 
306 |   return (solution_length, np.array([locks, keys]).T)
307 | 
308 | 
309 | def _check_spacing(art, x, y):
310 |   """Check that there's room for key and adjacent lock (incl. surround)."""
311 |   bg = BACKGROUND
312 |   space_for_key = all(
313 |       art[i][j] == bg
314 |       for i, j in itertools.product(range(y - 1, y + 2), range(x - 1, x + 2)))
315 |   also_space_for_box = all(art[i][x + 2] == bg for i in range(y - 1, y + 2))
316 |   return space_for_key and also_space_for_box
317 | 
318 | 
319 | def _generate_random_game(rand, grid_size, solution_length, num_forward,
320 |                           num_backward, branch_length, max_num_steps):
321 |   """Generate game proceduraly; aborts if `MAX_PLACEMENT_TRIES` is reached."""
322 | 
323 |   # Sample new problem.
324 |   solution_length, locks_keys = _sample_keys_locks_long(rand,
325 |                                                         solution_length,
326 |                                                         num_forward,
327 |                                                         num_backward,
328 |                                                         branch_length)
329 | 
330 |   # By randomizing the list of keys and locks we use all the possible colors.
331 |   key_lock_ids = list(zip(KEYS, LOCKS))
332 |   rand.shuffle(key_lock_ids)
333 | 
334 |   full_map_size = grid_size + WALL_WIDTH * 2
335 |   art = [
336 |       [BACKGROUND for i in range(full_map_size)] for _ in range(full_map_size)
337 |   ]
338 | 
339 |   art = np.array(art)
340 |   art[:WALL_WIDTH, :] = BORDER
341 |   art[-WALL_WIDTH:, :] = BORDER
342 |   art[:, :WALL_WIDTH] = BORDER
343 |   art[:, -WALL_WIDTH:] = BORDER
344 | 
345 |   drapes = {}
346 |   distractors = []
347 |   placement_tries = 0
348 | 
349 |   # Place items necessary for the sampled problem
350 |   for i, (l, k) in enumerate(locks_keys):
351 |     is_distractor = False
352 |     if i > solution_length:
353 |       is_distractor = True
354 |     placed = False
355 |     while not placed:
356 |       if placement_tries > MAX_PLACEMENT_TRIES:
357 |         return False
358 |       x = rand.randint(0, grid_size - 3) + WALL_WIDTH
359 |       y = rand.randint(1, grid_size - 1) + WALL_WIDTH
360 |       if _check_spacing(art, x, y):
361 |         placed = True
362 |         # Check if box contains the gem
363 |         if k == -1:
364 |           art[y][x] = GEM
365 |           drapes[GEM] = ascii_art.Partial(GemDrape, x=x, y=y)
366 |         else:
367 |           key = key_lock_ids[k - 1][0]
368 |           art[y][x] = key
369 |           drapes[key] = ascii_art.Partial(KeyDrape, x=x, y=y)
370 |         # Check if box has a lock
371 |         if l != 0:
372 |           lock = key_lock_ids[l - 1][1]
373 |           art[y][x + 1] = lock
374 |           drapes[lock] = ascii_art.Partial(LockDrape, x=x + 1, y=y)
375 |           if is_distractor:
376 |             distractors.append((x + 1, y))
377 |       else:
378 |         placement_tries += 1
379 | 
380 |   # Place player
381 |   placed = False
382 |   while not placed:
383 |     if placement_tries > MAX_PLACEMENT_TRIES:
384 |       return False
385 |     x = rand.randint(0, grid_size - 1) + WALL_WIDTH
386 |     y = rand.randint(1, grid_size - 1) + WALL_WIDTH
387 |     if art[y][x] == BACKGROUND:
388 |       sprites = {
389 |           PLAYER:
390 |               ascii_art.Partial(PlayerSprite, grid_size, x, y, distractors,
391 |                                 max_num_steps)
392 |       }
393 |       placed = True
394 |       art[y][x] = PLAYER
395 |     else:
396 |       placement_tries += 1
397 | 
398 |   order = sorted(drapes.keys())
399 |   update_schedule = [PLAYER] + order
400 |   z_order = order + [PLAYER]
401 | 
402 |   art_as_list_of_strings = []
403 |   for art_ in art:
404 |     art_as_list_of_strings.append(''.join(art_))
405 |   art = art_as_list_of_strings
406 | 
407 |   art = [''.join(a) for a in art]
408 |   game = ascii_art.ascii_art_to_game(
409 |       art=art,
410 |       what_lies_beneath=BACKGROUND,
411 |       sprites=sprites,
412 |       drapes=drapes,
413 |       update_schedule=update_schedule,
414 |       z_order=z_order)
415 |   return game
416 | 
417 | 
418 | def make_game(grid_size,
419 |               solution_length,
420 |               num_forward,
421 |               num_backward,
422 |               branch_length,
423 |               random_state=None,
424 |               max_num_steps=120):
425 |   """Create a new Box-World game."""
426 | 
427 |   if random_state is None:
428 |     random_state = np.random.RandomState(None)
429 | 
430 |   game = False
431 |   tries = 0
432 |   while tries < MAX_GENERATION_TRIES and not game:
433 |     game = _generate_random_game(
434 |         random_state,
435 |         grid_size=grid_size,
436 |         solution_length=solution_length,
437 |         num_forward=num_forward,
438 |         num_backward=num_backward,
439 |         branch_length=branch_length,
440 |         max_num_steps=max_num_steps)
441 |     tries += 1
442 | 
443 |   if not game:
444 |     raise RuntimeError('Could not generate game in MAX_GENERATION_TRIES tries.')
445 |   return game
446 | 
447 | 
448 | def main(unused_argv):
449 | 
450 |   game = make_game(
451 |       grid_size=FLAGS.grid_size,
452 |       solution_length=FLAGS.solution_length,
453 |       num_forward=FLAGS.num_forward,
454 |       num_backward=FLAGS.num_backward,
455 |       branch_length=FLAGS.branch_length,
456 |       max_num_steps=FLAGS.max_num_steps,
457 |       random_state=FLAGS.random_state,
458 |   )
459 | 
460 |   ui = human_ui.CursesUi(
461 |       keys_to_actions={
462 |           'w': ACTION_NORTH,
463 |           's': ACTION_SOUTH,
464 |           'a': ACTION_WEST,
465 |           'd': ACTION_EAST,
466 |           -1: ACTION_DELAY,
467 |       },
468 |       delay=50,
469 |       colour_fg=OBJECT_COLORS)
470 |   ui.play(game)
471 | 
472 | 
473 | if __name__ == '__main__':
474 |   main(sys.argv)
475 | 


--------------------------------------------------------------------------------
/pycolab/examples/research/lp-rnn/README.md:
--------------------------------------------------------------------------------
 1 | # Tasks for LP-RNN
 2 | 
 3 | Three tasks that are featured in an upcoming paper that discusses a memory
 4 | architecture for reinforcement learning (more info available soon). Refer to the
 5 | top-level docstrings inside each `.py` file for details.
 6 | 
 7 | Note: These files are "research code" and should not necessarily be considered
 8 | good examples of pycolab programs (see the README for the parent directory).
 9 | Additionally, while pycolab is compatible with recent versions of python 2.7 and
10 | python 3, these files have only been used with python 2.7. Python 3
11 | compatibility has not been tested.
12 | 


--------------------------------------------------------------------------------
/pycolab/examples/research/lp-rnn/cued_catch.py:
--------------------------------------------------------------------------------
  1 | # Copyright 2018 the pycolab Authors
  2 | #
  3 | # Licensed under the Apache License, Version 2.0 (the "License");
  4 | # you may not use this file except in compliance with the License.
  5 | # You may obtain a copy of the License at
  6 | #
  7 | #     https://www.apache.org/licenses/LICENSE-2.0
  8 | #
  9 | # Unless required by applicable law or agreed to in writing, software
 10 | # distributed under the License is distributed on an "AS IS" BASIS,
 11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 12 | # See the License for the specific language governing permissions and
 13 | # limitations under the License.
 14 | 
 15 | """Position yourself to catch blocks based on a visual cue.
 16 | 
 17 | This game proceeds in two phases. The first phase is a "programming phase",
 18 | where the player sees each of the four visual cues (green blocks at the bottom
 19 | of the game board) paired randomly with either of two additional visual cues
 20 | (larger green blocks just above the cues, called "ball symbols"). These pairings
 21 | tell the player what actions they should take in the second phase of the game.
 22 | 
 23 | In the second phase of the game, the player must repeatedly move itself up or
 24 | down to position itself in front of either of two blocks: a yellow block or a
 25 | cyan block. These blocks approach the player from right to left. If the player
 26 | "catches" the correct block, it receives a point. The correct block is indicted
 27 | by the visual cue shown as the blocks begin to approach the player. If the cue
 28 | was paired with the left "ball symbol" during the programming phase, the player
 29 | should catch the yellow block; otherwise it should catch the cyan block.
 30 | 
 31 | Each episode of "Cued Catch" starts with a different mapping from cues to
 32 | blocks.  The player must learn to remember these associations in order to play
 33 | the game successfully.
 34 | """
 35 | 
 36 | from __future__ import absolute_import
 37 | from __future__ import division
 38 | from __future__ import print_function
 39 | 
 40 | import argparse
 41 | import curses
 42 | import random
 43 | import sys
 44 | 
 45 | from pycolab import ascii_art
 46 | from pycolab import human_ui
 47 | from pycolab import things as plab_things
 48 | from pycolab.prefab_parts import sprites as prefab_sprites
 49 | 
 50 | 
 51 | # ASCII art for the game board. Not too representative: there is usually some
 52 | # cue showing on some part of the board.
 53 | GAME_ART = [
 54 |     '            ',
 55 |     '   P    a   ',
 56 |     '        b   ',
 57 |     '            ',
 58 |     '            ',
 59 |     '            ',
 60 |     '            ',
 61 | ]
 62 | 
 63 | 
 64 | if __name__ == '__main__':  # Avoid defining flags when used as a library.
 65 |   parser = argparse.ArgumentParser(
 66 |       description='Play Cued Catch.',
 67 |       epilog=(
 68 |           'NOTE: Default options configure the game as the agents in the paper '
 69 |           'played it. These settings may not be much fun for humans, though.'))
 70 |   parser.add_argument('--initial_cue_duration', metavar='t', type=int,
 71 |                       default=10, help='Programming cue duration.')
 72 |   parser.add_argument('--cue_duration', metavar='t', type=int, default=10,
 73 |                       help='Query cue duration.')
 74 |   parser.add_argument('--num_trials', metavar='K', type=int, default=100,
 75 |                       help='Number of trials per episode.')
 76 |   # This flag is for establishing a control that requires no long term memory.
 77 |   parser.add_argument('--always_show_ball_symbol', action='store_true',
 78 |                       help='Control case: show ball symbols during trials.')
 79 |   # This flag is for experiments that require noise-tolerant memory.
 80 |   parser.add_argument('--reward_sigma', metavar='s', type=float, default=0.0,
 81 |                       help='Stddev for noise to add to ball-catch rewards.')
 82 |   # This flag is for experiments that require very long term memory.
 83 |   parser.add_argument('--reward_free_trials', metavar='K', type=int, default=40,
 84 |                       help='Provide no reward for the first K trials')
 85 |   FLAGS = parser.parse_args()
 86 | 
 87 | 
 88 | # These colours are only for humans to see in the CursesUi.
 89 | COLOURS = {' ': (0, 0, 0),        # Black background
 90 |            'P': (999, 999, 999),  # This is you, the player
 91 |            'Q': (0, 999, 0),      # Cue blocks
 92 |            'a': (999, 999, 0),    # Top ball
 93 |            'b': (0, 999, 999)}    # Bottom ball
 94 | 
 95 | 
 96 | def make_game(initial_cue_duration, cue_duration, num_trials,
 97 |               always_show_ball_symbol=False,
 98 |               reward_sigma=0.0,
 99 |               reward_free_trials=0):
100 |   return ascii_art.ascii_art_to_game(
101 |       art=GAME_ART,
102 |       what_lies_beneath=' ',
103 |       sprites={'P': ascii_art.Partial(
104 |           PlayerSprite,
105 |           reward_sigma=reward_sigma,
106 |           reward_free_trials=reward_free_trials),
107 |                'a': BallSprite,
108 |                'b': BallSprite},
109 |       drapes={'Q': ascii_art.Partial(
110 |           CueDrape,
111 |           initial_cue_duration, cue_duration, num_trials,
112 |           always_show_ball_symbol)},
113 |       update_schedule=['P', 'a', 'b', 'Q'])
114 | 
115 | 
116 | class PlayerSprite(prefab_sprites.MazeWalker):
117 |   """A `Sprite` for our player, the catcher."""
118 | 
119 |   def __init__(self, corner, position, character,
120 |                reward_sigma=0.0, reward_free_trials=0):
121 |     """Initialise a PlayerSprite.
122 | 
123 |     Args:
124 |       corner: standard `Sprite` constructor parameter.
125 |       position: standard `Sprite` constructor parameter.
126 |       character: standard `Sprite` constructor parameter.
127 |       reward_sigma: standard deviation of reward for catching the ball (or
128 |           not). A value of 0.0 means rewards with no noise.
129 |       reward_free_trials: number of trials before any reward can be earned.
130 |     """
131 |     super(PlayerSprite, self).__init__(
132 |         corner, position, character, impassable='', confined_to_board=True)
133 |     self._reward_sigma = reward_sigma
134 |     self._trials_till_reward = reward_free_trials
135 | 
136 |   def update(self, actions, board, layers, backdrop, things, the_plot):
137 |     # Our motions are quite constrained: we can only move up or down one spot.
138 |     if actions == 1 and self.virtual_position.row > 1:    # go up?
139 |       self._north(board, the_plot)
140 |     elif actions == 2 and self.virtual_position.row < 2:  # go down?
141 |       self._south(board, the_plot)
142 |     elif actions in [0, 4]:                               # quit the game?
143 |       the_plot.terminate_episode()
144 |     else:                                                 # do nothing?
145 |       self._stay(board, the_plot)                         # (or can't move?)
146 | 
147 |     # Give ourselves a point if we landed on the correct ball.
148 |     correct_ball = 'a' if the_plot.get('which_ball') == 'top' else 'b'
149 |     if self._reward_sigma:
150 |       if (self.position.col == things[correct_ball].position.col and
151 |           self._trials_till_reward <= 0):
152 |         the_plot.add_reward(
153 |             float(self.position == things[correct_ball].position) +
154 |             random.normalvariate(mu=0, sigma=self._reward_sigma))
155 |       else:
156 |         the_plot.add_reward(0)
157 | 
158 |     else:
159 |       the_plot.add_reward(int(
160 |           self.position == things[correct_ball].position and
161 |           self._trials_till_reward <= 0
162 |       ))
163 | 
164 |     # Decrement trials left till reward.
165 |     if (self.position.col == things[correct_ball].position.col and
166 |         self._trials_till_reward > 0):
167 |       self._trials_till_reward -= 1
168 | 
169 | 
170 | class BallSprite(plab_things.Sprite):
171 |   """A `Sprite` for the balls approaching the player."""
172 | 
173 |   def __init__(self, corner, position, character):
174 |     """Mark ourselves as invisible at first."""
175 |     super(BallSprite, self).__init__(corner, position, character)
176 |     # Save start position.
177 |     self._start_position = position
178 |     # But mark ourselves invisible for now.
179 |     self._visible = False
180 | 
181 |   def update(self, actions, board, layers, backdrop, things, the_plot):
182 |     # Wait patiently until the initial programming cues have been shown.
183 |     if not the_plot.get('programming_complete'): return
184 | 
185 |     # Cues are shown; we are visible now.
186 |     self._visible = True
187 | 
188 |     # If we're to the left of the player, reposition ourselves back at the start
189 |     # position and tell the cue drape to pick a new correct ball.
190 |     if self.position.col < things['P'].position.col:
191 |       self._position = self._start_position
192 |       the_plot['last_ball_reset'] = the_plot.frame
193 |     else:
194 |       self._position = self.Position(self.position.row, self.position.col - 1)
195 | 
196 | 
197 | class CueDrape(plab_things.Drape):
198 |   """"Programs" the player, then chooses correct balls and shows cues.
199 | 
200 |   The cue drape goes through two phases.
201 | 
202 |   In the first phase, it presents each of the four cues serially along with a
203 |   symbol that indicates whether the top ball or the bottom ball is the correct
204 |   choice for that cue. (The symbol does not resemble one of the balls.) During
205 |   this phase, no balls appear. Agent actions can move the player but accomplish
206 |   nothing else. Each associational cue presentation lasts for a number of
207 |   timesteps controlled by the `initial_cue_duration` constructor argument.
208 | 
209 |   Once all four cues have been shown in this way, the second phase presents a
210 |   sequence of `num_trials` fixed-length trials. In each trial, one of the four
211 |   cues is shown for `cue_duration` timesteps, and the two balls advance toward
212 |   the player from the right-hand side of the screen. The agent must position the
213 |   player to "catch" the ball that matches the cue shown at the beginning of the
214 |   trial.
215 | 
216 |   The two phases can also be visually distinguished by the presence of some
217 |   additional markers on the board.
218 |   """
219 | 
220 |   _NUM_CUES = 4  # Must divide 12 evenly and be divisible by 2. So, 2, 4, 6, 12.
221 | 
222 |   def __init__(self, curtain, character,
223 |                initial_cue_duration,
224 |                cue_duration,
225 |                num_trials,
226 |                always_show_ball_symbol):
227 |     super(CueDrape, self).__init__(curtain, character)
228 | 
229 |     self._initial_cue_duration = initial_cue_duration
230 |     self._cue_duration = cue_duration
231 |     self._num_trials_left = num_trials
232 |     self._always_show_ball_symbol = always_show_ball_symbol
233 | 
234 |     # Assign balls to each of the cues.
235 |     self._cues_to_balls = random.sample(
236 |         ['top'] * (self._NUM_CUES // 2) + ['bottom'] * (self._NUM_CUES // 2),
237 |         self._NUM_CUES)
238 | 
239 |     self._phase = 'first'
240 |     # State for first phase.
241 |     self._first_phase_tick = self._NUM_CUES * self._initial_cue_duration
242 |     # State for second phase, initialised to bogus values.
243 |     self._second_phase_cue_choice = -1
244 |     self._second_phase_tick = -1
245 |     self._second_phase_last_reset = -float('inf')
246 | 
247 |   def update(self, actions, board, layers, backdrop, things, the_plot):
248 |     # Show the agent which phase we're in.
249 |     self._show_phase_cue(self._phase)
250 |     # Do phase-specific update.
251 |     if self._phase == 'first':
252 |       self._do_first_phase(the_plot)
253 |     elif self._phase == 'second':
254 |       self._do_second_phase(the_plot)
255 | 
256 |   ## Phase-specific updates.
257 | 
258 |   def _do_first_phase(self, the_plot):
259 |     # Iterate through showing each of the cues.
260 |     self._first_phase_tick -= 1  # Decrement number of steps left in this phase.
261 |     cue = self._first_phase_tick // self._initial_cue_duration
262 |     self._show_ball_symbol(self._cues_to_balls[cue])
263 |     self._show_cue(cue)
264 |     # End of phase? Move on to the next phase.
265 |     if self._first_phase_tick <= 0:
266 |       self._phase = 'second'
267 |       the_plot['programming_complete'] = True
268 |       self._second_phase_reset(the_plot)
269 | 
270 |   def _do_second_phase(self, the_plot):
271 |     self._show_ball_symbol('neither')  # Clear ball symbol.
272 |     # Reset ourselves if the balls have moved beyond the player.
273 |     if the_plot.get('last_ball_reset') > self._second_phase_last_reset:
274 |       self._second_phase_reset(the_plot)
275 |     # Show the cue if it's still visible in this trial.
276 |     if self._second_phase_tick > 0:
277 |       self._show_cue(self._second_phase_cue_choice)
278 |       if self._always_show_ball_symbol: self._show_ball_symbol(
279 |           self._cues_to_balls[self._second_phase_cue_choice])
280 |     else:
281 |       self._show_cue(None)
282 |       self._show_ball_symbol(None)
283 |     # Countdown second phase clock.
284 |     self._second_phase_tick -= 1
285 | 
286 |   def _second_phase_reset(self, the_plot):
287 |     self._second_phase_cue_choice = random.randrange(self._NUM_CUES)
288 |     the_plot['which_ball'] = self._cues_to_balls[self._second_phase_cue_choice]
289 |     self._second_phase_tick = self._cue_duration
290 |     self._second_phase_last_reset = the_plot.frame
291 |     # Terminate if we've run out of trials.
292 |     if self._num_trials_left <= 0: the_plot.terminate_episode()
293 |     self._num_trials_left -= 1
294 | 
295 |   ## Display helpers
296 | 
297 |   def _show_phase_cue(self, phase):
298 |     self.curtain[1:3, :] = False
299 |     if phase == 'first':
300 |       self.curtain[1:3, 0:2] = True
301 |       self.curtain[1:3, -2:] = True
302 |     # No cue for the second phase.
303 | 
304 |   def _show_ball_symbol(self, ball):
305 |     self.curtain[3:5, :] = False
306 |     if ball == 'top':
307 |       self.curtain[3:5, 0:6] = True
308 |     elif ball == 'bottom':
309 |       self.curtain[3:5, -6:] = True
310 | 
311 |   def _show_cue(self, cue=None):
312 |     self.curtain[-2:, :] = False
313 |     if 0 <= cue < self._NUM_CUES:
314 |       width = self.curtain.shape[1] // self._NUM_CUES
315 |       l = cue * width
316 |       r = l + width
317 |       self.curtain[-2:, l:r] = True
318 | 
319 | 
320 | def main(argv):
321 |   del argv  # Unused.
322 | 
323 |   # Build a cued_catch game.
324 |   game = make_game(FLAGS.initial_cue_duration,
325 |                    FLAGS.cue_duration, FLAGS.num_trials,
326 |                    FLAGS.always_show_ball_symbol,
327 |                    FLAGS.reward_sigma,
328 |                    FLAGS.reward_free_trials)
329 | 
330 |   # Make a CursesUi to play it with.
331 |   ui = human_ui.CursesUi(
332 |       keys_to_actions={curses.KEY_UP: 1, curses.KEY_DOWN: 2,
333 |                        -1: 3,
334 |                        'q': 4, 'Q': 4},
335 |       delay=200, colour_fg=COLOURS)
336 | 
337 |   # Let the game begin!
338 |   ui.play(game)
339 | 
340 | 
341 | if __name__ == '__main__':
342 |   main(sys.argv)
343 | 


--------------------------------------------------------------------------------
/pycolab/examples/research/lp-rnn/sequence_recall.py:
--------------------------------------------------------------------------------
  1 | # Copyright 2018 the pycolab Authors
  2 | #
  3 | # Licensed under the Apache License, Version 2.0 (the "License");
  4 | # you may not use this file except in compliance with the License.
  5 | # You may obtain a copy of the License at
  6 | #
  7 | #     https://www.apache.org/licenses/LICENSE-2.0
  8 | #
  9 | # Unless required by applicable law or agreed to in writing, software
 10 | # distributed under the License is distributed on an "AS IS" BASIS,
 11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 12 | # See the License for the specific language governing permissions and
 13 | # limitations under the License.
 14 | 
 15 | """The agent must remember a sequence of lights.
 16 | 
 17 | This task is reminiscent of electronic toy memory games from the '70s and '80s.
 18 | The player starts out immobilised at the centre of the screen while a sequence
 19 | of coloured lights flashes on the four surrounding "pads". After the sequence
 20 | ends, the agent is free to move. It must visit the pads in the same order as the
 21 | sequence it was just shown, as quickly as possible. If the same pad flashes
 22 | twice in a row, then the agent must enter, exit, and re-enter the pad in order
 23 | to replicate the sequence.
 24 | """
 25 | 
 26 | from __future__ import absolute_import
 27 | from __future__ import division
 28 | from __future__ import print_function
 29 | 
 30 | import argparse
 31 | import curses
 32 | import random
 33 | import sys
 34 | 
 35 | import enum
 36 | 
 37 | import numpy as np
 38 | from pycolab import ascii_art
 39 | from pycolab import human_ui
 40 | from pycolab import rendering
 41 | from pycolab import things as plab_things
 42 | from pycolab.prefab_parts import sprites as prefab_sprites
 43 | 
 44 | 
 45 | if __name__ == '__main__':  # Avoid defining flags when used as a library.
 46 |   parser = argparse.ArgumentParser(
 47 |       description='Play Sequence Recall.',
 48 |       epilog=(
 49 |           'NOTE: Default options configure the game as the agents in the paper '
 50 |           'played it. These settings may not be much fun for humans, though.'))
 51 |   parser.add_argument(
 52 |       '--sequence_length', metavar='K', type=int, default=4,
 53 |       help='Length of the light sequence the player must learn.')
 54 |   parser.add_argument(
 55 |       '--demo_light_on_frames', metavar='t', type=int, default=60,
 56 |       help='Lights during the "demo" stay on for t frames.')
 57 |   parser.add_argument(
 58 |       '--demo_light_off_frames', metavar='t', type=int, default=30,
 59 |       help='Lights during the "demo" stay off for t frames.')
 60 |   parser.add_argument(
 61 |       '--pause_frames', metavar='t', type=int, default=1,
 62 |       help='Agent is held t frames after the demo before moving.')
 63 |   parser.add_argument(
 64 |       '--timeout_frames', metavar='t', type=int, default=1000,
 65 |       help='Frames before game times out (-1 for infinity).')
 66 |   FLAGS = parser.parse_args()
 67 | 
 68 | 
 69 | GAME_ART = [
 70 |     '#####################',
 71 |     '#        222        #',
 72 |     '#      2222222      #',
 73 |     '#      2222222      #',
 74 |     '#      2222222      #',
 75 |     '#        222        #',
 76 |     '#  111         333  #',
 77 |     '#1111111 %%% 3333333#',
 78 |     '#1111111 %P% 3333333#',
 79 |     '#1111111 %%% 3333333#',
 80 |     '#  111         333  #',
 81 |     '#        444        #',
 82 |     '#      4444444      #',
 83 |     '#      4444444      #',
 84 |     '#      4444444      #',
 85 |     '#        444        #',
 86 |     '#####################',
 87 | ]
 88 | 
 89 | 
 90 | # Repaints the WaitForSeekDrape to look identical to the maze walls.
 91 | REPAINT_MAPPING = {
 92 |     '%': '#',
 93 | }
 94 | 
 95 | 
 96 | # These colours are only for humans to see in the CursesUi.
 97 | COLOURS = {' ': (0, 0, 0),        # Black background
 98 |            '#': (764, 0, 999),    # Board walls
 99 |            '1': (0, 999, 0),      # Green light
100 |            '2': (999, 0, 0),      # Red light
101 |            '3': (0, 0, 999),      # Blue light
102 |            '4': (999, 999, 0),    # Yellow light
103 |            'M': (300, 300, 300),  # Mask drape (for turning a light "off")
104 |            'P': (0, 999, 999)}    # Player
105 | 
106 | 
107 | class _State(enum.Enum):
108 |   """States for the game's internal state machine.
109 | 
110 |   In the game, states are placed in tuples with arguments like duration, etc.
111 |   """
112 |   # All lights off for some duration. Agent is frozen.
113 |   OFF = 0
114 | 
115 |   # Specified light on for some duration. Agent is frozen.
116 |   ON = 1
117 | 
118 |   # Agent is free to move. If they enter a specified light, they get a point. If
119 |   # they enter any other light, they lose a point.
120 |   SEEK = 2
121 | 
122 |   # Agent is free to move and is currently in one of the lights. This state is
123 |   # basically the game waiting for the agent to leave the light.
124 |   EXIT = 3
125 | 
126 |   # Game over!
127 |   QUIT = 4
128 | 
129 | 
130 | def make_game(sequence_length=4,
131 |               demo_light_on_frames=60,
132 |               demo_light_off_frames=30,
133 |               pause_frames=30,
134 |               timeout_frames=-1):
135 |   """Builds and returns a sequence_recall game."""
136 | 
137 |   # Sample a game-controlling state machine program.
138 |   program = _make_program(sequence_length,
139 |                           demo_light_on_frames, demo_light_off_frames,
140 |                           pause_frames)
141 | 
142 |   # Build the game engine.
143 |   engine = ascii_art.ascii_art_to_game(
144 |       GAME_ART, what_lies_beneath=' ',
145 |       sprites={'P': PlayerSprite},
146 |       drapes={'M': MaskDrape,
147 |               '%': WaitForSeekDrape},
148 |       update_schedule=['P', 'M', '%'],
149 |       z_order='MP%')
150 | 
151 |   # Save the program and other global state in the game's Plot object.
152 |   engine.the_plot['program'] = program
153 |   engine.the_plot['frames_in_state'] = 0  # How long in the current state?
154 |   engine.the_plot['timeout_frames'] = (  # Frames left until the game times out.
155 |       float('inf') if timeout_frames < 0 else timeout_frames)
156 | 
157 |   return engine
158 | 
159 | 
160 | def _make_program(sequence_length,
161 |                   demo_light_on_frames, demo_light_off_frames,
162 |                   pause_frames):
163 |   """Sample a game-controlling state machine program."""
164 |   # Select the sequence of lights that we'll illuminate.
165 |   sequence = [random.choice('1234') for _ in range(sequence_length)]
166 | 
167 |   # Now create the state machine program that will control the game.
168 |   program = []
169 |   # Phase 1. Present the sequence.
170 |   for g in sequence:
171 |     program.extend([
172 |         (_State.OFF, demo_light_off_frames),   # All lights off.
173 |         (_State.ON, demo_light_on_frames, g),  # Turn on light g
174 |     ])
175 |   # Phase 2. Detain the agent for a little while.
176 |   program.append(
177 |       (_State.OFF, max(1, pause_frames)),  # At least 1 to turn off the light.
178 |   )
179 |   # Phase 3. The agent tries to replicate the sequence.
180 |   for g in sequence:
181 |     program.extend([
182 |         (_State.SEEK, g),  # Agent should try to enter light g.
183 |         (_State.EXIT,),    # Agent must leave whatever light it's in.
184 |     ])
185 |   # Phase 4. Quit the game.
186 |   program[-1] = (_State.QUIT,)  # Replace final EXIT with a QUIT.
187 | 
188 |   return program
189 | 
190 | 
191 | class MaskDrape(plab_things.Drape):
192 |   """A `Drape` for the mask that obscures the game's lights.
193 | 
194 |   Also controls the state machine and performs score keeping.
195 |   """
196 | 
197 |   def __init__(self, curtain, character):
198 |     super(MaskDrape, self).__init__(curtain, character)
199 | 
200 |     # Both of the following to be filled by self._set_up_masks.
201 |     # What the contents of the curtain should be when all lights are off.
202 |     self._all_off_mask = None
203 |     # Which parts of the curtain cover which light.
204 |     self._mask_for_light = {g: None for g in '1234'}
205 | 
206 |   def _set_up_masks(self, backdrop):
207 |     self._all_off_mask = np.zeros_like(backdrop.curtain, dtype=np.bool)
208 |     for g in '1234':
209 |       mask = (backdrop.curtain == backdrop.palette[g])
210 |       self._mask_for_light[g] = mask
211 |       self._all_off_mask |= mask
212 | 
213 |   def update(self, actions, board, layers, backdrop, things, the_plot):
214 |     # One-time: set up our mask data.
215 |     if self._all_off_mask is None: self._set_up_masks(backdrop)
216 | 
217 |     state = the_plot['program'][0][0]  # Get current game state.
218 |     args = the_plot['program'][0][1:]  # Get all arguments for the state.
219 | 
220 |     # Get player position---it's often useful.
221 |     pos = things['P'].position
222 | 
223 |     # Increment the number of frames we will have been in this state at the end
224 |     # of this game step.
225 |     the_plot['frames_in_state'] += 1
226 |     frames_in_state = the_plot['frames_in_state']  # Abbreviated
227 | 
228 |     # Behave as dictated by the state machine.
229 |     if state == _State.QUIT:
230 |       if frames_in_state == 1:              # If we just entered the QUIT state,
231 |         the_plot['timeout_frames'] = 1      #     direct the game to time out.
232 | 
233 |     elif state == _State.OFF:
234 |       if frames_in_state == 1:              # If we just entered the OFF state,
235 |         self.curtain[:] |= self._all_off_mask  #  turn out all the lights.
236 |       elif frames_in_state >= args[0]:      # If we've been here long enough,
237 |         the_plot['program'].pop(0)          #     move on to the next state.
238 |         the_plot['frames_in_state'] = 0
239 | 
240 |     elif state == _State.ON:                # If we just entered the ON state,
241 |       if frames_in_state == 1:              #     turn on the specified light.
242 |         self.curtain[:] -= self._mask_for_light[args[1]]
243 |       elif frames_in_state >= args[0]:      # If we've been here long enough,
244 |         the_plot['program'].pop(0)          #     move on to the next state.
245 |         the_plot['frames_in_state'] = 0
246 | 
247 |     elif state == _State.SEEK:              # In the SEEK state, wait for the
248 |       agent_above = chr(backdrop.curtain[pos])  # agent to enter a light.
249 |       if agent_above != ' ':                    # Entry!
250 |         self.curtain[:] -= self._mask_for_light[agent_above]  # Light goes on.
251 |         the_plot.add_reward(                    # Was it the right light?
252 |             1.0 if agent_above == args[0]       #    Yes, reward for you!
253 |             else 0.0)                           #    No. You get nothing.
254 |         the_plot['program'].pop(0)              # On to the next state.
255 |         the_plot['frames_in_state'] = 0
256 | 
257 |     elif state == _State.EXIT:              # In the EXIT state, wait for the
258 |       agent_above = chr(backdrop.curtain[pos])  # agent to exit a light.
259 |       if agent_above == ' ':                    # Exit!
260 |         self.curtain[:] |= self._all_off_mask   # All lights go out.
261 |         the_plot['program'].pop(0)              # On to the next state.
262 |         the_plot['frames_in_state'] = 0
263 | 
264 | 
265 | class WaitForSeekDrape(plab_things.Drape):
266 |   """A `Drape` that disappears when the game first enters a SEEK state."""
267 | 
268 |   def update(self, actions, board, layers, backdrop, things, the_plot):
269 |     if (the_plot['frames_in_state'] == 1 and
270 |         the_plot['program'][0][0] == _State.SEEK and
271 |         self.curtain.any()): self.curtain[:] = False
272 | 
273 | 
274 | class PlayerSprite(prefab_sprites.MazeWalker):
275 |   """A `Sprite` for our player.
276 | 
277 |   This `Sprite` ties actions to going in the four cardinal directions. In
278 |   interactive settings, the user can also quit. `PlayerSprite` also administers
279 |   a small penalty at every timestep to motivate the agent to act quickly.
280 |   Finally, `PlayerSprite` handles episode timeout and all termination.
281 |   """
282 | 
283 |   def __init__(self, corner, position, character):
284 |     """Tells superclass we can't walk off the board or through walls."""
285 |     super(PlayerSprite, self).__init__(
286 |         corner, position, character, impassable='#', confined_to_board=True)
287 | 
288 |   def update(self, actions, board, layers, backdrop, things, the_plot):
289 |     del layers, backdrop, things  # Unused.
290 | 
291 |     state = the_plot['program'][0][0]  # Get current game state.
292 | 
293 |     if actions in [0, 6]:
294 |       # Humans can quit the game at any time.
295 |       the_plot['timeout_frames'] = 1
296 | 
297 |     elif state in (_State.SEEK, _State.EXIT):
298 |       # But no agent is allowed to move unless the game state permits it.
299 |       if actions == 1:    # go upward?
300 |         self._north(board, the_plot)
301 |       elif actions == 2:  # go downward?
302 |         self._south(board, the_plot)
303 |       elif actions == 3:  # go leftward?
304 |         self._west(board, the_plot)
305 |       elif actions == 4:  # go rightward?
306 |         self._east(board, the_plot)
307 |       elif actions == 5:  # do nothing?
308 |         self._stay(board, the_plot)
309 | 
310 |     # Quit the game if timeout occurs.
311 |     if the_plot['timeout_frames'] <= 0:
312 |       the_plot.terminate_episode()
313 |     else:
314 |       # Otherwise, add a slight penalty for all episode frames (except the
315 |       # first) to encourage the agent to act efficiently.
316 |       if the_plot.frame > 1: the_plot.add_reward(-0.005)
317 |       the_plot['timeout_frames'] -= 1
318 | 
319 | 
320 | def main(argv):
321 |   del argv  # Unused.
322 | 
323 |   # Build a sequence_recall game.
324 |   game = make_game(FLAGS.sequence_length,
325 |                    FLAGS.demo_light_on_frames,
326 |                    FLAGS.demo_light_off_frames,
327 |                    FLAGS.pause_frames,
328 |                    FLAGS.timeout_frames)
329 | 
330 |   # Build an ObservationCharacterRepainter that will turn the light numbers into
331 |   # actual colours.
332 |   repainter = rendering.ObservationCharacterRepainter(REPAINT_MAPPING)
333 | 
334 |   # Make a CursesUi to play it with.
335 |   ui = human_ui.CursesUi(
336 |       keys_to_actions={curses.KEY_UP: 1, curses.KEY_DOWN: 2,
337 |                        curses.KEY_LEFT: 3, curses.KEY_RIGHT: 4,
338 |                        -1: 5,
339 |                        'q': 6, 'Q': 6},
340 |       delay=100, repainter=repainter, colour_fg=COLOURS)
341 | 
342 |   # Let the game begin!
343 |   ui.play(game)
344 | 
345 | 
346 | if __name__ == '__main__':
347 |   main(sys.argv)
348 | 


--------------------------------------------------------------------------------
/pycolab/examples/shockwave.py:
--------------------------------------------------------------------------------
  1 | # Copyright 2017 the pycolab Authors
  2 | #
  3 | # Licensed under the Apache License, Version 2.0 (the "License");
  4 | # you may not use this file except in compliance with the License.
  5 | # You may obtain a copy of the License at
  6 | #
  7 | #     https://www.apache.org/licenses/LICENSE-2.0
  8 | #
  9 | # Unless required by applicable law or agreed to in writing, software
 10 | # distributed under the License is distributed on an "AS IS" BASIS,
 11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 12 | # See the License for the specific language governing permissions and
 13 | # limitations under the License.
 14 | 
 15 | """Get to the top to win, but watch out for the fiery shockwaves of death.
 16 | 
 17 | Command-line usage: `shockwave.py <level>`, where `<level>` is an optional
 18 | integer argument that is either -1 (selecting a randomly-generated map) or
 19 | 0 (selecting the map hard-coded in this module).
 20 | 
 21 | Tip: Try hiding in the blue bunkers.
 22 | 
 23 | Keys: up, left, right.
 24 | """
 25 | 
 26 | from __future__ import absolute_import
 27 | from __future__ import division
 28 | from __future__ import print_function
 29 | 
 30 | import curses
 31 | import sys
 32 | 
 33 | import numpy as np
 34 | from pycolab import ascii_art
 35 | from pycolab import human_ui
 36 | from pycolab import things as plab_things
 37 | from pycolab.prefab_parts import sprites as prefab_sprites
 38 | from scipy import ndimage
 39 | 
 40 | 
 41 | # Just one level for now.
 42 | LEVELS = [
 43 |     ['^^^^^^^^^^^^^^^',
 44 |      '               ',
 45 |      '  +           +',
 46 |      '  ==   ++  == +',
 47 |      '              +',
 48 |      '=======       +',
 49 |      ' +            +',
 50 |      '   +      ++   ',
 51 |      '+        ==    ',
 52 |      '+        +     ',
 53 |      '   =           ',
 54 |      ' +++ P    ++   '],
 55 | ]
 56 | 
 57 | 
 58 | COLOURS = {'+': (0, 0, 999),      # Blue background. Safe from fire here.
 59 |            'P': (0, 999, 0),      # The green player.
 60 |            ' ': (500, 500, 500),  # Exposed areas where the player might die.
 61 |            '^': (700, 700, 700),  # Permanent safe zone.
 62 |            '=': (999, 600, 200),  # Impassable wall.
 63 |            '@': (999, 0, 0)}      # The fiery shockwave.
 64 | 
 65 | 
 66 | def random_level(height=12, width=12, safety_density=0.15):
 67 |   """Returns a random level."""
 68 |   level = np.full((height, width), ' ', dtype='|S1')
 69 | 
 70 |   # Add some safe areas.
 71 |   level[np.random.random_sample(level.shape) < safety_density] = '+'
 72 | 
 73 |   # Place walls on random, but not consecutive, rows. Also not on the top or
 74 |   # bottom rows.
 75 |   valid_rows = set(range(1, height))
 76 | 
 77 |   while valid_rows:
 78 |     row = np.random.choice(list(valid_rows))
 79 | 
 80 |     n_walls = np.random.randint(2, width - 1 - 2)
 81 |     mask = np.zeros((width,), dtype=np.bool)
 82 |     mask[:n_walls] = True
 83 |     np.random.shuffle(mask)
 84 |     level[row, mask] = '='
 85 | 
 86 |     valid_rows.discard(row - 1)
 87 |     valid_rows.discard(row)
 88 |     valid_rows.discard(row + 1)
 89 | 
 90 |   # Add the player.
 91 |   level[-1, np.random.randint(0, width - 1)] = 'P'
 92 |   # Add the safe zone.
 93 |   level[0] = '^'
 94 |   return [row.tostring() for row in level]
 95 | 
 96 | 
 97 | class PlayerSprite(prefab_sprites.MazeWalker):
 98 | 
 99 |   def __init__(self, corner, position, character):
100 |     super(PlayerSprite, self).__init__(
101 |         corner, position, character, impassable='=', confined_to_board=True)
102 | 
103 |   def update(self, actions, board, layers, backdrop, things, the_plot):
104 | 
105 |     if actions == 0:    # go upward?
106 |       self._north(board, the_plot)
107 |     elif actions == 1:  # go leftward?
108 |       self._west(board, the_plot)
109 |     elif actions == 2:  # go rightward?
110 |       self._east(board, the_plot)
111 |     elif actions == 3:  # stay put!
112 |       self._stay(board, the_plot)
113 | 
114 | 
115 | class ShockwaveDrape(plab_things.Drape):
116 |   """Drape for the shockwave."""
117 | 
118 |   def __init__(self, curtain, character, width=2):
119 |     """Initializes the `ShockwaveDrape`.
120 | 
121 |     Args:
122 |       curtain: The curtain.
123 |       character: Character for this drape.
124 |       width: Integer width of the shockwave.
125 |     """
126 |     super(ShockwaveDrape, self).__init__(curtain, character)
127 |     self._width = width
128 |     self._distance_from_impact = np.zeros(self.curtain.shape)
129 |     self._steps_since_impact = 0
130 | 
131 |   def update(self, actions, board, layers, backdrop, things, the_plot):
132 | 
133 |     if not self.curtain.any():
134 |       impact_point = np.unravel_index(
135 |           np.random.randint(0, self.curtain.size),
136 |           self.curtain.shape)
137 | 
138 |       impact_map = np.full_like(self.curtain, True)
139 |       impact_map[impact_point] = False
140 | 
141 |       self._distance_from_impact = ndimage.distance_transform_edt(impact_map)
142 |       self._steps_since_impact = 0
143 | 
144 |       the_plot.log('BOOM! Shockwave initiated at {} at frame {}.'.format(
145 |           impact_point, the_plot.frame))
146 | 
147 |     self.curtain[:] = (
148 |         (self._distance_from_impact > self._steps_since_impact) &
149 |         (self._distance_from_impact <= self._steps_since_impact + self._width) &
150 |         (np.logical_not(layers['=']))
151 |     )
152 | 
153 |     # Check if the player is safe, dead, or has won.
154 |     player_position = things['P'].position
155 | 
156 |     if layers['^'][player_position]:
157 |       the_plot.add_reward(1)
158 |       the_plot.terminate_episode()
159 | 
160 |     under_fire = self.curtain[player_position]
161 |     in_danger_zone = things[' '].curtain[player_position]
162 | 
163 |     if under_fire and in_danger_zone:
164 |       the_plot.add_reward(-1)
165 |       the_plot.terminate_episode()
166 | 
167 |     self._steps_since_impact += 1
168 | 
169 | 
170 | class MinimalDrape(plab_things.Drape):
171 |   """A Drape that just holds a curtain and contains no game logic."""
172 | 
173 |   def update(self, actions, board, layers, backdrop, things, the_plot):
174 |     del actions, board, layers, backdrop, things, the_plot  # Unused.
175 | 
176 | 
177 | def make_game(level):
178 |   """Builds and returns a Shockwave game."""
179 |   if level == -1:
180 |     level_art = random_level()
181 |   else:
182 |     level_art = LEVELS[level]
183 | 
184 |   return ascii_art.ascii_art_to_game(
185 |       level_art,
186 |       what_lies_beneath='+',
187 |       sprites={'P': PlayerSprite},
188 |       drapes={'@': ShockwaveDrape, ' ': MinimalDrape, '^': MinimalDrape},
189 |       update_schedule=[' ', '^', 'P', '@'],
190 |       z_order=[' ', '^', '@', 'P'],
191 |   )
192 | 
193 | 
194 | def main(argv=()):
195 |   game = make_game(int(argv[1]) if len(argv) > 1 else 0)
196 | 
197 |   keys_to_actions = {
198 |       curses.KEY_UP: 0,
199 |       curses.KEY_LEFT: 1,
200 |       curses.KEY_RIGHT: 2,
201 |       -1: 3,
202 |   }
203 | 
204 |   ui = human_ui.CursesUi(
205 |       keys_to_actions=keys_to_actions,
206 |       delay=500, colour_fg=COLOURS)
207 | 
208 |   ui.play(game)
209 | 
210 | 
211 | if __name__ == '__main__':
212 |   main(sys.argv)
213 | 


--------------------------------------------------------------------------------
/pycolab/examples/tennnnnnnnnnnnnnnnnnnnnnnnis.py:
--------------------------------------------------------------------------------
  1 | # Copyright 2017 the pycolab Authors
  2 | #
  3 | # Licensed under the Apache License, Version 2.0 (the "License");
  4 | # you may not use this file except in compliance with the License.
  5 | # You may obtain a copy of the License at
  6 | #
  7 | #     https://www.apache.org/licenses/LICENSE-2.0
  8 | #
  9 | # Unless required by applicable law or agreed to in writing, software
 10 | # distributed under the License is distributed on an "AS IS" BASIS,
 11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 12 | # See the License for the specific language governing permissions and
 13 | # limitations under the License.
 14 | 
 15 | """What would it be like to play tennis in a long corridor?
 16 | 
 17 | This tennis-like uses a court too big to fit on your screen, so instead you
 18 | and your opponent get three separate views: one of your paddle, one of your
 19 | opponent's paddle, and one that follows the ball.
 20 | 
 21 | The game terminates when either player gets four points.
 22 | """
 23 | 
 24 | from __future__ import absolute_import
 25 | from __future__ import division
 26 | from __future__ import print_function
 27 | 
 28 | import random
 29 | 
 30 | import enum
 31 | import numpy as np
 32 | 
 33 | from pycolab import ascii_art
 34 | from pycolab import cropping
 35 | from pycolab import human_ui
 36 | from pycolab import things as plab_things
 37 | 
 38 | 
 39 | # pylint: disable=line-too-long
 40 | MAZE_ART = [
 41 |     '%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%',
 42 |     '%                   ##                                               # ###   # ###                                                                ###    ###                                              #             %',
 43 |     '%   1          #####                                         # ###   ##   #  ##   #  # ###                                                 ###   #      #       ###                                      #              %',
 44 |     '%   1   @     #   #    ###                           # ###   ##   #  #    #  #    #  ##   #  # ###                            #     ###   #       #      #     #       ###                               #              %',
 45 |     '%                 #   #   #                  # ###   ##   #  #    # #    #  #    #   #    #  ##   #  # ###                         #       #   ###    ###       #     #       ###                  ###  #               %',
 46 |     '%                 #  #####   # ###   # ###   ##   #  #    # #    #                  #    #   #    #  ##   #  # ###   # ###    #     #   ###                  ###       #     #       ###    ###   #                     %',
 47 |     '%                #   #       ##   #  ##   #  #    # #    #                                  #    #   #    #  ##   #  ##   #   #  ###                                ###       #     #      #       #   #            2   %',
 48 |     '%                     ####   #    #  #    # #    #                                                  #    #   #    #  #    #  #                                             ###       #      #   ###                 2   %',
 49 |     '%                           #    #  #    #                                                                  #    #  #    #                                                        ###    ###                            %',
 50 |     '%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%']
 51 | # pylint: enable=line-too-long
 52 | 
 53 | 
 54 | # These colours are only for humans to see in the CursesUi.
 55 | COLOUR_FG = {' ': (0, 0, 0),        # Default black background
 56 |              '%': (82, 383, 86),    # Dark green court border
 57 |              '#': (123, 574, 129),  # Lighter green court features
 58 |              '1': (999, 999, 999),  # White player-1 paddle
 59 |              '2': (999, 999, 999),  # White player-2 paddle
 60 |              '@': (787, 999, 227)}  # Tennis ball
 61 | 
 62 | COLOUR_BG = {'@': (0, 0, 0)}  # So the tennis ball looks like @ and not a block.
 63 | 
 64 | 
 65 | class Actions(enum.IntEnum):
 66 |   """Actions for paddle movement."""
 67 |   STAY = 0
 68 |   UP = 1
 69 |   DOWN = 2
 70 |   QUIT = 3
 71 | 
 72 | 
 73 | def make_game():
 74 |   """Builds and returns a game of tennnnnnnnnnnnnnnnnnnnnnnnis."""
 75 |   return ascii_art.ascii_art_to_game(
 76 |       MAZE_ART, what_lies_beneath=' ',
 77 |       sprites={
 78 |           '@': BallSprite},
 79 |       drapes={
 80 |           '1': PaddleDrape,
 81 |           '2': PaddleDrape},
 82 |       update_schedule=['1', '2', '@'])
 83 | 
 84 | 
 85 | def make_croppers():
 86 |   """Builds and returns three `ObservationCropper`s for tennnn...nnnnis."""
 87 |   return [
 88 |       # Player 1 view.
 89 |       cropping.FixedCropper(
 90 |           top_left_corner=(0, 0), rows=10, cols=10),
 91 | 
 92 |       # The ball view.
 93 |       cropping.ScrollingCropper(
 94 |           rows=10, cols=31, to_track=['@'], scroll_margins=(0, None)),
 95 | 
 96 |       # Player 2 view.
 97 |       cropping.FixedCropper(
 98 |           top_left_corner=(0, len(MAZE_ART[0])-10), rows=10, cols=10),
 99 |   ]
100 | 
101 | 
102 | class BallSprite(plab_things.Sprite):
103 |   """A Sprite that handles the ball, and also scoring and quitting."""
104 | 
105 |   def __init__(self, corner, position, character):
106 |     super(BallSprite, self).__init__(corner, position, character)
107 |     self._y_shift_modulus = 1       # Every this-many frames...
108 |     self._dy = 0                    # ...shift the ball vertically this amount.
109 |     self._dx = -1                   # Horizontally this amount in *all* frames.
110 |     self._score = np.array([0, 0])  # We keep track of score internally.
111 | 
112 |   def update(self, actions, board, layers, backdrop, things, the_plot):
113 |     row, col = self._position  # Current ball position.
114 |     reward = np.array([0, 0])  # Default reward for a game step.
115 | 
116 |     def horizontal_bounce(new_dx):  # Shared actions for horizontal bounces.
117 |       self._y_shift_modulus = random.randrange(1, 6)
118 |       self._dy = random.choice([-1, 1])
119 |       self._dx = new_dx
120 | 
121 |     # Handle vertical motion.
122 |     self._dy = {1: 1, 8: -1}.get(row, self._dy)  # Bounce off top/bottom walls!
123 |     if the_plot.frame % self._y_shift_modulus == 0: row += self._dy
124 | 
125 |     # Handle horizontal motion.
126 |     col += self._dx
127 |     # Have we hit a paddle?
128 |     if things['1'].curtain[row, col-1]:
129 |       horizontal_bounce(new_dx=1)
130 |     elif things['2'].curtain[row, col+1]:
131 |       horizontal_bounce(new_dx=-1)
132 |     # Have we hit a wall? Same as for a paddle, but awards the opponent a point.
133 |     elif layers['%'][row, col-1]:
134 |       reward = np.array([0, 1])
135 |       horizontal_bounce(new_dx=1)
136 |     elif layers['%'][row, col+1]:
137 |       reward = np.array([1, 0])
138 |       horizontal_bounce(new_dx=-1)
139 | 
140 |     # Update the position and the score.
141 |     self._position = self.Position(row=row, col=col)
142 |     the_plot.add_reward(reward)
143 |     self._score += reward
144 | 
145 |     # Finally, see if a player has won, or if a user wants to quit.
146 |     if any(self._score >= 4): the_plot.terminate_episode()
147 |     if actions is not None and actions.get('quit'): the_plot.terminate_episode()
148 | 
149 | 
150 | class PaddleDrape(plab_things.Drape):
151 |   """A Drape that handles a paddle."""
152 | 
153 |   def __init__(self, curtain, character):
154 |     """Finds out where the paddle is."""
155 |     super(PaddleDrape, self).__init__(curtain, character)
156 |     self._paddle_top = min(np.where(self.curtain)[0])
157 |     self._paddle_col = min(np.where(self.curtain)[1])
158 | 
159 |   def update(self, actions, board, layers, backdrop, things, the_plot):
160 |     # Move up or down as directed if there is room.
161 |     action = Actions.STAY if actions is None else actions[self.character]
162 |     if action == Actions.UP:
163 |       if self._paddle_top > 1: self._paddle_top -= 1
164 |     elif action == Actions.DOWN:
165 |       if self._paddle_top < 7: self._paddle_top += 1
166 | 
167 |     # Repaint the paddle. Note "blinking" effect if the ball slips past us.
168 |     self.curtain[:, self._paddle_col] = False
169 |     blink = (things['@'].position.col <= self._paddle_col   # "past" us depends
170 |              if self.character == '1' else                  # on which paddle
171 |              things['@'].position.col >= self._paddle_col)  # we are.
172 |     if not blink or (the_plot.frame % 2 == 0):
173 |       paddle_rows = np.s_[self._paddle_top:(self._paddle_top + 2)]
174 |       self.curtain[paddle_rows, self._paddle_col] = True
175 | 
176 | 
177 | def main():
178 |   # Build a game of tennnnnnnnnnnnnnnnnnnnnnnnis.
179 |   game = make_game()
180 |   # Build the croppers we'll use to make the observations.
181 |   croppers = make_croppers()
182 | 
183 |   # Make a CursesUi to play it with.
184 |   ui = human_ui.CursesUi(
185 |       # Multi-agent arguments don't have to be dicts---they can be just about
186 |       # anything; numpy arrays, scalars, nests, whatever.
187 |       keys_to_actions={
188 |           'r': {'1': Actions.UP, '2': Actions.STAY},
189 |           'f': {'1': Actions.DOWN, '2': Actions.STAY},
190 |           'u': {'1': Actions.STAY, '2': Actions.UP},
191 |           'j': {'1': Actions.STAY, '2': Actions.DOWN},
192 |           'q': {'1': Actions.STAY, '2': Actions.STAY, 'quit': True},
193 |           -1: {'1': Actions.STAY, '2': Actions.STAY},
194 |       },
195 |       delay=33, colour_fg=COLOUR_FG, colour_bg=COLOUR_BG,
196 |       croppers=croppers)
197 | 
198 |   # Let the game begin!
199 |   ui.play(game)
200 | 
201 | 
202 | if __name__ == '__main__':
203 |   main()
204 | 


--------------------------------------------------------------------------------
/pycolab/examples/warehouse_manager.py:
--------------------------------------------------------------------------------
  1 | # Copyright 2017 the pycolab Authors
  2 | #
  3 | # Licensed under the Apache License, Version 2.0 (the "License");
  4 | # you may not use this file except in compliance with the License.
  5 | # You may obtain a copy of the License at
  6 | #
  7 | #     https://www.apache.org/licenses/LICENSE-2.0
  8 | #
  9 | # Unless required by applicable law or agreed to in writing, software
 10 | # distributed under the License is distributed on an "AS IS" BASIS,
 11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 12 | # See the License for the specific language governing permissions and
 13 | # limitations under the License.
 14 | 
 15 | """A game that has absolutely nothing to do with Sokoban.
 16 | 
 17 | Command-line usage: `warehouse_manager.py <level>`, where `<level>` is an
 18 | optional integer argument selecting Warehouse Manager levels 0, 1, or 2.
 19 | 
 20 | Keys: up, down, left, right - move. q - quit.
 21 | """
 22 | 
 23 | from __future__ import absolute_import
 24 | from __future__ import division
 25 | from __future__ import print_function
 26 | 
 27 | import curses
 28 | 
 29 | import numpy as np
 30 | 
 31 | import sys
 32 | 
 33 | from pycolab import ascii_art
 34 | from pycolab import human_ui
 35 | from pycolab import rendering
 36 | from pycolab import things as plab_things
 37 | from pycolab.prefab_parts import sprites as prefab_sprites
 38 | 
 39 | 
 40 | WAREHOUSES_ART = [
 41 |     # Legend:
 42 |     #     '#': impassable walls.            '.': outdoor scenery.
 43 |     #     '_': goal locations for boxes.    'P': player starting location.
 44 |     #     '0'-'9': box starting locations.  ' ': boring old warehouse floor.
 45 | 
 46 |     ['..........',
 47 |      '..######..',     # Map #0, "Tyro"
 48 |      '..#  _ #..',
 49 |      '.##12 ##..',     # In this map, all of the sprites have the same thing
 50 |      '.#  _3 #..',     # underneath them: regular warehouse floor (' ').
 51 |      '.#_  4P#..',     # (Contrast with Map #1.) This allows us to use just a
 52 |      '.#_######.',     # single character as the what_lies_beneath argument to
 53 |      '.# # ## #.',     # ascii_art_to_game.
 54 |      '.# 5  _ #.',
 55 |      '.########.',
 56 |      '..........'],
 57 | 
 58 |     ['.............',
 59 |      '.....#######.',  # Map #1, "Pretty Easy Randomly Generated Map"
 60 |      '....##    _#.',
 61 |      '.#### ## __#.',  # This map starts one of the boxes (5) atop one of the
 62 |      '.#         #.',  # box goal locations, and since that means that there are
 63 |      '.# 1__# 2  #.',  # different kinds of things under the sprites depending
 64 |      '.# 3 ###   #.',  # on the map location, we have to use a whole separate
 65 |      '.#  45  67##.',  # ASCII art diagram for the what_lies_beneath argument to
 66 |      '.#      P #..',  # ascii_art_to_game.
 67 |      '.##########..',
 68 |      '.............'],
 69 | 
 70 |     ['.............',
 71 |      '....########.',  # Map #2, "The Open Source Release Will Be Delayed if I
 72 |      '....#  _ 1 #.',  #          Can't Think of a Name for This Map"
 73 |      '.#### 2 #  #.',
 74 |      '.#_ # 3 ## #.',  # This map also requires a full-map what_lies_beneath
 75 |      '.#   _  _#P#.',  # argument.
 76 |      '.# 45_6 _# #.',
 77 |      '.#   #78#  #.',
 78 |      '.#  _    9 #.',
 79 |      '.###########.',
 80 |      '.............'],
 81 | ]
 82 | 
 83 | 
 84 | WAREHOUSES_WHAT_LIES_BENEATH = [
 85 |     # What lies below Sprite characters in WAREHOUSES_ART?
 86 | 
 87 |     ' ',               # In map #0, ' ' lies beneath all sprites.
 88 | 
 89 |     ['.............',
 90 |      '.....#######.',  # In map #1, different characters lie beneath sprites.
 91 |      '....##    _#.',
 92 |      '.#### ## __#.',  # This ASCII art map shows an entirely sprite-free
 93 |      '.#         #.',  # rendering of Map #1, but this is mainly for human
 94 |      '.# ___#    #.',  # convenience. The ascii_art_to_game function will only
 95 |      '.#   ###   #.',  # consult cells that are "underneath" characters
 96 |      '.#   _    ##.',  # corresponding to Sprites and Drapes in the original
 97 |      '.#        #..',  # ASCII art map.
 98 |      '.##########..',
 99 |      '.............'],
100 | 
101 |     ['.............',
102 |      '....########.',     # For map #2.
103 |      '....#  _   #.',
104 |      '.####   #  #.',
105 |      '.#_ # _ ## #.',
106 |      '.#   _  _# #.',
107 |      '.#  __  _# #.',
108 |      '.#   #  #  #.',
109 |      '.#  _      #.',
110 |      '.###########.',
111 |      '.............'],
112 | ]
113 | 
114 | 
115 | # Using the digits 0-9 in the ASCII art maps is how we allow each box to be
116 | # represented with a different sprite. The only reason boxes should look
117 | # different (to humans or AIs) is when they are in a goal location vs. still
118 | # loose in the warehouse.
119 | #
120 | # Boxes in goal locations are rendered with the help of an overlying Drape that
121 | # paints X characters (see JudgeDrape), but for loose boxes, we will use a
122 | # rendering.ObservationCharacterRepainter to convert the digits to identical 'x'
123 | # characters.
124 | WAREHOUSE_REPAINT_MAPPING = {c: 'x' for c in '0123456789'}
125 | 
126 | 
127 | # These colours are only for humans to see in the CursesUi.
128 | WAREHOUSE_FG_COLOURS = {' ': (870, 838, 678),  # Warehouse floor.
129 |                         '#': (428, 135, 0),    # Warehouse walls.
130 |                         '.': (39, 208, 67),    # External scenery.
131 |                         'x': (729, 394, 51),   # Boxes loose in the warehouse.
132 |                         'X': (850, 603, 270),  # Boxes on goal positions.
133 |                         'P': (388, 400, 999),  # The player.
134 |                         '_': (834, 588, 525)}  # Box goal locations.
135 | 
136 | WAREHOUSE_BG_COLOURS = {'X': (729, 394, 51)}   # Boxes on goal positions.
137 | 
138 | 
139 | def make_game(level):
140 |   """Builds and returns a Warehouse Manager game for the selected level."""
141 |   warehouse_art = WAREHOUSES_ART[level]
142 |   what_lies_beneath = WAREHOUSES_WHAT_LIES_BENEATH[level]
143 | 
144 |   # Create a Sprite for every box in the game ASCII-art.
145 |   sprites = {c: BoxSprite for c in '1234567890' if c in ''.join(warehouse_art)}
146 |   sprites['P'] = PlayerSprite
147 |   # We also have a "Judge" drape that marks all boxes that are in goal
148 |   # locations, and that holds the game logic for determining if the player has
149 |   # won the game.
150 |   drapes = {'X': JudgeDrape}
151 | 
152 |   # This update schedule simplifies the game logic considerably. The boxes
153 |   # move first, and they only move if there is already a Player next to them
154 |   # to push in the same direction as the action. (This condition can only be
155 |   # satisfied by one box at a time.
156 |   #
157 |   # The Judge runs next, and handles various adminstrative tasks: scorekeeping,
158 |   # deciding whether the player has won, and listening for the 'q'(uit) key. If
159 |   # neither happen, the Judge draws Xs over all boxes that are in a goal
160 |   # position. The Judge runs in its own update group so that it can detect a
161 |   # winning box configuration the instant it is made---and so it can clean up
162 |   # any out-of-date X marks in time for the Player to move into the place where
163 |   # they used to be.
164 |   #
165 |   # The Player moves last, and by the time they try to move into the spot where
166 |   # the box they were pushing used to be, the box (and any overlying drape) will
167 |   # have moved out of the way---since it's in a third update group (see `Engine`
168 |   # docstring).
169 |   update_schedule = [[c for c in '1234567890' if c in ''.join(warehouse_art)],
170 |                      ['X'],
171 |                      ['P']]
172 | 
173 |   # We are also relying on the z order matching a depth-first traversal of the
174 |   # update schedule by default---that way, the JudgeDrape gets to make its mark
175 |   # on top of all of the boxes.
176 |   return ascii_art.ascii_art_to_game(
177 |       warehouse_art, what_lies_beneath, sprites, drapes,
178 |       update_schedule=update_schedule)
179 | 
180 | 
181 | class BoxSprite(prefab_sprites.MazeWalker):
182 |   """A `Sprite` for boxes in our warehouse.
183 | 
184 |   These boxes listen for motion actions, but it only obeys them if a
185 |   PlayerSprite happens to be in the right place to "push" the box, and only if
186 |   there's no obstruction in the way. A `BoxSprite` corresponding to the digit
187 |   `2` can go left in this circumstance, for example:
188 | 
189 |       .......
190 |       .#####.
191 |       .#   #.
192 |       .# 2P#.
193 |       .#####.
194 |       .......
195 | 
196 |   but in none of these circumstances:
197 | 
198 |       .......     .......     .......
199 |       .#####.     .#####.     .#####.
200 |       .#   #.     .#P  #.     .#   #.
201 |       .#P2 #.     .# 2 #.     .##2P#.
202 |       .#####.     .#####.     .#####.
203 |       .......     .......     .......
204 | 
205 |   The update schedule we selected in `make_game` will ensure that the player
206 |   will soon "catch up" to the box they have pushed.
207 |   """
208 | 
209 |   def __init__(self, corner, position, character):
210 |     """Constructor: simply supplies characters that boxes can't traverse."""
211 |     impassable = set('#.0123456789PX') - set(character)
212 |     super(BoxSprite, self).__init__(corner, position, character, impassable)
213 | 
214 |   def update(self, actions, board, layers, backdrop, things, the_plot):
215 |     del backdrop, things  # Unused.
216 | 
217 |     # Implements the logic described in the class docstring.
218 |     rows, cols = self.position
219 |     if actions == 0:    # go upward?
220 |       if layers['P'][rows+1, cols]: self._north(board, the_plot)
221 |     elif actions == 1:  # go downward?
222 |       if layers['P'][rows-1, cols]: self._south(board, the_plot)
223 |     elif actions == 2:  # go leftward?
224 |       if layers['P'][rows, cols+1]: self._west(board, the_plot)
225 |     elif actions == 3:  # go rightward?
226 |       if layers['P'][rows, cols-1]: self._east(board, the_plot)
227 | 
228 | 
229 | class JudgeDrape(plab_things.Drape):
230 |   """A `Drape` that marks boxes atop goals, and also decides whether you've won.
231 | 
232 |   This `Drape` sits atop all of the box `Sprite`s and provides a "luxury"
233 |   Sokoban feature: if one of the boxes is sitting on one of the goal states, it
234 |   marks the box differently from others that are loose in the warehouse.
235 | 
236 |   While doing so, the `JudgeDrape` also counts the number of boxes on goal
237 |   states, and uses this information to update the game score and to decide
238 |   whether the game has finished.
239 |   """
240 | 
241 |   def __init__(self, curtain, character):
242 |     super(JudgeDrape, self).__init__(curtain, character)
243 |     self._last_num_boxes_on_goals = 0
244 | 
245 |   def update(self, actions, board, layers, backdrop, things, the_plot):
246 |     # Clear our curtain and mark the locations of all the boxes True.
247 |     self.curtain.fill(False)
248 |     for box_char in (c for c in '0123456789' if c in layers):
249 |       self.curtain[things[box_char].position] = True
250 |     # We can count the number of boxes we have now:
251 |     num_boxes = np.sum(self.curtain)
252 |     # Now logically-and the box locations with the goal locations. These are
253 |     # all of the goals that are occupied by boxes at the moment.
254 |     np.logical_and(self.curtain, (backdrop.curtain == backdrop.palette._),
255 |                    out=self.curtain)
256 | 
257 |     # Compute the reward to credit to the player: the change in how many goals
258 |     # are occupied by boxes at the moment.
259 |     num_boxes_on_goals = np.sum(self.curtain)
260 |     the_plot.add_reward(num_boxes_on_goals - self._last_num_boxes_on_goals)
261 |     self._last_num_boxes_on_goals = num_boxes_on_goals
262 | 
263 |     # See if we should quit: it happens if the user solves the puzzle or if
264 |     # they give up and execute the 'quit' action.
265 |     if (actions == 5) or (num_boxes_on_goals == num_boxes):
266 |       the_plot.terminate_episode()
267 | 
268 | 
269 | class PlayerSprite(prefab_sprites.MazeWalker):
270 |   """A `Sprite` for our player, the Warehouse Manager.
271 | 
272 |   This `Sprite` requires no logic beyond tying actions to `MazeWalker`
273 |   motion action helper methods, which keep the player from walking on top of
274 |   obstacles. If the player has pushed a box, then the update schedule has
275 |   already made certain that the box is out of the way (along with any
276 |   overlying characters from the `JudgeDrape`) by the time the `PlayerSprite`
277 |   gets to move.
278 |   """
279 | 
280 |   def __init__(self, corner, position, character):
281 |     """Constructor: simply supplies characters that players can't traverse."""
282 |     super(PlayerSprite, self).__init__(
283 |         corner, position, character, impassable='#.0123456789X')
284 | 
285 |   def update(self, actions, board, layers, backdrop, things, the_plot):
286 |     del backdrop, things, layers  # Unused.
287 | 
288 |     if actions == 0:    # go upward?
289 |       self._north(board, the_plot)
290 |     elif actions == 1:  # go downward?
291 |       self._south(board, the_plot)
292 |     elif actions == 2:  # go leftward?
293 |       self._west(board, the_plot)
294 |     elif actions == 3:  # go rightward?
295 |       self._east(board, the_plot)
296 | 
297 | 
298 | def main(argv=()):
299 |   # Build a Warehouse Manager game.
300 |   game = make_game(int(argv[1]) if len(argv) > 1 else 0)
301 | 
302 |   # Build an ObservationCharacterRepainter that will make all of the boxes in
303 |   # the warehouse look the same.
304 |   repainter = rendering.ObservationCharacterRepainter(WAREHOUSE_REPAINT_MAPPING)
305 | 
306 |   # Make a CursesUi to play it with.
307 |   ui = human_ui.CursesUi(
308 |       keys_to_actions={curses.KEY_UP: 0, curses.KEY_DOWN: 1,
309 |                        curses.KEY_LEFT: 2, curses.KEY_RIGHT: 3,
310 |                        -1: 4,
311 |                        'q': 5, 'Q': 5},
312 |       repainter=repainter, delay=100,
313 |       colour_fg=WAREHOUSE_FG_COLOURS,
314 |       colour_bg=WAREHOUSE_BG_COLOURS)
315 | 
316 |   # Let the game begin!
317 |   ui.play(game)
318 | 
319 | 
320 | if __name__ == '__main__':
321 |   main(sys.argv)
322 | 


--------------------------------------------------------------------------------
/pycolab/plot.py:
--------------------------------------------------------------------------------
  1 | # Copyright 2018 the pycolab Authors
  2 | #
  3 | # Licensed under the Apache License, Version 2.0 (the "License");
  4 | # you may not use this file except in compliance with the License.
  5 | # You may obtain a copy of the License at
  6 | #
  7 | #     https://www.apache.org/licenses/LICENSE-2.0
  8 | #
  9 | # Unless required by applicable law or agreed to in writing, software
 10 | # distributed under the License is distributed on an "AS IS" BASIS,
 11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 12 | # See the License for the specific language governing permissions and
 13 | # limitations under the License.
 14 | 
 15 | """The pycolab "Plot" blackboard.
 16 | 
 17 | All details are in the docstring for `Plot`.
 18 | """
 19 | 
 20 | from __future__ import absolute_import
 21 | from __future__ import division
 22 | from __future__ import print_function
 23 | 
 24 | from pycolab.protocols import logging as plab_logging
 25 | 
 26 | 
 27 | class Plot(dict):
 28 |   """The pycolab "plot" object---a blackboard for communication.
 29 | 
 30 |   By design, a game's `Engine`, its `Backdrop`, and its `Sprite`s and `Drape`s
 31 |   are not really meant to talk to each other directly; instead, they leave
 32 |   messages for each other in the game's `Plot` object. With the exception of
 33 |   messages to the `Engine`, these messages are free-form and persistent; in
 34 |   fact, let's drop the pretense---a `Plot` is really just a `dict` with some
 35 |   extra methods and properties, and to talk to each other, these entities should
 36 |   just modify the dict in the usual ways, adding, removing, and modyfing
 37 |   whatever entries they like. (Responsibly, of course.)
 38 | 
 39 |   (Note: in a limited way, the `Backdrop` and `Sprite`s and `Drape`s are allowed
 40 |   to inspect each other, but these inspections should be limited to small,
 41 |   read-only interactions via public methods and attributes. This capability is
 42 |   only really there to simplify things like getting the row/col coordinates of
 43 |   other sprites. Try not to abuse it!)
 44 | 
 45 |   (Note 2: Game developers who are designing ways for `Sprite`s, `Drape`s and
 46 |   the `Backdrop` to communicate with each other via the `Plot` object might find
 47 |   the `setdefault` method on `dict` to be especially useful. Or not.)
 48 | 
 49 |   The messages to the Engine have more structure. A `Backdrop`, a `Sprite`, or
 50 |   a `Drape` can direct the engine to do any of the following:
 51 | 
 52 |   * Change the z-ordering of the `Sprite`s and `Drape`s.
 53 |   * Add some quantity to the reward being returned to the player (or player)
 54 |     in response to their action.
 55 |   * Terminate the game.
 56 | 
 57 |   A dedicated method exists within the `Plot` object for all of these actions.
 58 | 
 59 |   The `Plot` object also has an imporant role for games that participate in
 60 |   `Story`s: values in the `Plot` persist between games; in fact, they are the
 61 |   only data guaranteed to do so. Individual games that participate in a `Story`
 62 |   also use the `Plot` to control what happens next after they terminate.
 63 | 
 64 |   Lastly, the `Plot` object contains a number of public attributes that the
 65 |   `Engine` can use to communicate various statistics about the game to the
 66 |   various other game entities.
 67 |   """
 68 | 
 69 |   class _EngineDirectives(object):
 70 |     """A container for instructions for modifying an `Engine`'s internal state.
 71 | 
 72 |     External code is not meant to manipulate `_EngineDirectives` objects
 73 |     directly, but `Engine` and `Plot` objects can do it. Properties of this
 74 |     class include:
 75 | 
 76 |     * `z_updates`: an indexable of 2-tuples `(c1, c2)`, whose semantics are
 77 |       "move the `Sprite` or `Drape` that paints with character `c1` in front of
 78 |       the `Sprite` or `Drape` that paints with character `c2`." If `c2` is
 79 |       None, the `Sprite` or `Drape` is moved all the way to the back of the
 80 |       z-order (i.e. behind everything except the `Backdrop`).
 81 |     * `summed_reward`: None, if there is no reward to be reported to the player
 82 |       (or players) during a game iteration. Otherwise, this can be any value
 83 |       appropriate to the game. If there's ever any chance that more than one
 84 |       entity (`Sprite`, `Drape`, or `Backdrop`) would supply a reward during a
 85 |       game iteration, the value should probably be of a type that supports the
 86 |       `+` operator in a relevant way.
 87 |     * `game_over`: a boolean value which... is True until the game is over.
 88 |     * `discount`: reinforcement learning discount factor to report to the
 89 |       player during a game iteration; typically a fixed value until the end of
 90 |       the game is reached, then 0.
 91 |     """
 92 |     # For fast access
 93 |     __slots__ = ('z_updates', 'summed_reward', 'game_over', 'discount')
 94 | 
 95 |     def __init__(self):
 96 |       """Construct a no-op `_EngineDirectives`.
 97 | 
 98 |       Builds an `_EngineDirectives` that indicate that the `Engine` should not
 99 |       change any state.
100 |       """
101 |       self.z_updates = []
102 |       self.summed_reward = None
103 |       self.game_over = False
104 |       self.discount = 1.0
105 | 
106 |   def __init__(self):
107 |     """Construct a new `Plot` object."""
108 |     super(Plot, self).__init__()
109 | 
110 |     # The current frame actually starts at -1, but before any of the
111 |     # non-`Engine` entities see it, it will have been incremented to 0 for the
112 |     # first frame.
113 |     self._frame = -1
114 | 
115 |     # Will hold the current update group when the `update` methods of `Sprite`s
116 |     # and `Drape`s are called.
117 |     self._update_group = None
118 | 
119 |     # For Storys only: holds keys or indices indicating which game preceded
120 |     # the current game, which game is the current game, and which game should
121 |     # be started next after the current game terminates, respectively. A None
122 |     # value for _next_chapter means that the story should terminate after the
123 |     # current game ends.
124 |     #
125 |     # These members are not used if a Story is not underway.
126 |     self._prior_chapter = None
127 |     self._this_chapter = None
128 |     self._next_chapter = None
129 | 
130 |     # Set an initial set of engine directives (at self._engine_directives),
131 |     # which basically amount to telling the Engine do nothing.
132 |     self._clear_engine_directives()
133 | 
134 |   ### Public methods for changing global game state. ###
135 | 
136 |   def change_z_order(self, move_this, in_front_of_that):
137 |     """Tell an `Engine` to change the z-order of some `Sprite`s and/or `Drape`s.
138 | 
139 |     During a game iteration, the `Backdrop` and each `Sprite` or `Drape` has
140 |     an opportunity to call this method as often as it wants. Each call indicates
141 |     that one `Sprite` or `Drape` should appear in front of another---or in
142 |     front of no other if it should be moved all the way to the back of the
143 |     z-order, behind everything else except for the `Backdrop`.
144 | 
145 |     These requests are processed at each game iteration after the `Engine` has
146 |     consulted the `Backdrop` and all `Sprite`s and `Layer`s for updates, but
147 |     before the finished observation to be shown to the player (or players) is
148 |     rendered. The requests are processed in the order they are made to the
149 |     `Plot` object, so this ordering of requests:
150 | 
151 |          '#' in front of '$'
152 |          '$' in front of '%'
153 | 
154 |     could result in a different `Sprite` or `Drape` being foremost from the
155 |     reverse ordering.
156 | 
157 |     Args:
158 |       move_this: the character corresponding to the `Sprite` or `Drape` to move
159 |           in the z-order.
160 |       in_front_of_that: the character corresponding to the `Sprite` or `Drape`
161 |           that the moving entity should move in front of, or None if the moving
162 |           entity should go all the way to the back (just in front of the
163 |           `Backdrop`).
164 | 
165 |     Raises:
166 |       ValueError: if `move_this` or `in_front_of_that` are not both single ASCII
167 |           characters.
168 |     """
169 |     self._value_error_if_character_is_bad(move_this)
170 |     if in_front_of_that is not None:
171 |       self._value_error_if_character_is_bad(in_front_of_that)
172 | 
173 |     # Construct a new set of engine directives with updated z-update directives.
174 |     self._engine_directives.z_updates.append((move_this, in_front_of_that))
175 | 
176 |   def terminate_episode(self, discount=0.0):
177 |     """Tell an `Engine` to terminate the current episode.
178 | 
179 |     During a game iteration, any `Backdrop`, `Sprite`, or `Drape` can call this
180 |     method. Once the `Engine` has finished consulting these entities for
181 |     updates, it will mark the episode as complete, render the final observation,
182 |     and return it to the player (or players).
183 | 
184 |     Args:
185 |       discount: reinforcement learning discount factor to associate with this
186 |           episode termination; must be in the range [0, 1]. Ordinary episode
187 |           terminations should use the default value 0.0; rarely, some
188 |           environments may use different values to mark interruptions or other
189 |           abnormal termination conditions.
190 | 
191 |     Raises:
192 |       ValueError: if `discount` is not in the [0,1] range.
193 |     """
194 |     if not 0.0 <= discount <= 1.0:
195 |       raise ValueError('Discount must be in range [0,1].')
196 | 
197 |     # Construct a new set of engine directives with the death warrant signed.
198 |     self._engine_directives.game_over = True
199 |     self._engine_directives.discount = discount
200 | 
201 |   def add_reward(self, reward):
202 |     """Add a value to the reward the `Engine` will return to the player(s).
203 | 
204 |     During a game iteration, any `Backdrop`, `Sprite`, or `Drape` can call this
205 |     method to add value to the reward that the `Engine` will return to the
206 |     player (or players) for having taken the action (or actions) supplied in the
207 |     `actions` argument to `Engine`'s `play` method.
208 | 
209 |     This value need not be a number, but can be any kind of value appropriate to
210 |     the game.  If there's ever any chance that more than one `Sprite`, `Drape`,
211 |     or `Backdrop` would supply a reward during a game iteration, the value
212 |     should probably be of a type that supports the `+=` operator in a relevant
213 |     way, since this method uses addition to accumulate reward. (For custom
214 |     classes, this typically means implementing the `__iadd__` method.)
215 | 
216 |     If this method is never called during a game iteration, the `Engine` will
217 |     supply None to the player (or players) as the reward.
218 | 
219 |     Args:
220 |       reward: reward value to accumulate into the current game iteration's
221 |           reward for the player(s). See discussion for details.
222 |     """
223 |     if self._engine_directives.summed_reward is None:
224 |       self._engine_directives.summed_reward = reward
225 |     else:
226 |       self._engine_directives.summed_reward += reward
227 | 
228 |   def log(self, message):
229 |     """Log a message for eventual disposal by the game engine user.
230 | 
231 |     Here, "game engine user" means a user interface or an environment interface,
232 |     which may eventually display the message to an actual human being in a
233 |     useful way (writing it to a file, showing it in a game console, etc.).
234 | 
235 |     **Calling this method doesn't mean that a log message will appear in the
236 |     process logs.** It's up to your program to collect your logged messages from
237 |     the `Plot` object and dispose of them appropriately.
238 | 
239 |     See `protocols/logging.py` for more details. (This function is sugar for the
240 |     `log` function in that module.)
241 | 
242 |     Args:
243 |       message: A string message to convey to the game engine user.
244 |     """
245 |     plab_logging.log(self, message)
246 | 
247 |   def change_default_discount(self, discount):
248 |     """Change the discount reported by the `Engine` for non-terminal steps.
249 | 
250 |     During a game iteration, the `Backdrop` and each `Sprite` or `Drape` have an
251 |     opportunity to call this method (but don't have to). The last one to call
252 |     will determine the new reinforcement learning discount factor that will be
253 |     supplied to the player at every non-terminal step (until this method is
254 |     called again).
255 | 
256 |     Even for the same game, discounts often need to be different for different
257 |     agent architectures, so conventional approaches to setting a fixed
258 |     non-terminal discount factor include building a discount multiplier into
259 |     your agent or using some kind of wrapper that intercepts and changes
260 |     discounts before the agent sees them. This method here is mainly reserved
261 |     for rare settings where those approaches would not be suitable. Most games
262 |     will not need to use it.
263 | 
264 |     Args:
265 |       discount: New value of discount in the range [0,1].
266 | 
267 |     Raises:
268 |       ValueError: if `discount` is not in the [0,1] range.
269 |     """
270 |     if not 0.0 <= discount <= 1.0:
271 |       raise ValueError('Default discount must be in range [0,1].')
272 |     self._engine_directives.discount = discount
273 | 
274 |   @property
275 |   def frame(self):
276 |     """Counts game iterations, with the first iteration starting at 0."""
277 |     return self._frame
278 | 
279 |   @property
280 |   def update_group(self):
281 |     """The current update group being consulted by the `Engine`."""
282 |     return self._update_group
283 | 
284 |   @property
285 |   def default_discount(self):
286 |     """The current non-terminal discount factor used by the `Engine`."""
287 |     return self._engine_directives.discount
288 | 
289 |   ### Public properties for global story state. ###
290 | 
291 |   @property
292 |   def prior_chapter(self):
293 |     """Key/index for the prior game in a `Story`, or None for no prior game."""
294 |     return self._prior_chapter
295 | 
296 |   @property
297 |   def this_chapter(self):
298 |     """Key/index for the current game in a `Story`."""
299 |     return self._this_chapter
300 | 
301 |   @property
302 |   def next_chapter(self):
303 |     """Key/index for the next game in a `Story`, or None for no next game."""
304 |     return self._next_chapter
305 | 
306 |   @next_chapter.setter
307 |   def next_chapter(self, next_chapter):
308 |     """Indicate which game should appear next in the current `Story`.
309 | 
310 |     If the current game is running as part of a `Story`, and if that `Story` was
311 |     initialised with a dict as the `chapters` constructor argument, this method
312 |     allows game entities to indicate which entry in the `chapters` dict holds
313 |     the next game that the `Story` should run after the current game terminates.
314 |     Or, if called with a `None` argument, this method directs the `Story` to
315 |     report termination to the player after the current game terminates. Either
316 |     way, the last call to this method before termination is the one that
317 |     determines what actually happens.
318 | 
319 |     This method only does something meaningful if a `Story` is actually underway
320 |     and if the `Story`'s `chapters` constructor argument was a dict; otherwise
321 |     it has no effect whatsoever.
322 | 
323 |     Args:
324 |       next_chapter: A key into the dict passed as the `chapters` argument to the
325 |           `Story` constructor, or None. No checking is done against `chapters`
326 |           to ensure that this argument is a valid key.
327 |     """
328 |     self._next_chapter = next_chapter
329 | 
330 |   ### Setters and other helpers for Engine ###
331 | 
332 |   @frame.setter
333 |   def frame(self, val):
334 |     """Update game iterations. Only `Engine` and tests may use this setter."""
335 |     assert val == self._frame + 1  # Frames increase one-by-one.
336 |     self._frame = val
337 | 
338 |   @update_group.setter
339 |   def update_group(self, group):
340 |     """Set the current update group. Only `Engine` and tests may do this."""
341 |     self._update_group = group
342 | 
343 |   def _clear_engine_directives(self):
344 |     """Reset this `Plot`'s set of directives to the `Engine`.
345 | 
346 |     The reset directives essentially tell the `Engine` to make no changes to
347 |     its internal state. The `Engine` will typically call this method at the
348 |     end of every game iteration, once all of the existing directives have been
349 |     consumed.
350 | 
351 |     Only `Engine` and `Plot` methods may call this method.
352 |     """
353 |     self._engine_directives = self._EngineDirectives()
354 | 
355 |   def _get_engine_directives(self):
356 |     """Accessor for this `Plot`'s set of directives to the `Engine`.
357 | 
358 |     Only `Engine` and `Plot` methods may call this method.
359 | 
360 |     Returns:
361 |       This `Plot`'s set of directions to the `Engine`.
362 |     """
363 |     return self._engine_directives
364 | 
365 |   ### Setters and other helpers for Story ###
366 | 
367 |   @prior_chapter.setter
368 |   def prior_chapter(self, val):
369 |     """Update last chapter. Only `Story` and tests may use this setter."""
370 |     self._prior_chapter = val
371 | 
372 |   @this_chapter.setter
373 |   def this_chapter(self, val):
374 |     """Update current chapter. Only `Story` and tests may use this setter."""
375 |     self._this_chapter = val
376 | 
377 |   ### Private helpers for error detection ###
378 | 
379 |   def _value_error_if_character_is_bad(self, character):
380 |     try:
381 |       ord(character)
382 |     except TypeError:
383 |       raise ValueError(
384 |           '{} was used as an argument in a call to change_z_order, but only '
385 |           'single ASCII characters are valid arguments'.format(repr(character)))
386 | 


--------------------------------------------------------------------------------
/pycolab/prefab_parts/__init__.py:
--------------------------------------------------------------------------------
 1 | # Copyright 2017 the pycolab Authors
 2 | #
 3 | # Licensed under the Apache License, Version 2.0 (the "License");
 4 | # you may not use this file except in compliance with the License.
 5 | # You may obtain a copy of the License at
 6 | #
 7 | #     https://www.apache.org/licenses/LICENSE-2.0
 8 | #
 9 | # Unless required by applicable law or agreed to in writing, software
10 | # distributed under the License is distributed on an "AS IS" BASIS,
11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12 | # See the License for the specific language governing permissions and
13 | # limitations under the License.
14 | 


--------------------------------------------------------------------------------
/pycolab/protocols/__init__.py:
--------------------------------------------------------------------------------
 1 | # Copyright 2017 the pycolab Authors
 2 | #
 3 | # Licensed under the Apache License, Version 2.0 (the "License");
 4 | # you may not use this file except in compliance with the License.
 5 | # You may obtain a copy of the License at
 6 | #
 7 | #     https://www.apache.org/licenses/LICENSE-2.0
 8 | #
 9 | # Unless required by applicable law or agreed to in writing, software
10 | # distributed under the License is distributed on an "AS IS" BASIS,
11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12 | # See the License for the specific language governing permissions and
13 | # limitations under the License.
14 | 


--------------------------------------------------------------------------------
/pycolab/protocols/logging.py:
--------------------------------------------------------------------------------
 1 | # Copyright 2017 the pycolab Authors
 2 | #
 3 | # Licensed under the Apache License, Version 2.0 (the "License");
 4 | # you may not use this file except in compliance with the License.
 5 | # You may obtain a copy of the License at
 6 | #
 7 | #     https://www.apache.org/licenses/LICENSE-2.0
 8 | #
 9 | # Unless required by applicable law or agreed to in writing, software
10 | # distributed under the License is distributed on an "AS IS" BASIS,
11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12 | # See the License for the specific language governing permissions and
13 | # limitations under the License.
14 | 
15 | """Simple message logging for pycolab game entities.
16 | 
17 | The interactive nature of pycolab games makes it difficult to do useful things
18 | like "printf debugging"---if you're using an interface like the Curses UI, you
19 | won't be able to see printed strings. This protocol allows game entities to log
20 | messages to the Plot object. User interfaces can query this object and display
21 | accumulated messages to the user in whatever way is best.
22 | 
23 | Most game implementations will not need to import this protocol directly---
24 | logging is so fundamental that the Plot object expresses a `log` method that's
25 | syntactic sugar for the log function in this file.
26 | """
27 | 
28 | from __future__ import absolute_import
29 | from __future__ import division
30 | from __future__ import print_function
31 | 
32 | 
33 | def log(the_plot, message):
34 |   """Log a message for eventual disposal by the game engine user.
35 | 
36 |   Here, "game engine user" means a user interface or an environment interface,
37 |   for example. (Clients are not required to deal with messages, but if they do,
38 |   this is how to get a message to them.)
39 | 
40 |   Most game implementations will not need to call this function directly---
41 |   logging is so fundamental that the Plot object expresses a `log` method that's
42 |   syntactic sugar for this function.
43 | 
44 |   Args:
45 |     the_plot: the pycolab game's `Plot` object.
46 |     message: A string message to convey to the game engine user.
47 |   """
48 |   the_plot.setdefault('log_messages', []).append(message)
49 | 
50 | 
51 | def consume(the_plot):
52 |   """Obtain messages logged by game entities since the last call to `consume`.
53 | 
54 |   This function is meant to be called by "game engine users" (user interfaces,
55 |   environment interfaces, etc.) to obtain the latest set of log messages
56 |   emitted by the game entities. These systems can then dispose of these messages
57 |   in whatever manner is the most appropriate.
58 | 
59 |   Args:
60 |     the_plot: the pycolab game's `Plot` object.
61 | 
62 |   Returns:
63 |     The list of all log messages supplied by the `log` method since the last
64 |     time `consume` was called (or ever, if `consume` has never been called).
65 |   """
66 |   messages = the_plot.setdefault('log_messages', [])
67 |   # Hand off the current messages to a new list that we return.
68 |   our_messages = messages[:]
69 |   del messages[:]
70 |   return our_messages
71 | 


--------------------------------------------------------------------------------
/pycolab/tests/__init__.py:
--------------------------------------------------------------------------------
 1 | # Copyright 2017 the pycolab Authors
 2 | #
 3 | # Licensed under the Apache License, Version 2.0 (the "License");
 4 | # you may not use this file except in compliance with the License.
 5 | # You may obtain a copy of the License at
 6 | #
 7 | #     https://www.apache.org/licenses/LICENSE-2.0
 8 | #
 9 | # Unless required by applicable law or agreed to in writing, software
10 | # distributed under the License is distributed on an "AS IS" BASIS,
11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12 | # See the License for the specific language governing permissions and
13 | # limitations under the License.
14 | 


--------------------------------------------------------------------------------
/pycolab/tests/ascii_art_test.py:
--------------------------------------------------------------------------------
 1 | # coding=utf8
 2 | 
 3 | # Copyright 2017 the pycolab Authors
 4 | #
 5 | # Licensed under the Apache License, Version 2.0 (the "License");
 6 | # you may not use this file except in compliance with the License.
 7 | # You may obtain a copy of the License at
 8 | #
 9 | #     https://www.apache.org/licenses/LICENSE-2.0
10 | #
11 | # Unless required by applicable law or agreed to in writing, software
12 | # distributed under the License is distributed on an "AS IS" BASIS,
13 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
14 | # See the License for the specific language governing permissions and
15 | # limitations under the License.
16 | 
17 | """Basic tests for the ascii_art module.
18 | """
19 | 
20 | from __future__ import absolute_import
21 | from __future__ import division
22 | from __future__ import print_function
23 | 
24 | import sys
25 | import unittest
26 | 
27 | from pycolab import ascii_art
28 | 
29 | import six
30 | 
31 | 
32 | class AsciiArtTest(unittest.TestCase):
33 | 
34 |   def testRaiseErrors(self):
35 |     """Checks how `ascii_art_to_uint8_nparray` handles incorrect input."""
36 |     # Correct input.
37 |     art = ['ab', 'ba']
38 |     _ = ascii_art.ascii_art_to_uint8_nparray(art)
39 | 
40 |     # Incorrect input: not all the same length.
41 |     art = ['ab', 'bab']
42 |     with six.assertRaisesRegex(
43 |         self,
44 |         ValueError, 'except for the concatenation axis must match exactly'):
45 |       _ = ascii_art.ascii_art_to_uint8_nparray(art)
46 | 
47 |     # Incorrect input: not all strings.
48 |     art = ['a', 2]
49 |     with six.assertRaisesRegex(
50 |         self,
51 |         TypeError, 'the argument to ascii_art_to_uint8_nparray must be a list'):
52 |       _ = ascii_art.ascii_art_to_uint8_nparray(art)
53 | 
54 |     # Incorrect input: list of list (special case of the above).
55 |     art = [['a', 'b'], ['b', 'a']]
56 |     with six.assertRaisesRegex(
57 |         self,
58 |         TypeError, 'Did you pass a list of list of single characters?'):
59 |       _ = ascii_art.ascii_art_to_uint8_nparray(art)
60 | 
61 | 
62 | def main(argv=()):
63 |   del argv  # Unused.
64 |   unittest.main()
65 | 
66 | 
67 | if __name__ == '__main__':
68 |   main(sys.argv)
69 | 


--------------------------------------------------------------------------------
/pycolab/tests/story_test.py:
--------------------------------------------------------------------------------
  1 | # Copyright 2018 the pycolab Authors
  2 | #
  3 | # Licensed under the Apache License, Version 2.0 (the "License");
  4 | # you may not use this file except in compliance with the License.
  5 | # You may obtain a copy of the License at
  6 | #
  7 | #     https://www.apache.org/licenses/LICENSE-2.0
  8 | #
  9 | # Unless required by applicable law or agreed to in writing, software
 10 | # distributed under the License is distributed on an "AS IS" BASIS,
 11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 12 | # See the License for the specific language governing permissions and
 13 | # limitations under the License.
 14 | 
 15 | """Tests of the "Story" framework for making big games out of smaller ones."""
 16 | 
 17 | from __future__ import absolute_import
 18 | from __future__ import division
 19 | from __future__ import print_function
 20 | 
 21 | import sys
 22 | import unittest
 23 | 
 24 | from pycolab import ascii_art
 25 | from pycolab import cropping
 26 | from pycolab import storytelling
 27 | from pycolab import things as plab_things
 28 | from pycolab.prefab_parts import sprites as prefab_sprites
 29 | from pycolab.tests import test_things as tt
 30 | 
 31 | import six
 32 | 
 33 | 
 34 | class StoryTest(tt.PycolabTestCase):
 35 | 
 36 |   ### Helpers ###
 37 | 
 38 |   class _DieRightward(prefab_sprites.MazeWalker):
 39 |     """A boring Sprite that pauses, walks one step rightward, and terminates.
 40 | 
 41 |     Some additional functions are present to support other tests:
 42 |     - If the_plot['chapter_names'] is a list, then when this Sprite terminates
 43 |       a game, it will pop that list and assign the resulting value to
 44 |       the_plot.next_chapter, directing which chapter should follow.
 45 |     """
 46 | 
 47 |     def __init__(self, corner, position, character):
 48 |       super(StoryTest._DieRightward, self).__init__(
 49 |           corner, position, character, impassable='', confined_to_board=True)
 50 | 
 51 |     def update(self, actions, board, layers, backdrop, things, the_plot):
 52 |       if the_plot.frame == 1:
 53 |         self._east(board, the_plot)
 54 |         the_plot.terminate_episode()
 55 |         if 'chapter_names' in the_plot:
 56 |           the_plot.next_chapter = the_plot['chapter_names'].pop(0)
 57 | 
 58 |   class _IdleDrape(plab_things.Drape):
 59 |     """An even more boring Drape that does nothing at all."""
 60 | 
 61 |     def update(self, *args, **kwargs):
 62 |       pass
 63 | 
 64 |   def _make_game(self, art):
 65 |     """Make a game with one _DieRightward sprite. Valid chars are [P.]."""
 66 |     return ascii_art.ascii_art_to_game(
 67 |         art, what_lies_beneath='.', sprites={'P': self._DieRightward})
 68 | 
 69 |   ### Tests ###
 70 | 
 71 |   def testSequenceOfGames(self):
 72 |     """A `Story` will play through a sequence of games automatically."""
 73 | 
 74 |     # The four games in the sequence use these four worlds.
 75 |     arts = list(zip(['P..', '...', '...', '...'],
 76 |                     ['...', '...', 'P..', '...'],
 77 |                     ['...', 'P..', '...', '.P.']))
 78 | 
 79 |     # Create and start the story.
 80 |     story = storytelling.Story(
 81 |         [lambda art=a: self._make_game(art) for a in arts])
 82 |     observation, reward, pcontinue = story.its_showtime()
 83 |     del reward, pcontinue  # unused
 84 | 
 85 |     # The first frame is the first game's first frame.
 86 |     self.assertBoard(observation.board, arts[0])
 87 | 
 88 |     # This should set us up to get the following sequence of observations.
 89 |     self.assertMachinima(
 90 |         engine=story,
 91 |         frames=[
 92 |             # The second frame is the second game's first frame. The terminal
 93 |             # frame from the first game is discarded, so we never get to see
 94 |             # the Sprite move rightward. (None was the action just prior to the
 95 |             # second frame. Actions are not used in this test.)
 96 |             (None, arts[1]),
 97 | 
 98 |             # The third frame is the third game's first frame. Same reasoning.
 99 |             (None, arts[2]),
100 | 
101 |             # The fourth frame is the fourth game's first frame.
102 |             (None, arts[3]),
103 | 
104 |             # The fifth frame is the fourth game's last frame. In a Story, you
105 |             # always see the terminal frame of the very last game (and you see
106 |             # no other game's terminal frame).
107 |             (None, ['...',
108 |                     '...',
109 |                     '..P'])
110 |         ],
111 |     )
112 | 
113 |   def testDictOfGames(self):
114 |     """A `Story` will run as directed through a dict of games."""
115 | 
116 |     # The two games in this dict will use these two worlds.
117 |     arts = {'one': ['P..',
118 |                     '...',
119 |                     '...'],
120 |             'two': ['...',
121 |                     '.P.',
122 |                     '...']}
123 | 
124 |     # Create and start the story. Note how we inject a value into the first
125 |     # game's Plot that _DieRightward uses to tell Story how to advance through
126 |     # the various subgames. A bit roundabout, but it has the side-effect of also
127 |     # testing that Plot contents persist between games.
128 |     story = storytelling.Story(
129 |         {k: lambda art=v: self._make_game(art) for k, v in arts.items()},
130 |         first_chapter='one')
131 |     story.current_game.the_plot['chapter_names'] = ['two', 'one', 'one', None]
132 |     observation, reward, pcontinue = story.its_showtime()
133 |     del reward, pcontinue  # unused
134 | 
135 |     # For better narration of these observation comparisons, read through
136 |     # testSequenceOfGames.
137 |     self.assertBoard(observation.board, arts['one'])  # First frame.
138 |     self.assertMachinima(engine=story, frames=[(None, arts['two']),
139 |                                                (None, arts['one']),
140 |                                                (None, arts['one']),
141 |                                                (None, ['.P.',
142 |                                                        '...',
143 |                                                        '...'])])
144 | 
145 |   def testCropping(self):
146 |     """Observations from subgames can be cropped for mutual compatibility."""
147 | 
148 |     # The games will use these worlds.
149 |     arts = [
150 |         ['P..',
151 |          '...',
152 |          '...'],
153 | 
154 |         ['...............',
155 |          '...............',
156 |          '.......P.......',
157 |          '...............',
158 |          '...............'],
159 | 
160 |         ['...',
161 |          '...',
162 |          'P..'],
163 |     ]
164 | 
165 |     # But these croppers will ensure that they all have a reasonable size.
166 |     croppers = [
167 |         None,
168 |         cropping.FixedCropper(top_left_corner=(1, 6), rows=3, cols=3),
169 |         cropping.FixedCropper(top_left_corner=(0, 0), rows=3, cols=3),
170 |     ]
171 | 
172 |     # Create and start the story.
173 |     story = storytelling.Story(
174 |         chapters=[lambda art=a: self._make_game(art) for a in arts],
175 |         croppers=croppers)
176 |     observation, reward, pcontinue = story.its_showtime()
177 |     del reward, pcontinue  # unused
178 | 
179 |     # For better narration of these observation comparisons, read through
180 |     # testSequenceOfGames.
181 |     self.assertBoard(observation.board, arts[0])
182 |     self.assertMachinima(engine=story, frames=[(None, ['...',
183 |                                                        '.P.',
184 |                                                        '...']),
185 |                                                (None, arts[2]),
186 |                                                (None, ['...',
187 |                                                        '...',
188 |                                                        '.P.'])])
189 | 
190 |   def testInterGameRewardAccumulation(self):
191 |     """Inter-game terminal rewards are carried into the next game."""
192 | 
193 |     class GenerousQuitterDrape(plab_things.Drape):
194 |       """This Drape gives a reward of 5 and quits immediately."""
195 | 
196 |       def update(self, actions, board, layers, backdrop, things, the_plot):
197 |         the_plot.add_reward(5.0)
198 |         the_plot.terminate_episode()
199 | 
200 |     # Create a new Story with all subgames using the usual art.
201 |     art = ['P..',
202 |            '...',
203 |            '...']
204 |     story = storytelling.Story([
205 |         # this is perfectly readable :-P
206 |         # pylint: disable=g-long-lambda
207 | 
208 |         # We should see the initial observation from this first game, but not
209 |         # the second, terminal observation. The agent receives no reward.
210 |         lambda: ascii_art.ascii_art_to_game(art, what_lies_beneath='.',
211 |                                             sprites={'P': self._DieRightward}),
212 | 
213 |         # We should see no observations from the next three games, since they
214 |         # all terminate immediately (courtesy of GenerousQuitterDrape). However,
215 |         # they also each contribute an additional 5.0 to the summed reward the
216 |         # agent will eventually see...
217 |         lambda: ascii_art.ascii_art_to_game(art, what_lies_beneath='.',
218 |                                             sprites={'P': self._DieRightward},
219 |                                             drapes={'Q': GenerousQuitterDrape}),
220 |         lambda: ascii_art.ascii_art_to_game(art, what_lies_beneath='.',
221 |                                             sprites={'P': self._DieRightward},
222 |                                             drapes={'Q': GenerousQuitterDrape}),
223 |         lambda: ascii_art.ascii_art_to_game(art, what_lies_beneath='.',
224 |                                             sprites={'P': self._DieRightward},
225 |                                             drapes={'Q': GenerousQuitterDrape}),
226 | 
227 |         # ...when it sees the first observation of this game here. The second,
228 |         # terminal observation is dropped, but then...
229 |         lambda: ascii_art.ascii_art_to_game(art, what_lies_beneath='.',
230 |                                             sprites={'P': self._DieRightward}),
231 | 
232 |         # ...we finally see an observation from a game involving a
233 |         # GenerousQuitterDrape when we see its first and terminal step, as the
234 |         # terminal step of the story. We also receive another 5.0 reward.
235 |         lambda: ascii_art.ascii_art_to_game(art, what_lies_beneath='.',
236 |                                             sprites={'P': self._DieRightward},
237 |                                             drapes={'Q': GenerousQuitterDrape}),
238 |         # pylint: enable=g-long-lambda
239 |     ])
240 | 
241 |     # Now to see if our predictions are true.
242 |     observation, reward, pcontinue = story.its_showtime()  # First step.
243 |     self.assertBoard(observation.board, art)
244 |     self.assertIsNone(reward)  # Nobody assigned any reward in this step.
245 |     self.assertEqual(pcontinue, 1.0)
246 | 
247 |     observation, reward, pcontinue = story.play(None)  # Second step.
248 |     self.assertBoard(observation.board, art)  # First obs. of penultimate game.
249 |     self.assertEqual(reward, 15.0)  # Summed across final steps of games 2-4.
250 |     self.assertEqual(pcontinue, 1.0)
251 | 
252 |     observation, reward, pcontinue = story.play(None)  # Third, final step.
253 |     self.assertBoard(observation.board, art)  # Terminal obs. of final game.
254 |     self.assertEqual(reward, 5.0)  # From the GenerousQuitterDrape.
255 |     self.assertEqual(pcontinue, 0.0)
256 | 
257 |   def testStandIns(self):
258 |     """The "abstraction breakers" in `Story` are suitably simulated."""
259 | 
260 |     # The games will use these worlds.
261 |     arts = list(zip(['P~~', '..S'],
262 |                     ['~~~', '..D'],
263 |                     ['www', '###']))
264 | 
265 |     # Create the story.
266 |     story = storytelling.Story([
267 |         # this is perfectly readable :-P
268 |         # pylint: disable=g-long-lambda
269 |         lambda: ascii_art.ascii_art_to_game(arts[0], what_lies_beneath='~',
270 |                                             sprites={'P': self._DieRightward},
271 |                                             drapes={'w': self._IdleDrape}),
272 |         lambda: ascii_art.ascii_art_to_game(arts[1], what_lies_beneath='.',
273 |                                             sprites={'S': self._DieRightward},
274 |                                             drapes={'D': self._IdleDrape})
275 |         # pylint: enable=g-long-lambda
276 |     ])
277 | 
278 |     # The "abstraction breaker" methods should fake the same sorts of results
279 |     # that you would get if the Story were actually implemented by a single
280 |     # Engine. Sprites and Drapes should contain an entry for any character
281 |     # associated with a Sprite or a Drape across all the games, and the
282 |     # contrived Backdrop's palette should have all the backdrop characters
283 |     # from anywhere in the story.
284 |     self.assertEqual(sorted(story.backdrop.palette), ['#', '.', '~'])
285 |     self.assertEqual(sorted(story.things), ['D', 'P', 'S', 'w'])
286 | 
287 |     # The only real Sprites and Drapes in story.things are the ones from the
288 |     # current game; the others are dummies. Here we test the module function
289 |     # that identifies dummies.
290 |     self.assertTrue(storytelling.is_fictional(story.things['D']))
291 |     self.assertFalse(storytelling.is_fictional(story.things['P']))
292 |     self.assertTrue(storytelling.is_fictional(story.things['S']))
293 |     self.assertFalse(storytelling.is_fictional(story.things['w']))
294 | 
295 |   def testCompatibilityChecking(self):
296 |     """The `Story` constructor spots compatibility problems between games."""
297 | 
298 |     # The first Story will fail because these game worlds have different sizes,
299 |     # and there are no croppers in use.
300 |     arts = [
301 |         ['P..',
302 |          '...',
303 |          '...'],
304 | 
305 |         ['...............',
306 |          '...............',
307 |          '.......P.......',
308 |          '...............',
309 |          '...............'],
310 |     ]
311 | 
312 |     with six.assertRaisesRegex(self, ValueError,
313 |                                'observations that are the same'):
314 |       _ = storytelling.Story([lambda art=a: self._make_game(art) for a in arts])
315 | 
316 |     # This next Story will fail because characters are shared between Sprites,
317 |     # Drapes, and at least one Backdrop.
318 |     art = ['1..',
319 |            '.2.',
320 |            '..3']
321 | 
322 |     error_regexp = (r'.*same character in two different ways'
323 |                     r'.*both a Sprite and a Drape: \[2\];'
324 |                     r'.*both a Sprite and in a Backdrop: \[1\];'
325 |                     r'.*both a Drape and in a Backdrop: \[3\].*')
326 |     with six.assertRaisesRegex(self, ValueError, error_regexp):
327 |       _ = storytelling.Story([
328 |           # pylint: disable=g-long-lambda
329 |           lambda: ascii_art.ascii_art_to_game(art, what_lies_beneath='.',
330 |                                               sprites={'1': self._DieRightward},
331 |                                               drapes={'2': self._IdleDrape}),
332 |           lambda: ascii_art.ascii_art_to_game(art, what_lies_beneath='.',
333 |                                               sprites={'2': self._DieRightward},
334 |                                               drapes={'3': self._IdleDrape}),
335 |           # pylint: enable=g-long-lambda
336 |       ])
337 | 
338 | 
339 | def main(argv=()):
340 |   del argv  # Unused.
341 |   unittest.main()
342 | 
343 | 
344 | if __name__ == '__main__':
345 |   main(sys.argv)
346 | 


--------------------------------------------------------------------------------
/setup.py:
--------------------------------------------------------------------------------
  1 | # Copyright 2017 the pycolab Authors
  2 | #
  3 | # Licensed under the Apache License, Version 2.0 (the "License");
  4 | # you may not use this file except in compliance with the License.
  5 | # You may obtain a copy of the License at
  6 | #
  7 | #     https://www.apache.org/licenses/LICENSE-2.0
  8 | #
  9 | # Unless required by applicable law or agreed to in writing, software
 10 | # distributed under the License is distributed on an "AS IS" BASIS,
 11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 12 | # See the License for the specific language governing permissions and
 13 | # limitations under the License.
 14 | 
 15 | """pycolab PyPI package setup."""
 16 | 
 17 | from __future__ import absolute_import
 18 | from __future__ import division
 19 | from __future__ import print_function
 20 | 
 21 | try:
 22 |   import setuptools
 23 | except ImportError:
 24 |   from ez_setup import use_setuptools
 25 |   use_setuptools()
 26 |   import setuptools
 27 | 
 28 | # Warn user about how curses is required to play games yourself.
 29 | try:
 30 |   import curses
 31 | except ImportError:
 32 |   import warnings
 33 |   warnings.warn(
 34 |       'The human_ui module and all of the example games (when run as '
 35 |       'standalone programs) require the curses library. Without curses, you '
 36 |       'can still use pycolab as a library, but you won\'t be able to play '
 37 |       'pycolab games on the console.')
 38 | 
 39 | setuptools.setup(
 40 |     name='pycolab',
 41 |     version='1.2',
 42 |     description='An engine for small games for reinforcement learning agents.',
 43 |     long_description=(
 44 |         'A highly-customisable all-Python gridworld game engine with some '
 45 |         'batteries included. Make your own gridworld games to demonstrate '
 46 |         'reinforcement learning problems and test your agents!'),
 47 |     url='https://github.com/deepmind/pycolab/',
 48 |     author='The pycolab authors',
 49 |     author_email='pycolab@deepmind.com',
 50 |     license='Apache 2.0',
 51 |     classifiers=[
 52 |         'Development Status :: 5 - Production/Stable',
 53 |         'Environment :: Console',
 54 |         'Environment :: Console :: Curses',
 55 |         'Intended Audience :: Developers',
 56 |         'Intended Audience :: Education',
 57 |         'Intended Audience :: Science/Research',
 58 |         'License :: OSI Approved :: Apache Software License',
 59 |         'Operating System :: MacOS :: MacOS X',
 60 |         'Operating System :: Microsoft :: Windows',
 61 |         'Operating System :: POSIX',
 62 |         'Operating System :: Unix',
 63 |         'Programming Language :: Python :: 2.7',
 64 |         'Programming Language :: Python :: 3.4',
 65 |         'Programming Language :: Python :: 3.5',
 66 |         'Programming Language :: Python :: 3.6',
 67 |         'Topic :: Games/Entertainment :: Arcade',
 68 |         'Topic :: Scientific/Engineering :: Artificial Intelligence',
 69 |         'Topic :: Software Development :: Libraries',
 70 |         'Topic :: Software Development :: Testing',
 71 |     ],
 72 |     keywords=(
 73 |         'ai '
 74 |         'ascii art '
 75 |         'game engine '
 76 |         'gridworld '
 77 |         'reinforcement learning '
 78 |         'retro retrogaming'),
 79 | 
 80 |     install_requires=[
 81 |         'numpy>=1.9',
 82 |         'six',
 83 |     ],
 84 |     extras_require={
 85 |         'ndimage': ['scipy>=0.13.3'],
 86 |     },
 87 | 
 88 |     packages=setuptools.find_packages(),
 89 |     zip_safe=True,
 90 |     entry_points={
 91 |         'console_scripts': [
 92 |             'aperture = pycolab.examples.aperture:main',
 93 |             'apprehend = pycolab.examples.apprehend:main',
 94 |             ('extraterrestrial_marauders = '
 95 |              'pycolab.examples.extraterrestrial_marauders:main'),
 96 |             'fluvial_natation = pycolab.examples.fluvial_natation:main',
 97 |             'hello_world = pycolab.examples.hello_world:main',
 98 |             'scrolly_maze = pycolab.examples.scrolly_maze:main',
 99 |             'shockwave = pycolab.examples.shockwave:main [ndimage]',
100 |             'warehouse_manager = pycolab.examples.warehouse_manager:main',
101 |             'chain_walk = pycolab.examples.classics.chain_walk:main',
102 |             'cliff_walk = pycolab.examples.classics.cliff_walk:main',
103 |             'four_rooms = pycolab.examples.classics.four_rooms:main',
104 |         ],
105 |     },
106 | 
107 |     test_suite='nose.collector',
108 |     tests_require=['nose'],
109 | )
110 | 


--------------------------------------------------------------------------------