├── .gitignore
├── README.md
├── celery_app
    ├── app
    │   ├── __init__.py
    │   ├── db.py
    │   ├── service.py
    │   └── tasks.py
    ├── demo.py
    ├── docker
    │   ├── celery.dockerfile
    │   ├── flower.dockerfile
    │   └── postgis
    │   │   ├── Dockerfile
    │   │   ├── README.txt
    │   │   ├── geopython.dump
    │   │   ├── initdb-postgis.sh
    │   │   └── update-postgis.sh
    ├── requirements
    │   ├── celery_app.txt
    │   └── flower_app.txt
    └── tests
    │   ├── __init__.py
    │   └── tests_unit
    │       ├── __init__.py
    │       └── test_tasks.py
├── docker-compose.yml
└── extras
    ├── geopython-final.key
    └── map.qgs


/.gitignore:
--------------------------------------------------------------------------------
1 | .idea/
2 | __pycache__/
3 | build_and_run.sh
4 | .pytest_cache
5 | 


--------------------------------------------------------------------------------
/README.md:
--------------------------------------------------------------------------------
  1 | # GeoPython Workshop 2018
  2 | ## Task queues with Celery and RabbitMQ
  3 | 
  4 | Celery is a popular task queue library for the Python programming language. This demo application asynchronously executes a simple geoprocessing task which creates a geometry and writes it to a PostGIS database.
  5 | 
  6 | This application is a [docker-compose](https://docs.docker.com/compose/) orchestration of four Docker containers:
  7 | 
  8 | - [Celery](http://docs.celeryproject.org/en/latest/index.html) - Python Celery application to produce/consume messages (the worker)
  9 | - [RabbitMQ](https://www.rabbitmq.com/#getstarted) - Message broker (the queue)
 10 | - [Flower](http://flower.readthedocs.io/en/latest/) - Web app to monitor tasks
 11 | - [PostGIS](https://postgis.net/) - Spatial database to store task output
 12 | 
 13 | ## Before you start...
 14 | 
 15 | This application requires [Docker Community Edition and Docker Compose](https://www.docker.com/community-edition).
 16 | 
 17 | #### Build and run the containers
 18 | 
 19 | 1. Open a terminal and change directory to the location of `docker-compose.yml`.
 20 | 
 21 | 2. Build and start the containers (some images are pulled from Docker hub - be patient!):
 22 | 
 23 |   ```docker
 24 |   docker-compose up --build
 25 |   ```
 26 | 
 27 |   The console will display the stdout from each container as they build and run.
 28 | 
 29 |   You can inspect the docker-compose file to gain a further understanding of the application structure.
 30 | 
 31 | 3. Check everything is running OK, in a new terminal window (as it is starting up interactively in the first console):
 32 | 
 33 |   ```docker
 34 |   docker-compose ps
 35 |   ```
 36 | 
 37 |   You should see the following containers ids listed
 38 |     ```docker
 39 |     celeryworkshopgeopython2018_celery_1                                                                                
 40 |     celeryworkshopgeopython2018_flower_1                                                         
 41 |     celeryworkshopgeopython2018_postgis_1                                                                                   
 42 |     celeryworkshopgeopython2018_rabbitmq_1   
 43 |     ```
 44 |   
 45 |   The state of each container should be "Up".
 46 | 
 47 | 4. You may wish to execute commands inside a container:
 48 | 
 49 |   ```docker
 50 |   docker exec -it <container id> /bin/sh -c "<command for container>"
 51 |   ```
 52 |   For example:
 53 |   ```docker
 54 |   docker exec -it celeryworkshopgeopython2018_celery_1  /bin/sh -c "ls"
 55 |   ```  
 56 | 
 57 | 5. Stop the containers by cancelling the process in terminal window.
 58 | 
 59 | #### Re-deploying changes to the code
 60 | 
 61 | Each time you make a change to any Python files run the command in step 2 - `docker-compose up --build`. This will rebuild the images incorporating any changes.
 62 | 
 63 | #### Monitoring tasks with Flower
 64 | 
 65 | Flower is a web application for monitoring Celery tasks. Once the application is running, in your web browser navigate to http://localhost:5555.
 66 | 
 67 | Flower provides monitoring of task executed by the worker. Some of Flowers features include:
 68 | 
 69 | - View tasks on the queue
 70 | - Inspect and control tasks
 71 | - Inspect failed tasks exception messages
 72 | - Control worker process pool size
 73 | - Monitor
 74 | - [periodic tasks](http://docs.celeryproject.org/en/latest/userguide/periodic-tasks.html)
 75 | 
 76 | Not all of these features are available in this project.
 77 | 
 78 | #### Troubleshooting
 79 | 
 80 | The logs of each container can be inspected with the command:
 81 | 
 82 | ```docker
 83 | docker logs <container id>
 84 | ```
 85 | 
 86 | The container ID's can be identified with the docker ps command.
 87 | 
 88 | If you think you container is broken beyond repair run:
 89 | 
 90 | ```docker
 91 | docker-compose up --force-recreate
 92 | ```
 93 | 
 94 | **This will force a complete rebuild so be aware this will also pull external images again.**
 95 | 
 96 | ## Workshop begins...
 97 | 
 98 | #### Instantiate Celery application object
 99 | 
100 | Before defining any tasks, a Celery application object must be instantiated.
101 | 
102 | *celery_app/app/tasks.py*
103 | ```python
104 | from celery import Celery
105 | 
106 | # URL to connect to broker
107 | broker_url = 'amqp://celery_user:secret@rabbitmq:5672/celery_app'
108 | 
109 | # Create Celery application
110 | application = Celery('tasks', broker=broker_url)
111 | ...
112 | ```
113 | 
114 | The `Celery` object has two required parameters:
115 | - Name of the current module - 'tasks'
116 | - URL of the message broker as a keyword argument
117 | 
118 | #### Defining a task
119 | 
120 | To define a task to run asynchronously, simply apply the `task` decorator to a function. This application has a single task `do_task` defined in `tasks.py`.
121 | 
122 | *celery_app/app/tasks.py*
123 | ```python
124 | ...
125 | 
126 | @application.task(base=AppBaseTask, bind=True, max_retries=3, soft_time_limit=5)
127 | def do_task(self, x, y):
128 |     """Performs simple geoprocessing task.
129 | 
130 |     Failed tasks are retried x times by the Task classes on_retry method.
131 |     When tasks fail completely they are handled by the Task classes on_failure method
132 | 
133 |     Args:
134 |         self: instance of the Task
135 |         x: integer
136 |         y: integer
137 | 
138 |     Raises:
139 |         TaskError: failed tasked are handled by the parent task class.
140 | 
141 |     Returns:
142 |         None
143 |     """
144 |     try:
145 |         geoprocess(x, y)
146 |     except ServiceError as se:
147 |         self.retry(countdown=10, exc=se)
148 |     except Exception as exc:
149 |         raise TaskError(exc)
150 | ...
151 | ```
152 | 
153 | The `task` decorator takes several optional keyword arguments including the maximum number of retries to attempt in the event of a failure and a timeout limit for hanging tasks.
154 | 
155 | #### Calling tasks asynchronously
156 | 
157 | To "queue" a task use the `delay` function and provide the arguments which the `do_task` function requires.
158 | 
159 | *celery_app/demo.py*
160 | ```python
161 | from app.tasks import do_task
162 | 
163 | def call_do_task():
164 |     """Call the do_task task x times."""
165 |     iterations = 2500
166 | 
167 |     for task_execution in range(iterations):
168 | 
169 |         x = get_random_x()
170 |         y = get_random_y()
171 | 
172 |         do_task.delay(x, y)
173 | 
174 |         print(f"called do_task({x}, {y}) asynchronously")
175 | ...
176 | ```
177 | 
178 | ### Challenge 1
179 | 
180 | The challenge is to write code to add tasks to the queue.
181 | 
182 | Hints:
183 | 
184 | 1. Checkout branch `challenge-1`:
185 | ```bash
186 | git checkout challenge-1
187 | ```
188 | 
189 | 2. Look in *celery_app/demo.py* for the TODO comment relating to challenge 1.
190 | 
191 | 3. Look in *celery_app/app/tasks.py* to identify the function names and expected parameters of the the celery task.
192 | 
193 | 4. In *celery_app/demo.py* write a loop that will iterate 2500 times.  Inside the loop, use the two helper functions `get_random_x` and `get_random_y` to get suitable x and y values, add these values to the queue.  You will need to use the special celery `delay` function.
194 | 
195 | 5. Rebuild the docker containers after you have made your changes, [then run `demo.py`](#build-and-run-the-containers) within the docker container using docker exec. Check the process of the running/queued tasks in the [Flower web app](#monitoring-tasks-with-flower).
196 | 
197 | 6. If you have pgadmin or similar, you can connect to the database and see records being added to the grid_squares table.  If you have desktop GIS package (such as QGIS), you will be able to add the table as a vector layer to visualise the data.
198 | 
199 | The database connection details are:
200 | - DB = geopython-db
201 | - Port = 5432
202 | - User = geopython
203 | - Password = geopython
204 | - Schema = geopython
205 | 
206 | ### Challenge 2
207 | 
208 | The challenge is to get celery to retry processing a task in the event of a recoverable error.
209 | 
210 | Hints:
211 | 
212 | 1. Checkout branch `challenge-2`. **make sure you commit any changes you wish to keep or discard changes before switching branch**
213 | 
214 | 2. Rebuild the docker containers, then run `demo.py`.  Observe that some tasks fail and none are retried.
215 | 
216 | 3. Check the celery docs for [retrying failed tasks](http://docs.celeryproject.org/en/latest/userguide/tasks.html#retrying).
217 | 
218 | 4. Look in *celery_app/app/service.py* to see where a random artificial error is thrown.  Note the exception type.
219 | 
220 | 5. In the place where the celery task is executed (look for `TODO - challenge 2`), handle the above error using python exception handling, and trigger the Celery retry mechanism.  See if you can also add a retry delay.
221 | 
222 | 6. Rebuild the docker containers after you have made your changes, then run `demo.py` - If you have been successful, you should see the retry counter in flower incrementing, and most likely no fails.  All tasks should pass after being retried.
223 | 
224 | ### Challenge 3
225 | 
226 | This challenge will show multiple celery workers on different machines all pulling tasks from the same queue so that tasks are processed faster.
227 | 
228 | In this exercise, if the technology allows us, you will connect your celery worker to a central queue, process tasks, and write the results to a central database.
229 | 
230 | 1. Checkout branch `challenge-3`.
231 | 
232 | 2. Edit *celery_app/app/tasks.py* line #7 to replace `***IP_ADDRESS_HERE***` with the IP address given to you in the workshop.  This allows the celery worker to connect to the central queue.
233 | 
234 | 3. Edit *celery_app/app/db.py* line #14 to replace `***IP_ADDRESS_HERE***` with the IP address given to you in the workshop. This is for connection to write results to a central database.
235 | 
236 | 4. Rebuild the Docker containers
237 | 
238 | 5. Open up the Flower web interface.  If everything is working correctly you should be able to see all the workers that are connected to the central queue and the speeds at which they are processing tasks.
239 | 
240 | The results of the processing are being shown on the screen.
241 | 
242 | 
243 | 
244 | ### Tests
245 | 
246 | ```bash
247 | python -m pytest tests
248 | ```
249 | 


--------------------------------------------------------------------------------
/celery_app/app/__init__.py:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/thinkWhere/Celery-Workshop-GeoPython-2018/355beb3ba38d96cedd7d3680a6a54e60d0c541f8/celery_app/app/__init__.py


--------------------------------------------------------------------------------
/celery_app/app/db.py:
--------------------------------------------------------------------------------
 1 | from contextlib import contextmanager
 2 | 
 3 | from psycopg2.pool import ThreadedConnectionPool
 4 | 
 5 | 
 6 | pool = None
 7 | 
 8 | def create_connection_pool():
 9 |     """Create a connection pool for the worker"""
10 |     pool = ThreadedConnectionPool(1, 4,
11 |                                   dbname="geopython-db",
12 |                                   user="geopython",
13 |                                   host="postgis",
14 |                                   port="5432",
15 |                                   password="geopython")
16 |     return pool
17 | 
18 | 
19 | @contextmanager
20 | def get_db_connection():
21 |     """Context manager for a database connection"""
22 |     try:
23 |         # Setup connection pool is it doesn't exist
24 |         global pool
25 |         if pool is None:
26 |             pool = create_connection_pool()
27 | 
28 |         connection = pool.getconn()
29 |         yield connection
30 |     finally:
31 |         pool.putconn(connection)
32 | 
33 | 
34 | @contextmanager
35 | def get_db_cursor():
36 |     """Context manager for database cursor"""
37 |     with get_db_connection() as connection:
38 |       cursor = connection.cursor()
39 |       try:
40 |           yield cursor
41 |           connection.commit()
42 |       finally:
43 |           cursor.close()
44 | 


--------------------------------------------------------------------------------
/celery_app/app/service.py:
--------------------------------------------------------------------------------
 1 | import math
 2 | import psycopg2
 3 | 
 4 | from app.db import get_db_cursor
 5 | 
 6 | 
 7 | class ServiceError(Exception):
 8 |     """Custom exception for service errors"""
 9 |     pass
10 | 
11 | 
12 | def geoprocess(x, y):
13 |     """
14 |     Implementation code for celery task
15 |     :param x: x value for task
16 |     :param y: y value for task
17 |     """
18 |     with get_db_cursor() as cur:
19 | 
20 |         # check for intersection
21 |         query = f"""SELECT gu_a3
22 |              from uk_boundary
23 |              WHERE ST_Intersects(geom, ST_GeomFromEWKT('SRID=27700;POINT({x} {y})'));
24 |              """
25 |         cur.execute(query)
26 | 
27 |         # determine style code to use
28 |         rows = cur.fetchall()
29 |         if rows:
30 |             style = rows[0][0]
31 |         else:
32 |             style = 'SEA'
33 | 
34 |         # get ref code
35 |         ref = get_ref(x, y)
36 | 
37 |         # make object for db and insert it. Insert exceptions will raise to task.
38 |         size = 1000
39 |         xmin = int(x / size) * size
40 |         xmax = xmin + size
41 |         ymin = int(y / size) * size
42 |         ymax = ymin + size
43 |         string_for_db = f"ST_GeometryFromText('POLYGON(({xmin} {ymin},{xmax} {ymin},{xmax} {ymax},{xmin} {ymax},{xmin} {ymin}))',27700)"
44 |         cur.execute(f"INSERT INTO grid_squares (grid_ref, style_cat, geom) VALUES('{ref}', '{style}', {string_for_db})")
45 | 
46 | 
47 | def get_ref(e, n):
48 |     # Derived from
49 |     # http://www.movable-type.co.uk/scripts/latlong-gridref.html
50 |     gridChars = "ABCDEFGHJKLMNOPQRSTUVWXYZ"
51 |     e100k = math.floor(e / 100000)
52 |     n100k = math.floor(n / 100000)
53 |     l1 = (19 - n100k) - (19 - n100k) % 5 + math.floor((e100k + 10) / 5)
54 |     l2 = (19 - n100k) * 5 % 25 + e100k % 5
55 |     letPair = gridChars[int(l1)] + gridChars[int(l2)]
56 |     e100m = math.trunc(round(float(e) / 100))
57 |     egr = str(e100m).rjust(4, "0")[1:]
58 |     if n >= 1000000:
59 |         n = n - 1000000
60 |     n100m = math.trunc(round(float(n) / 100))
61 |     ngr = str(n100m).rjust(4, "0")[1:]
62 |     return letPair + egr + ngr
63 | 


--------------------------------------------------------------------------------
/celery_app/app/tasks.py:
--------------------------------------------------------------------------------
 1 | from celery import Celery, Task
 2 | 
 3 | from app.service import geoprocess, ServiceError
 4 | 
 5 | # URL to connect to broker
 6 | broker_url = 'amqp://celery_user:secret@rabbitmq:5672/celery_app'
 7 | 
 8 | # Create Celery application
 9 | application = Celery('tasks', broker=broker_url)
10 | 
11 | 
12 | class TaskError(Exception):
13 |     """Custom exception for handling task errors"""
14 |     pass
15 | 
16 | 
17 | class AppBaseTask(Task):
18 |     """Base Task
19 | 
20 |     The base Task class can be used to define
21 |     shared behaviour between tasks.
22 | 
23 |     """
24 | 
25 |     abstract = True
26 | 
27 |     def on_retry(self, exc, task_id, args, kwargs, einfo):
28 |         # TODO: log out here
29 |         super(AppBaseTask, self).on_retry(exc, task_id, args, kwargs, einfo)
30 | 
31 |     def on_failure(self, exc, task_id, args, kwargs, einfo):
32 |         # TODO: log out here
33 |         super(AppBaseTask, self).on_failure(exc, task_id, args, kwargs, einfo)
34 | 
35 | 
36 | @application.task(base=AppBaseTask, bind=True, max_retries=3, soft_time_limit=5)
37 | def do_task(self, x, y):
38 |     """Performs simple geoprocessing task.
39 | 
40 |     Failed tasks are retried x times by the Task classes on_retry method.
41 |     When tasks fail completely they are handled by the Task classes on_failure method
42 | 
43 |     Args:
44 |         self: instance of the Task
45 |         x: integer
46 |         y: integer
47 | 
48 |     Raises:
49 |         TaskError: failed tasked are handled by the parent task class.
50 | 
51 |     Returns:
52 |         None
53 |     """
54 |     try:
55 |         geoprocess(x, y)
56 |     except ServiceError as se:
57 |         self.retry(countdown=10, exc=se)
58 |     except Exception as exc:
59 |         raise TaskError(exc)
60 | 


--------------------------------------------------------------------------------
/celery_app/demo.py:
--------------------------------------------------------------------------------
 1 | from app.tasks import do_task
 2 | import random
 3 | 
 4 | 
 5 | def call_do_task():
 6 |     """Call the do_task task x times."""
 7 |     iterations = 2500
 8 | 
 9 |     for task_execution in range(iterations):
10 | 
11 |         x = get_random_x()
12 |         y = get_random_y()
13 | 
14 |         do_task.delay(x, y)
15 | 
16 |         print(f"called do_task({x}, {y}) asynchronously")
17 | 
18 | 
19 | def get_random_x():
20 |     return random.randint(0, 700000)
21 | 
22 | 
23 | def get_random_y():
24 |     return random.randint(0, 1300000)
25 | 
26 | 
27 | if __name__ == '__main__':
28 | 
29 |     call_do_task()
30 | 


--------------------------------------------------------------------------------
/celery_app/docker/celery.dockerfile:
--------------------------------------------------------------------------------
 1 | FROM python:3.6.4-alpine3.7
 2 | 
 3 | COPY requirements/celery_app.txt /celery_app/requirements/celery_app.txt
 4 | 
 5 | RUN pip install --upgrade pip
 6 | 
 7 | RUN apk update \
 8 |   && apk add --virtual build-deps gcc python3-dev musl-dev \
 9 |   && apk add postgresql-dev \
10 |   && pip install -r /celery_app/requirements/celery_app.txt \
11 |   && apk del build-deps
12 | 
13 | WORKDIR /celery_app
14 | 
15 | COPY . /celery_app
16 | 
17 | CMD ["celery", "-A", "app.tasks", "worker", "--loglevel=info"]
18 | 


--------------------------------------------------------------------------------
/celery_app/docker/flower.dockerfile:
--------------------------------------------------------------------------------
 1 | FROM python:3.6.4-alpine3.7
 2 | 
 3 | COPY requirements/flower_app.txt /celery_app/requirements/flower_app.txt
 4 | 
 5 | RUN pip install --upgrade pip
 6 | 
 7 | RUN apk update \
 8 |   && apk add --virtual build-deps gcc python3-dev musl-dev \
 9 |   && apk add postgresql-dev \
10 |   && pip install -r /celery_app/requirements/flower_app.txt \
11 |   && apk del build-deps
12 | 
13 | WORKDIR /celery_app
14 | 
15 | COPY . /celery_app
16 | 
17 | CMD ["celery", "-A", "app.tasks", "flower"]
18 | 


--------------------------------------------------------------------------------
/celery_app/docker/postgis/Dockerfile:
--------------------------------------------------------------------------------
 1 | FROM postgres:9.6-alpine
 2 | MAINTAINER Régis Belson <me@regisbelson.fr>
 3 | 
 4 | ENV POSTGIS_VERSION 2.4.3
 5 | ENV POSTGIS_SHA256 b9754c7b9cbc30190177ec34b570717b2b9b88ed271d18e3af68eca3632d1d95
 6 | 
 7 | RUN set -ex \
 8 |     \
 9 |     && apk add --no-cache --virtual .fetch-deps \
10 |         ca-certificates \
11 |         openssl \
12 |         tar \
13 |     \
14 |     && wget -O postgis.tar.gz "https://github.com/postgis/postgis/archive/$POSTGIS_VERSION.tar.gz" \
15 |     && echo "$POSTGIS_SHA256 *postgis.tar.gz" | sha256sum -c - \
16 |     && mkdir -p /usr/src/postgis \
17 |     && tar \
18 |         --extract \
19 |         --file postgis.tar.gz \
20 |         --directory /usr/src/postgis \
21 |         --strip-components 1 \
22 |     && rm postgis.tar.gz \
23 |     \
24 |     && apk add --no-cache --virtual .build-deps \
25 |         autoconf \
26 |         automake \
27 |         g++ \
28 |         json-c-dev \
29 |         libtool \
30 |         libxml2-dev \
31 |         make \
32 |         perl \
33 |     \
34 |     && apk add --no-cache --virtual .build-deps-testing \
35 |         --repository http://dl-cdn.alpinelinux.org/alpine/edge/testing \
36 |         gdal-dev \
37 |         geos-dev \
38 |         proj4-dev \
39 |         protobuf-c-dev \
40 |     && cd /usr/src/postgis \
41 |     && ./autogen.sh \
42 | # configure options taken from:
43 | # https://anonscm.debian.org/cgit/pkg-grass/postgis.git/tree/debian/rules?h=jessie
44 |     && ./configure \
45 | #       --with-gui \
46 |     && make \
47 |     && make install \
48 |     && apk add --no-cache --virtual .postgis-rundeps \
49 |         json-c \
50 |     && apk add --no-cache --virtual .postgis-rundeps-testing \
51 |         --repository http://dl-cdn.alpinelinux.org/alpine/edge/testing \
52 |         geos \
53 |         gdal \
54 |         proj4 \
55 |         protobuf-c \
56 |     && cd / \
57 |     && rm -rf /usr/src/postgis \
58 |     && apk del .fetch-deps .build-deps .build-deps-testing
59 | 
60 | COPY ./geopython.dump /geopython.dump
61 | RUN chmod 644 /geopython.dump
62 | COPY ./initdb-postgis.sh /docker-entrypoint-initdb.d/postgis.sh
63 | RUN chmod 644 /docker-entrypoint-initdb.d/postgis.sh
64 | COPY ./update-postgis.sh /usr/local/bin


--------------------------------------------------------------------------------
/celery_app/docker/postgis/README.txt:
--------------------------------------------------------------------------------
1 | CREDITS -
2 | 
3 | Postgis Alpine docker adapted from the container by Matt Dillon  https://github.com/appropriate/docker-postgis
4 | Mapping data from Natural Earth  http://www.naturalearthdata.com/downloads/10m-cultural-vectors/ 


--------------------------------------------------------------------------------
/celery_app/docker/postgis/initdb-postgis.sh:
--------------------------------------------------------------------------------
 1 | #!/bin/sh
 2 | 
 3 | #set -e
 4 | 
 5 | # Perform all actions as $POSTGRES_USER
 6 | export PGUSER="$POSTGRES_USER"
 7 | 
 8 | # Load PostGIS into both template_database and $POSTGRES_DB
 9 | echo "Loading PostGIS extensions into $DB"
10 | "${psql[@]}" --dbname=$POSTGRES_DB <<-'EOSQL'
11 | 		CREATE EXTENSION IF NOT EXISTS postgis;
12 | EOSQL
13 | 
14 | "${psql[@]}" --dbname=$POSTGRES_DB -f /geopython.dump


--------------------------------------------------------------------------------
/celery_app/docker/postgis/update-postgis.sh:
--------------------------------------------------------------------------------
 1 | #!/bin/sh
 2 | 
 3 | set -e
 4 | 
 5 | # Perform all actions as $POSTGRES_USER
 6 | export PGUSER="$POSTGRES_USER"
 7 | 
 8 | POSTGIS_VERSION="${POSTGIS_VERSION%%+*}"
 9 | 
10 | # Load PostGIS into both template_database and $POSTGRES_DB
11 | for DB in template_postgis "$POSTGRES_DB"; do
12 |     echo "Updating PostGIS extensions '$DB' to $POSTGIS_VERSION"
13 |     psql --dbname="$DB" -c "
14 |         -- Upgrade PostGIS (includes raster)
15 |         CREATE EXTENSION IF NOT EXISTS postgis VERSION '$POSTGIS_VERSION';
16 |         ALTER EXTENSION postgis  UPDATE TO '$POSTGIS_VERSION';
17 |         -- Upgrade Topology
18 |         CREATE EXTENSION IF NOT EXISTS postgis_topology VERSION '$POSTGIS_VERSION';
19 |         ALTER EXTENSION postgis_topology UPDATE TO '$POSTGIS_VERSION';
20 |         -- Upgrade US Tiger Geocoder
21 |         CREATE EXTENSION IF NOT EXISTS postgis_tiger_geocoder VERSION '$POSTGIS_VERSION';
22 |         ALTER EXTENSION postgis_tiger_geocoder UPDATE TO '$POSTGIS_VERSION';
23 |     "
24 | done


--------------------------------------------------------------------------------
/celery_app/requirements/celery_app.txt:
--------------------------------------------------------------------------------
 1 | amqp==2.2.2
 2 | attrs==17.4.0
 3 | billiard==3.5.0.3
 4 | celery==4.1.0
 5 | kombu==4.1.0
 6 | pluggy==0.6.0
 7 | py==1.5.2
 8 | pytest==3.4.1
 9 | pytz==2018.3
10 | six==1.11.0
11 | vine==1.1.4
12 | psycopg2==2.7.4
13 | 


--------------------------------------------------------------------------------
/celery_app/requirements/flower_app.txt:
--------------------------------------------------------------------------------
 1 | amqp==2.2.2
 2 | Babel==2.5.3
 3 | billiard==3.5.0.3
 4 | celery==4.1.0
 5 | flower==0.9.2
 6 | kombu==4.1.0
 7 | pytz==2018.3
 8 | tornado==5.0
 9 | vine==1.1.4
10 | psycopg2==2.7.4
11 | 


--------------------------------------------------------------------------------
/celery_app/tests/__init__.py:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/thinkWhere/Celery-Workshop-GeoPython-2018/355beb3ba38d96cedd7d3680a6a54e60d0c541f8/celery_app/tests/__init__.py


--------------------------------------------------------------------------------
/celery_app/tests/tests_unit/__init__.py:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/thinkWhere/Celery-Workshop-GeoPython-2018/355beb3ba38d96cedd7d3680a6a54e60d0c541f8/celery_app/tests/tests_unit/__init__.py


--------------------------------------------------------------------------------
/celery_app/tests/tests_unit/test_tasks.py:
--------------------------------------------------------------------------------
 1 | from unittest.mock import patch
 2 | 
 3 | from celery.exceptions import Retry
 4 | from pytest import raises
 5 | 
 6 | from app.service import ServiceError
 7 | from app.tasks import do_task, TaskError
 8 | 
 9 | 
10 | @patch('app.tasks.geoprocess')
11 | def test_do_task_success(mock_add):
12 |     # Act
13 |     do_task(2, 3)
14 | 
15 |     # Assert
16 |     mock_add.assert_called_with(2, 3)
17 | 
18 | 
19 | @patch('app.tasks.geoprocess')
20 | @patch('app.tasks.do_task.retry')
21 | def test_do_task_handles_serviceerror_with_retry(mock_add_task_retry, mock_geoprocess):
22 | 
23 |     # Arrange
24 |     mock_add_task_retry.side_effect = Retry()
25 |     mock_geoprocess.side_effect = ServiceError('Random fail!')
26 | 
27 |     # Act/assert
28 |     with raises(Retry):
29 |         do_task(2, 1)
30 | 
31 | 
32 | @patch('app.tasks.geoprocess')
33 | @patch('app.tasks.do_task.retry')
34 | def test_do_task_handles_unexpected_exception(mock_add_task_retry, mock_geoprocess):
35 | 
36 |     # Arrange
37 |     mock_add_task_retry.side_effect = Retry()
38 |     mock_geoprocess.side_effect = NotADirectoryError('Random fail!')
39 | 
40 |     # Act/assert
41 |     with raises(TaskError):
42 |         do_task(2, 1)
43 | 


--------------------------------------------------------------------------------
/docker-compose.yml:
--------------------------------------------------------------------------------
 1 | version: "2"
 2 | 
 3 | services:
 4 | 
 5 |   postgis:
 6 |     image: thinkwhere/postgis:geopy
 7 |     environment:
 8 |       - POSTGRES_USER=geopython
 9 |       - POSTGRES_PASSWORD=geopython
10 |       - POSTGRES_DB=geopython-db
11 |     ports:
12 |       - "5435:5432"
13 | 
14 |   rabbitmq:
15 |     # Message broker for Celery
16 |     image: rabbitmq:3.7-management-alpine
17 |     environment:
18 |       - RABBITMQ_DEFAULT_USER=celery_user
19 |       - RABBITMQ_DEFAULT_PASS=secret
20 |       - RABBITMQ_DEFAULT_VHOST=celery_app
21 |     ports:
22 |       # The rabbitMQ management plugin - running on http://localhost:15672
23 |       - "15672:15672"
24 |       - "5672:5672"
25 | 
26 |   celery:
27 |     # Python application which run the Celery worker
28 |     build:
29 |       context: celery_app
30 |       dockerfile: docker/celery.dockerfile
31 |     links:
32 |       - rabbitmq
33 |     depends_on:
34 |       - rabbitmq
35 |       - postgis
36 | 
37 |   flower:
38 |     # Monitoring application - running on http://localhost:5555
39 |     build:
40 |       context: celery_app
41 |       dockerfile: docker/flower.dockerfile
42 |     links:
43 |       - celery
44 |       - rabbitmq
45 |     depends_on:
46 |       - celery
47 |     ports:
48 |       - "5555:5555"
49 | 


--------------------------------------------------------------------------------
/extras/geopython-final.key:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/thinkWhere/Celery-Workshop-GeoPython-2018/355beb3ba38d96cedd7d3680a6a54e60d0c541f8/extras/geopython-final.key


--------------------------------------------------------------------------------
/extras/map.qgs:
--------------------------------------------------------------------------------
  1 | <!DOCTYPE qgis PUBLIC 'http://mrcc.com/qgis.dtd' 'SYSTEM'>
  2 | <qgis projectname="" version="2.18.1">
  3 |   <title></title>
  4 |   <autotransaction active="0"/>
  5 |   <evaluateDefaultValues active="0"/>
  6 |   <layer-tree-group expanded="1" checked="Qt::Checked" name="">
  7 |     <customproperties/>
  8 |     <layer-tree-layer expanded="1" checked="Qt::Checked" id="grid_squares20180405170855595" name="grid_squares">
  9 |       <customproperties>
 10 |         <property key="showFeatureCount" value="1"/>
 11 |       </customproperties>
 12 |     </layer-tree-layer>
 13 |   </layer-tree-group>
 14 |   <relations/>
 15 |   <mapcanvas>
 16 |     <units>meters</units>
 17 |     <extent>
 18 |       <xmin>86771.37079973210347816</xmin>
 19 |       <ymin>225830.19674059213139117</ymin>
 20 |       <xmax>431433.87079973221989349</xmax>
 21 |       <ymax>858948.69674059213139117</ymax>
 22 |     </extent>
 23 |     <rotation>0</rotation>
 24 |     <projections>0</projections>
 25 |     <destinationsrs>
 26 |       <spatialrefsys>
 27 |         <proj4>+proj=tmerc +lat_0=49 +lon_0=-2 +k=0.9996012717 +x_0=400000 +y_0=-100000 +ellps=airy +towgs84=446.448,-125.157,542.06,0.15,0.247,0.842,-20.489 +units=m +no_defs</proj4>
 28 |         <srsid>2437</srsid>
 29 |         <srid>27700</srid>
 30 |         <authid>EPSG:27700</authid>
 31 |         <description>OSGB 1936 / British National Grid</description>
 32 |         <projectionacronym>tmerc</projectionacronym>
 33 |         <ellipsoidacronym>airy</ellipsoidacronym>
 34 |         <geographicflag>false</geographicflag>
 35 |       </spatialrefsys>
 36 |     </destinationsrs>
 37 |     <rendermaptile>0</rendermaptile>
 38 |     <layer_coordinate_transform_info/>
 39 |   </mapcanvas>
 40 |   <layer-tree-canvas>
 41 |     <custom-order enabled="0">
 42 |       <item>grid_squares20180405170855595</item>
 43 |     </custom-order>
 44 |   </layer-tree-canvas>
 45 |   <legend updateDrawingOrder="true">
 46 |     <legendlayer drawingOrder="-1" open="true" checked="Qt::Checked" name="grid_squares" showFeatureCount="1">
 47 |       <filegroup open="true" hidden="false">
 48 |         <legendlayerfile isInOverview="0" layerid="grid_squares20180405170855595" visible="1"/>
 49 |       </filegroup>
 50 |     </legendlayer>
 51 |   </legend>
 52 |   <projectlayers>
 53 |     <maplayer simplifyAlgorithm="0" minimumScale="0" maximumScale="1e+08" simplifyDrawingHints="1" readOnly="0" minLabelScale="0" maxLabelScale="1e+08" simplifyDrawingTol="1" geometry="Polygon" simplifyMaxScale="1" type="vector" hasScaleBasedVisibilityFlag="0" simplifyLocal="1" scaleBasedLabelVisibilityFlag="0">
 54 |       <extent>
 55 |         <xmin>750</xmin>
 56 |         <ymin>16030</ymin>
 57 |         <xmax>657250</xmax>
 58 |         <ymax>1221970</ymax>
 59 |       </extent>
 60 |       <id>grid_squares20180405170855595</id>
 61 |       <datasource>dbname='geopython-db' host=localhost port=5435 sslmode=disable key='id' srid=27700 type=Polygon table="geopython"."grid_squares" (geom) sql=</datasource>
 62 |       <keywordList>
 63 |         <value></value>
 64 |       </keywordList>
 65 |       <layername>grid_squares</layername>
 66 |       <srs>
 67 |         <spatialrefsys>
 68 |           <proj4>+proj=tmerc +lat_0=49 +lon_0=-2 +k=0.9996012717 +x_0=400000 +y_0=-100000 +ellps=airy +towgs84=446.448,-125.157,542.06,0.15,0.247,0.842,-20.489 +units=m +no_defs</proj4>
 69 |           <srsid>2437</srsid>
 70 |           <srid>27700</srid>
 71 |           <authid>EPSG:27700</authid>
 72 |           <description>OSGB 1936 / British National Grid</description>
 73 |           <projectionacronym>tmerc</projectionacronym>
 74 |           <ellipsoidacronym>airy</ellipsoidacronym>
 75 |           <geographicflag>false</geographicflag>
 76 |         </spatialrefsys>
 77 |       </srs>
 78 |       <provider encoding="UTF-8">postgres</provider>
 79 |       <previewExpression></previewExpression>
 80 |       <vectorjoins/>
 81 |       <layerDependencies/>
 82 |       <expressionfields/>
 83 |       <defaults>
 84 |         <default field="id" expression=""/>
 85 |         <default field="grid_ref" expression=""/>
 86 |         <default field="style_cat" expression=""/>
 87 |       </defaults>
 88 |       <map-layer-style-manager current="">
 89 |         <map-layer-style name=""/>
 90 |       </map-layer-style-manager>
 91 |       <edittypes>
 92 |         <edittype widgetv2type="TextEdit" name="id">
 93 |           <widgetv2config IsMultiline="0" fieldEditable="1" constraint="" UseHtml="0" labelOnTop="0" constraintDescription="" notNull="0"/>
 94 |         </edittype>
 95 |         <edittype widgetv2type="TextEdit" name="grid_ref">
 96 |           <widgetv2config IsMultiline="0" fieldEditable="1" constraint="" UseHtml="0" labelOnTop="0" constraintDescription="" notNull="0"/>
 97 |         </edittype>
 98 |         <edittype widgetv2type="TextEdit" name="style_cat">
 99 |           <widgetv2config IsMultiline="0" fieldEditable="1" constraint="" UseHtml="0" labelOnTop="0" constraintDescription="" notNull="0"/>
100 |         </edittype>
101 |       </edittypes>
102 |       <renderer-v2 attr="style_cat" forceraster="0" symbollevels="0" type="categorizedSymbol" enableorderby="0">
103 |         <categories>
104 |           <category render="true" symbol="0" value="ENG" label="ENG"/>
105 |           <category render="true" symbol="1" value="IMN" label="IMN"/>
106 |           <category render="true" symbol="2" value="NIR" label="NIR"/>
107 |           <category render="true" symbol="3" value="SCT" label="SCT"/>
108 |           <category render="true" symbol="4" value="SEA" label="SEA"/>
109 |           <category render="true" symbol="5" value="WLS" label="WLS"/>
110 |         </categories>
111 |         <symbols>
112 |           <symbol alpha="1" clip_to_extent="1" type="fill" name="0">
113 |             <layer pass="0" class="SimpleFill" locked="0">
114 |               <prop k="border_width_map_unit_scale" v="0,0,0,0,0,0"/>
115 |               <prop k="color" v="207,8,31,255"/>
116 |               <prop k="joinstyle" v="bevel"/>
117 |               <prop k="offset" v="0,0"/>
118 |               <prop k="offset_map_unit_scale" v="0,0,0,0,0,0"/>
119 |               <prop k="offset_unit" v="MM"/>
120 |               <prop k="outline_color" v="0,0,0,0"/>
121 |               <prop k="outline_style" v="solid"/>
122 |               <prop k="outline_width" v="0.26"/>
123 |               <prop k="outline_width_unit" v="MM"/>
124 |               <prop k="style" v="solid"/>
125 |             </layer>
126 |           </symbol>
127 |           <symbol alpha="1" clip_to_extent="1" type="fill" name="1">
128 |             <layer pass="0" class="SimpleFill" locked="0">
129 |               <prop k="border_width_map_unit_scale" v="0,0,0,0,0,0"/>
130 |               <prop k="color" v="255,217,1,255"/>
131 |               <prop k="joinstyle" v="bevel"/>
132 |               <prop k="offset" v="0,0"/>
133 |               <prop k="offset_map_unit_scale" v="0,0,0,0,0,0"/>
134 |               <prop k="offset_unit" v="MM"/>
135 |               <prop k="outline_color" v="0,0,0,0"/>
136 |               <prop k="outline_style" v="solid"/>
137 |               <prop k="outline_width" v="0.26"/>
138 |               <prop k="outline_width_unit" v="MM"/>
139 |               <prop k="style" v="solid"/>
140 |             </layer>
141 |           </symbol>
142 |           <symbol alpha="1" clip_to_extent="1" type="fill" name="2">
143 |             <layer pass="0" class="SimpleFill" locked="0">
144 |               <prop k="border_width_map_unit_scale" v="0,0,0,0,0,0"/>
145 |               <prop k="color" v="22,155,98,255"/>
146 |               <prop k="joinstyle" v="bevel"/>
147 |               <prop k="offset" v="0,0"/>
148 |               <prop k="offset_map_unit_scale" v="0,0,0,0,0,0"/>
149 |               <prop k="offset_unit" v="MM"/>
150 |               <prop k="outline_color" v="0,0,0,0"/>
151 |               <prop k="outline_style" v="solid"/>
152 |               <prop k="outline_width" v="0.26"/>
153 |               <prop k="outline_width_unit" v="MM"/>
154 |               <prop k="style" v="solid"/>
155 |             </layer>
156 |           </symbol>
157 |           <symbol alpha="1" clip_to_extent="1" type="fill" name="3">
158 |             <layer pass="0" class="SimpleFill" locked="0">
159 |               <prop k="border_width_map_unit_scale" v="0,0,0,0,0,0"/>
160 |               <prop k="color" v="1,101,191,255"/>
161 |               <prop k="joinstyle" v="bevel"/>
162 |               <prop k="offset" v="0,0"/>
163 |               <prop k="offset_map_unit_scale" v="0,0,0,0,0,0"/>
164 |               <prop k="offset_unit" v="MM"/>
165 |               <prop k="outline_color" v="0,0,0,0"/>
166 |               <prop k="outline_style" v="solid"/>
167 |               <prop k="outline_width" v="0.26"/>
168 |               <prop k="outline_width_unit" v="MM"/>
169 |               <prop k="style" v="solid"/>
170 |             </layer>
171 |           </symbol>
172 |           <symbol alpha="1" clip_to_extent="1" type="fill" name="4">
173 |             <layer pass="0" class="SimpleFill" locked="0">
174 |               <prop k="border_width_map_unit_scale" v="0,0,0,0,0,0"/>
175 |               <prop k="color" v="14,197,200,255"/>
176 |               <prop k="joinstyle" v="bevel"/>
177 |               <prop k="offset" v="0,0"/>
178 |               <prop k="offset_map_unit_scale" v="0,0,0,0,0,0"/>
179 |               <prop k="offset_unit" v="MM"/>
180 |               <prop k="outline_color" v="0,0,0,0"/>
181 |               <prop k="outline_style" v="solid"/>
182 |               <prop k="outline_width" v="0.26"/>
183 |               <prop k="outline_width_unit" v="MM"/>
184 |               <prop k="style" v="solid"/>
185 |             </layer>
186 |           </symbol>
187 |           <symbol alpha="1" clip_to_extent="1" type="fill" name="5">
188 |             <layer pass="0" class="SimpleFill" locked="0">
189 |               <prop k="border_width_map_unit_scale" v="0,0,0,0,0,0"/>
190 |               <prop k="color" v="0,173,54,255"/>
191 |               <prop k="joinstyle" v="bevel"/>
192 |               <prop k="offset" v="0,0"/>
193 |               <prop k="offset_map_unit_scale" v="0,0,0,0,0,0"/>
194 |               <prop k="offset_unit" v="MM"/>
195 |               <prop k="outline_color" v="0,0,0,0"/>
196 |               <prop k="outline_style" v="solid"/>
197 |               <prop k="outline_width" v="0.26"/>
198 |               <prop k="outline_width_unit" v="MM"/>
199 |               <prop k="style" v="solid"/>
200 |             </layer>
201 |           </symbol>
202 |         </symbols>
203 |         <source-symbol>
204 |           <symbol alpha="1" clip_to_extent="1" type="fill" name="0">
205 |             <layer pass="0" class="SimpleFill" locked="0">
206 |               <prop k="border_width_map_unit_scale" v="0,0,0,0,0,0"/>
207 |               <prop k="color" v="112,233,239,255"/>
208 |               <prop k="joinstyle" v="bevel"/>
209 |               <prop k="offset" v="0,0"/>
210 |               <prop k="offset_map_unit_scale" v="0,0,0,0,0,0"/>
211 |               <prop k="offset_unit" v="MM"/>
212 |               <prop k="outline_color" v="0,0,0,0"/>
213 |               <prop k="outline_style" v="solid"/>
214 |               <prop k="outline_width" v="0.26"/>
215 |               <prop k="outline_width_unit" v="MM"/>
216 |               <prop k="style" v="solid"/>
217 |             </layer>
218 |           </symbol>
219 |         </source-symbol>
220 |         <rotation/>
221 |         <sizescale scalemethod="diameter"/>
222 |       </renderer-v2>
223 |       <labeling type="simple"/>
224 |       <customproperties>
225 |         <property key="embeddedWidgets/count" value="0"/>
226 |         <property key="labeling" value="pal"/>
227 |         <property key="labeling/addDirectionSymbol" value="false"/>
228 |         <property key="labeling/angleOffset" value="0"/>
229 |         <property key="labeling/blendMode" value="0"/>
230 |         <property key="labeling/bufferBlendMode" value="0"/>
231 |         <property key="labeling/bufferColorA" value="255"/>
232 |         <property key="labeling/bufferColorB" value="255"/>
233 |         <property key="labeling/bufferColorG" value="255"/>
234 |         <property key="labeling/bufferColorR" value="255"/>
235 |         <property key="labeling/bufferDraw" value="false"/>
236 |         <property key="labeling/bufferJoinStyle" value="128"/>
237 |         <property key="labeling/bufferNoFill" value="false"/>
238 |         <property key="labeling/bufferSize" value="1"/>
239 |         <property key="labeling/bufferSizeInMapUnits" value="false"/>
240 |         <property key="labeling/bufferSizeMapUnitScale" value="0,0,0,0,0,0"/>
241 |         <property key="labeling/bufferTransp" value="0"/>
242 |         <property key="labeling/centroidInside" value="false"/>
243 |         <property key="labeling/centroidWhole" value="false"/>
244 |         <property key="labeling/decimals" value="3"/>
245 |         <property key="labeling/displayAll" value="false"/>
246 |         <property key="labeling/dist" value="0"/>
247 |         <property key="labeling/distInMapUnits" value="false"/>
248 |         <property key="labeling/distMapUnitScale" value="0,0,0,0,0,0"/>
249 |         <property key="labeling/drawLabels" value="false"/>
250 |         <property key="labeling/enabled" value="false"/>
251 |         <property key="labeling/fieldName" value=""/>
252 |         <property key="labeling/fitInPolygonOnly" value="false"/>
253 |         <property key="labeling/fontCapitals" value="0"/>
254 |         <property key="labeling/fontFamily" value="MS Shell Dlg 2"/>
255 |         <property key="labeling/fontItalic" value="false"/>
256 |         <property key="labeling/fontLetterSpacing" value="0"/>
257 |         <property key="labeling/fontLimitPixelSize" value="false"/>
258 |         <property key="labeling/fontMaxPixelSize" value="10000"/>
259 |         <property key="labeling/fontMinPixelSize" value="3"/>
260 |         <property key="labeling/fontSize" value="8.25"/>
261 |         <property key="labeling/fontSizeInMapUnits" value="false"/>
262 |         <property key="labeling/fontSizeMapUnitScale" value="0,0,0,0,0,0"/>
263 |         <property key="labeling/fontStrikeout" value="false"/>
264 |         <property key="labeling/fontUnderline" value="false"/>
265 |         <property key="labeling/fontWeight" value="50"/>
266 |         <property key="labeling/fontWordSpacing" value="0"/>
267 |         <property key="labeling/formatNumbers" value="false"/>
268 |         <property key="labeling/isExpression" value="true"/>
269 |         <property key="labeling/labelOffsetInMapUnits" value="true"/>
270 |         <property key="labeling/labelOffsetMapUnitScale" value="0,0,0,0,0,0"/>
271 |         <property key="labeling/labelPerPart" value="false"/>
272 |         <property key="labeling/leftDirectionSymbol" value="&lt;"/>
273 |         <property key="labeling/limitNumLabels" value="false"/>
274 |         <property key="labeling/maxCurvedCharAngleIn" value="25"/>
275 |         <property key="labeling/maxCurvedCharAngleOut" value="-25"/>
276 |         <property key="labeling/maxNumLabels" value="2000"/>
277 |         <property key="labeling/mergeLines" value="false"/>
278 |         <property key="labeling/minFeatureSize" value="0"/>
279 |         <property key="labeling/multilineAlign" value="4294967295"/>
280 |         <property key="labeling/multilineHeight" value="1"/>
281 |         <property key="labeling/namedStyle" value="Normal"/>
282 |         <property key="labeling/obstacle" value="true"/>
283 |         <property key="labeling/obstacleFactor" value="1"/>
284 |         <property key="labeling/obstacleType" value="0"/>
285 |         <property key="labeling/offsetType" value="0"/>
286 |         <property key="labeling/placeDirectionSymbol" value="0"/>
287 |         <property key="labeling/placement" value="1"/>
288 |         <property key="labeling/placementFlags" value="10"/>
289 |         <property key="labeling/plussign" value="false"/>
290 |         <property key="labeling/predefinedPositionOrder" value="TR,TL,BR,BL,R,L,TSR,BSR"/>
291 |         <property key="labeling/preserveRotation" value="true"/>
292 |         <property key="labeling/previewBkgrdColor" value="#ffffff"/>
293 |         <property key="labeling/priority" value="5"/>
294 |         <property key="labeling/quadOffset" value="4"/>
295 |         <property key="labeling/repeatDistance" value="0"/>
296 |         <property key="labeling/repeatDistanceMapUnitScale" value="0,0,0,0,0,0"/>
297 |         <property key="labeling/repeatDistanceUnit" value="1"/>
298 |         <property key="labeling/reverseDirectionSymbol" value="false"/>
299 |         <property key="labeling/rightDirectionSymbol" value=">"/>
300 |         <property key="labeling/scaleMax" value="10000000"/>
301 |         <property key="labeling/scaleMin" value="1"/>
302 |         <property key="labeling/scaleVisibility" value="false"/>
303 |         <property key="labeling/shadowBlendMode" value="6"/>
304 |         <property key="labeling/shadowColorB" value="0"/>
305 |         <property key="labeling/shadowColorG" value="0"/>
306 |         <property key="labeling/shadowColorR" value="0"/>
307 |         <property key="labeling/shadowDraw" value="false"/>
308 |         <property key="labeling/shadowOffsetAngle" value="135"/>
309 |         <property key="labeling/shadowOffsetDist" value="1"/>
310 |         <property key="labeling/shadowOffsetGlobal" value="true"/>
311 |         <property key="labeling/shadowOffsetMapUnitScale" value="0,0,0,0,0,0"/>
312 |         <property key="labeling/shadowOffsetUnits" value="1"/>
313 |         <property key="labeling/shadowRadius" value="1.5"/>
314 |         <property key="labeling/shadowRadiusAlphaOnly" value="false"/>
315 |         <property key="labeling/shadowRadiusMapUnitScale" value="0,0,0,0,0,0"/>
316 |         <property key="labeling/shadowRadiusUnits" value="1"/>
317 |         <property key="labeling/shadowScale" value="100"/>
318 |         <property key="labeling/shadowTransparency" value="30"/>
319 |         <property key="labeling/shadowUnder" value="0"/>
320 |         <property key="labeling/shapeBlendMode" value="0"/>
321 |         <property key="labeling/shapeBorderColorA" value="255"/>
322 |         <property key="labeling/shapeBorderColorB" value="128"/>
323 |         <property key="labeling/shapeBorderColorG" value="128"/>
324 |         <property key="labeling/shapeBorderColorR" value="128"/>
325 |         <property key="labeling/shapeBorderWidth" value="0"/>
326 |         <property key="labeling/shapeBorderWidthMapUnitScale" value="0,0,0,0,0,0"/>
327 |         <property key="labeling/shapeBorderWidthUnits" value="1"/>
328 |         <property key="labeling/shapeDraw" value="false"/>
329 |         <property key="labeling/shapeFillColorA" value="255"/>
330 |         <property key="labeling/shapeFillColorB" value="255"/>
331 |         <property key="labeling/shapeFillColorG" value="255"/>
332 |         <property key="labeling/shapeFillColorR" value="255"/>
333 |         <property key="labeling/shapeJoinStyle" value="64"/>
334 |         <property key="labeling/shapeOffsetMapUnitScale" value="0,0,0,0,0,0"/>
335 |         <property key="labeling/shapeOffsetUnits" value="1"/>
336 |         <property key="labeling/shapeOffsetX" value="0"/>
337 |         <property key="labeling/shapeOffsetY" value="0"/>
338 |         <property key="labeling/shapeRadiiMapUnitScale" value="0,0,0,0,0,0"/>
339 |         <property key="labeling/shapeRadiiUnits" value="1"/>
340 |         <property key="labeling/shapeRadiiX" value="0"/>
341 |         <property key="labeling/shapeRadiiY" value="0"/>
342 |         <property key="labeling/shapeRotation" value="0"/>
343 |         <property key="labeling/shapeRotationType" value="0"/>
344 |         <property key="labeling/shapeSVGFile" value=""/>
345 |         <property key="labeling/shapeSizeMapUnitScale" value="0,0,0,0,0,0"/>
346 |         <property key="labeling/shapeSizeType" value="0"/>
347 |         <property key="labeling/shapeSizeUnits" value="1"/>
348 |         <property key="labeling/shapeSizeX" value="0"/>
349 |         <property key="labeling/shapeSizeY" value="0"/>
350 |         <property key="labeling/shapeTransparency" value="0"/>
351 |         <property key="labeling/shapeType" value="0"/>
352 |         <property key="labeling/substitutions" value="&lt;substitutions/>"/>
353 |         <property key="labeling/textColorA" value="255"/>
354 |         <property key="labeling/textColorB" value="0"/>
355 |         <property key="labeling/textColorG" value="0"/>
356 |         <property key="labeling/textColorR" value="0"/>
357 |         <property key="labeling/textTransp" value="0"/>
358 |         <property key="labeling/upsidedownLabels" value="0"/>
359 |         <property key="labeling/useSubstitutions" value="false"/>
360 |         <property key="labeling/wrapChar" value=""/>
361 |         <property key="labeling/xOffset" value="0"/>
362 |         <property key="labeling/yOffset" value="0"/>
363 |         <property key="labeling/zIndex" value="0"/>
364 |         <property key="variableNames"/>
365 |         <property key="variableValues"/>
366 |       </customproperties>
367 |       <blendMode>0</blendMode>
368 |       <featureBlendMode>0</featureBlendMode>
369 |       <layerTransparency>0</layerTransparency>
370 |       <displayfield>id</displayfield>
371 |       <label>0</label>
372 |       <labelattributes>
373 |         <label fieldname="" text="Label"/>
374 |         <family fieldname="" name="MS Shell Dlg 2"/>
375 |         <size fieldname="" units="pt" value="12"/>
376 |         <bold fieldname="" on="0"/>
377 |         <italic fieldname="" on="0"/>
378 |         <underline fieldname="" on="0"/>
379 |         <strikeout fieldname="" on="0"/>
380 |         <color fieldname="" red="0" blue="0" green="0"/>
381 |         <x fieldname=""/>
382 |         <y fieldname=""/>
383 |         <offset x="0" y="0" units="pt" yfieldname="" xfieldname=""/>
384 |         <angle fieldname="" value="0" auto="0"/>
385 |         <alignment fieldname="" value="center"/>
386 |         <buffercolor fieldname="" red="255" blue="255" green="255"/>
387 |         <buffersize fieldname="" units="pt" value="1"/>
388 |         <bufferenabled fieldname="" on=""/>
389 |         <multilineenabled fieldname="" on=""/>
390 |         <selectedonly on=""/>
391 |       </labelattributes>
392 |       <SingleCategoryDiagramRenderer diagramType="Histogram" sizeLegend="0" attributeLegend="1">
393 |         <DiagramCategory penColor="#000000" labelPlacementMethod="XHeight" penWidth="0" diagramOrientation="Up" sizeScale="0,0,0,0,0,0" minimumSize="0" barWidth="5" penAlpha="255" maxScaleDenominator="1e+08" backgroundColor="#ffffff" transparency="0" width="15" scaleDependency="Area" backgroundAlpha="255" angleOffset="1440" scaleBasedVisibility="0" enabled="0" height="15" lineSizeScale="0,0,0,0,0,0" sizeType="MM" lineSizeType="MM" minScaleDenominator="inf">
394 |           <fontProperties description="MS Shell Dlg 2,8.25,-1,5,50,0,0,0,0,0" style=""/>
395 |           <attribute field="" color="#000000" label=""/>
396 |         </DiagramCategory>
397 |         <symbol alpha="1" clip_to_extent="1" type="marker" name="sizeSymbol">
398 |           <layer pass="0" class="SimpleMarker" locked="0">
399 |             <prop k="angle" v="0"/>
400 |             <prop k="color" v="255,0,0,255"/>
401 |             <prop k="horizontal_anchor_point" v="1"/>
402 |             <prop k="joinstyle" v="bevel"/>
403 |             <prop k="name" v="circle"/>
404 |             <prop k="offset" v="0,0"/>
405 |             <prop k="offset_map_unit_scale" v="0,0,0,0,0,0"/>
406 |             <prop k="offset_unit" v="MM"/>
407 |             <prop k="outline_color" v="0,0,0,255"/>
408 |             <prop k="outline_style" v="solid"/>
409 |             <prop k="outline_width" v="0"/>
410 |             <prop k="outline_width_map_unit_scale" v="0,0,0,0,0,0"/>
411 |             <prop k="outline_width_unit" v="MM"/>
412 |             <prop k="scale_method" v="diameter"/>
413 |             <prop k="size" v="2"/>
414 |             <prop k="size_map_unit_scale" v="0,0,0,0,0,0"/>
415 |             <prop k="size_unit" v="MM"/>
416 |             <prop k="vertical_anchor_point" v="1"/>
417 |           </layer>
418 |         </symbol>
419 |       </SingleCategoryDiagramRenderer>
420 |       <DiagramLayerSettings yPosColumn="-1" showColumn="-1" linePlacementFlags="10" placement="0" dist="0" xPosColumn="-1" priority="0" obstacle="0" zIndex="0" showAll="1"/>
421 |       <annotationform>.</annotationform>
422 |       <aliases>
423 |         <alias field="id" index="0" name=""/>
424 |         <alias field="grid_ref" index="1" name=""/>
425 |         <alias field="style_cat" index="2" name=""/>
426 |       </aliases>
427 |       <excludeAttributesWMS/>
428 |       <excludeAttributesWFS/>
429 |       <attributeactions default="-1"/>
430 |       <attributetableconfig actionWidgetStyle="dropDown" sortExpression="" sortOrder="-1">
431 |         <columns>
432 |           <column width="-1" hidden="0" type="field" name="id"/>
433 |           <column width="-1" hidden="0" type="field" name="grid_ref"/>
434 |           <column width="-1" hidden="0" type="field" name="style_cat"/>
435 |           <column width="-1" hidden="1" type="actions"/>
436 |         </columns>
437 |       </attributetableconfig>
438 |       <editform>.</editform>
439 |       <editforminit/>
440 |       <editforminitcodesource>0</editforminitcodesource>
441 |       <editforminitfilepath>.</editforminitfilepath>
442 |       <editforminitcode><![CDATA[# -*- coding: utf-8 -*-
443 | """
444 | QGIS forms can have a Python function that is called when the form is
445 | opened.
446 | 
447 | Use this function to add extra logic to your forms.
448 | 
449 | Enter the name of the function in the "Python Init function"
450 | field.
451 | An example follows:
452 | """
453 | from qgis.PyQt.QtWidgets import QWidget
454 | 
455 | def my_form_open(dialog, layer, feature):
456 | 	geom = feature.geometry()
457 | 	control = dialog.findChild(QWidget, "MyLineEdit")
458 | ]]></editforminitcode>
459 |       <featformsuppress>0</featformsuppress>
460 |       <editorlayout>generatedlayout</editorlayout>
461 |       <widgets/>
462 |       <conditionalstyles>
463 |         <rowstyles/>
464 |         <fieldstyles/>
465 |       </conditionalstyles>
466 |     </maplayer>
467 |   </projectlayers>
468 |   <properties>
469 |     <Measurement>
470 |       <DistanceUnits type="QString"></DistanceUnits>
471 |       <AreaUnits type="QString"></AreaUnits>
472 |     </Measurement>
473 |     <SpatialRefSys>
474 |       <ProjectCRSProj4String type="QString">+proj=longlat +datum=WGS84 +no_defs</ProjectCRSProj4String>
475 |       <ProjectCrs type="QString">EPSG:4326</ProjectCrs>
476 |       <ProjectCRSID type="int">3452</ProjectCRSID>
477 |     </SpatialRefSys>
478 |     <Paths>
479 |       <Absolute type="bool">false</Absolute>
480 |     </Paths>
481 |     <Gui>
482 |       <SelectionColorBluePart type="int">0</SelectionColorBluePart>
483 |       <CanvasColorGreenPart type="int">255</CanvasColorGreenPart>
484 |       <CanvasColorRedPart type="int">255</CanvasColorRedPart>
485 |       <SelectionColorRedPart type="int">255</SelectionColorRedPart>
486 |       <SelectionColorAlphaPart type="int">255</SelectionColorAlphaPart>
487 |       <SelectionColorGreenPart type="int">255</SelectionColorGreenPart>
488 |       <CanvasColorBluePart type="int">255</CanvasColorBluePart>
489 |     </Gui>
490 |     <Digitizing>
491 |       <DefaultSnapToleranceUnit type="int">2</DefaultSnapToleranceUnit>
492 |       <SnappingMode type="QString">current_layer</SnappingMode>
493 |       <DefaultSnapType type="QString">off</DefaultSnapType>
494 |       <DefaultSnapTolerance type="double">0</DefaultSnapTolerance>
495 |     </Digitizing>
496 |     <PositionPrecision>
497 |       <DecimalPlaces type="int">2</DecimalPlaces>
498 |       <Automatic type="bool">true</Automatic>
499 |     </PositionPrecision>
500 |     <Legend>
501 |       <filterByMap type="bool">false</filterByMap>
502 |     </Legend>
503 |   </properties>
504 |   <visibility-presets/>
505 | </qgis>
506 | 


--------------------------------------------------------------------------------