├── LICENSE
├── README.md
├── assets
├── cbf_cf.png
├── deepmind_forgoogle_recsys.png
├── mf.png
├── netflix_prize.jpeg
├── recsys_io.png
├── retrieval_ranking.png
├── tensorflow_two_tower.gif
└── youtube_retrieval.png
└── recommender_system_tutorial.ipynb
/LICENSE:
--------------------------------------------------------------------------------
1 | MIT License
2 |
3 | Copyright (c) 2020 Hamidreza Hosseinkhani
4 |
5 | Permission is hereby granted, free of charge, to any person obtaining a copy
6 | of this software and associated documentation files (the "Software"), to deal
7 | in the Software without restriction, including without limitation the rights
8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9 | copies of the Software, and to permit persons to whom the Software is
10 | furnished to do so, subject to the following conditions:
11 |
12 | The above copyright notice and this permission notice shall be included in all
13 | copies or substantial portions of the Software.
14 |
15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21 | SOFTWARE.
22 |
--------------------------------------------------------------------------------
/README.md:
--------------------------------------------------------------------------------
1 | # Build a recommendation system with TensorFlow and Keras
2 | It is a step-by-step tutorial on developing a practical recommendation system (`retrieval` and `ranking` tasks) using [TensorFlow Recommenders](https://www.tensorflow.org/recommenders) and [Keras](https://keras.io/) and deploy it using [TensorFlow Serving](https://www.tensorflow.org/tfx/guide/serving).
3 |
4 | Here, you can find an introduction to the information retrieval and the recommendation systems, then you can explore [the Jupyter notebook](https://github.com/xei/recommender_system_tutorial/blob/main/recommender_system_tutorial.ipynb) and run it in [Google Colab](https://colab.research.google.com/github/xei/recommender_system_tutorial/blob/main/recommender_system_tutorial.ipynb) in order to study the code.
5 |
6 | [](https://colab.research.google.com/github/xei/recommender_system_tutorial/blob/main/recommender_system_tutorial.ipynb)
7 |
8 |
9 |
10 |
11 | In the notebook, we load [MovieLens dataset](https://grouplens.org/datasets/movielens/) using [TesorFlow Datasets](https://www.tensorflow.org/datasets), preprocess its features using [Keras preprocessing layers](https://keras.io/guides/preprocessing_layers/), build the `retrieval` and `ranking` tasks using [TensorFlow Recommenders](https://www.tensorflow.org/recommenders) and index and search for similar items using [Spotify Annoy](https://github.com/spotify/annoy).
12 |
13 | This tutorial is recommended to both academic and industry enthusiasts.
14 |
15 | ## Introduction
16 |
17 | ### What is a recommendation system?!
18 | Online services usually provides thousands, millions or even billions of items like products, video clips, movies, musics, news, articles, blog posts, advertisements, etc. For example, the [Google Play Store](https://play.google.com/store) provides millions of apps and [YouTube](https://www.youtube.com/) provides billions of videos. [[1]](https://developers.google.com/machine-learning/recommendation/overview)
19 |
20 | However, users prefer to see a handful shortlist of likely items instead of struggling with the full corpora. They usually can search or filter the list to find the best handful items, but sometimes they even don't know what they really want (e.g. a birthday gift). In a physical store an expert seller would help in this case by useful recommendations. So, why not in an online store?!
21 |
22 | A recommedation system can retrieve, filter and recommend best personalized results for the user - results which the user is likely to buy. So it is one of the major requirements of modern businesses in order to increase their `conversion rate`. On September 21, 2009, Netflix gave a grand prize of $1,000,000 to a team which bested Netflix's own algorithm for predicting ratings by 10.06%. [[2]](https://web.archive.org/web/20090924184639/http://www.netflixprize.com/community/viewtopic.php?id=1537)
23 |
24 |
25 |
26 |
27 |
28 | A recommendation system ia a system that gives a `query (context)` which is what we know about the liking list, and filter the corpus (full catalog of items) to a shortlist of `candidates` (items, documents). A query (context) can be a ***user id***, ***user's geographical location*** or ***user's history of previous purchases*** and the resulting candidates can be some new items that we guess are interesting for the user.
29 |
30 | The query can also be an ***item id***, ***its image*** or ***its textual description*** and the candidates can be some similar or related items from the corpus.
31 |
32 |
33 |
34 |
35 |
36 |
37 | ### Recommendation stages (tasks)
38 | In practice, dealing with a large corpus and filter it to a shortlist is an intractable and inefficient task. So practical recommender systems has two (or three) filterng phases:
39 | 1. Retrieval (Candidate Generation)
40 | 2. Ranking (Scoring)
41 | 3. Re-ranking or optimazation or ...
42 |
43 |
44 |
45 |
46 |
47 |
48 |
49 |
50 |
51 | ### Content-based Filtering vs Collaborative Filtering
52 | Filtering items is based on similarities. we can filter the list based on similar candidates (`content-based filtering`) or based on the similarity between queries and candidates (`collaborative filtering`). Collaborative filtering algorithms usually perform better than content-based methods.
53 |
54 |
55 |
56 |
57 |
58 | ### Representation of a query or a candidate
59 | A query or a candidate has lots of different features. For example a query can be constructed by these features:
60 | - user_id
61 | - user_previous_history
62 | - user_job
63 | - etc.
64 |
65 | And a candidate can have features like:
66 | - item_description
67 | - item_image
68 | - item_price
69 | - posted_time
70 | - etc.
71 |
72 | These obviouse features can be `numerical variables`, `categorical variables`, `bitmaps` or `raw texts`. However, these low-level features are not enough and we should extract some more abstract `latent features` from these obvious features to represent the query or the candidate as a numerical high-dimensional vector - known as `Embedding Vector`.
73 |
74 | `Matrix Factorization` (MF) is a classic collaborative filtering method to learn some `latent factors` (latent features) from `user_id`, `item_id` and `rating` features and represent **users** and **items** by latent (embedding) vectors.
75 |
76 |
77 |
78 |
79 |
80 | Matrix Factorization method only uses `user_id` and `candidate_id` features collaboratively to learn the `latent features`. In fact it doesn't care about other side-features like `candidate_description`, `price`, `user_comment`, etc.
81 |
82 | To involve side-features as well as ids while learning latent features (embeddings), we can use deep neural network (DNN) architectures like `softmax` or `two-tower` neural models.
83 |
84 |
85 |
86 |
87 |
88 | YouTube two-tower neural model uses side-features to represent queries and candidates in an abstract high-dimentional embedding vector.
89 |
90 |
91 |
92 |
93 |
94 | ### Movielens dataset
95 |
96 | The `Movielens` dataset is a benchmark dataset in the field of recommender system research containing a set of *ratings* given to *movies* by a set of *users*, collected from the [MovieLens website](http://movielens.org/) - a movie recommendation service.
97 |
98 | There are 5 different versions of Movielens available for different purposes: "25m", "latest-small", "100k", "1m" and "20m". In this tutorial we are going to work with "100k" version. For more information about different versions visit the [official website](https://grouplens.org/datasets/movielens/).
99 |
100 | **movielens/100k-ratings**
101 |
102 | The oldest version of the MovieLens dataset containing 100,000 ratings from 943 users on 1,682 movies. Each user has rated at least 20 movies. Ratings are in whole-star increments. This dataset contains demographic data of users in addition to data on movies and ratings.
103 |
104 | **movielens/100k-movies**
105 |
106 | This dataset contains data of 1,682 movies rated in the ***movielens/100k-ratings*** dataset.
107 |
108 |
\n",
35 | "\n",
36 | "Here, we are going to learn the fundamentals of information retrieval and recommendation systems and build a practical movie recommender service using [TensorFlow Recommenders](https://www.tensorflow.org/recommenders) and [Keras](https://keras.io/) and deploy it using [TensorFlow Serving](https://www.tensorflow.org/tfx/guide/serving).\n",
37 | "\n",
38 | "This step-by-step tutorial is recommended to both academia and industry enthusiasts."
39 | ]
40 | },
41 | {
42 | "cell_type": "markdown",
43 | "metadata": {
44 | "id": "JR7yjVX9UUxs"
45 | },
46 | "source": [
47 | "## Introduction\n",
48 | "\n",
49 | "### What is a recommendation system?!\n",
50 | "Online services usually provide thousands, millions or even billions of items like products, video clips, movies, musics, news, articles, blog posts, advertisements, etc. For example, the [Google Play Store](https://play.google.com/store) provides millions of apps and [YouTube](https://www.youtube.com/) provides billions of videos. [[1]](https://developers.google.com/machine-learning/recommendation/overview)\n",
51 | "\n",
52 | "However, users prefer to see a handful shortlist of likely items instead of struggling with the full corpora. They usually can search or filter the list to find the best handful items, but sometimes they even don't know what they really want (e.g. a birthday gift). In a physical store an expert seller would help in this case by useful recommendations. So, why not in an online store?!\n",
53 | "\n",
54 | "A recommedation system can retrieve, filter and recommend best personalized results for the user - results which the user is likely to buy. So it is one of the major requirements of modern businesses in order to increase their `conversion rate`. On September 21, 2009, Netflix gave a grand prize of $1,000,000 to a team which bested Netflix's own algorithm for predicting ratings by 10.06%. [[2]](https://web.archive.org/web/20090924184639/http://www.netflixprize.com/community/viewtopic.php?id=1537)\n",
55 | "\n",
56 | "
\n",
57 | " \n",
58 | "
\n",
59 | "\n",
60 | "A recommendation system ia a system that gives a `query (context)` which is what we know about the liking list, and filter the corpus (full catalog of items) to a shortlist of `candidates` (items, documents). A query (context) can be a combination of ***user id***, ***user's geographical location***, ***user's history of previous purchases***, etc and the resulting candidates can be some new items that we guess are interesting for the user.\n",
61 | "\n",
62 | "The query can also be a combination of ***item id***, ***its image***, ***its textual description***, etc and the candidates can be some similar or related items from the corpus.\n",
63 | "
\n",
64 | "
\n",
65 | " \n",
66 | "
\n",
67 | "
\n",
68 | "\n",
69 | "### Recommendation stages (tasks)\n",
70 | "In practice, dealing with a large corpus and filter it to a shortlist is an intractable and inefficient task. So practical recommender systems has two (or three) filterng phases:\n",
71 | "1. Retrieval (Candidate Generation)\n",
72 | "2. Ranking (Scoring)\n",
73 | "3. Re-ranking or optimazation or ...\n",
74 | "\n",
75 | "
\n",
76 | " \n",
77 | "
\n",
78 | "\n",
79 | "
\n",
80 | " \n",
81 | "
\n",
82 | "\n",
83 | "### Content-based Filtering vs Collaborative Filtering\n",
84 | "Filtering items is based on similarities. we can filter the list based on similar candidates (`content-based filtering`) or based on the similarity between queries and candidates (`collaborative filtering`). Collaborative filtering algorithms usually perform better than content-based methods.\n",
85 | "\n",
86 | "
\n",
87 | " \n",
88 | "
\n",
89 | "\n",
90 | "### Representation of a query or a candidate\n",
91 | "A query or a candidate has lots of different features. For example a query can be constructed by these features:\n",
92 | "- user_id\n",
93 | "- user_previous_history\n",
94 | "- user_job\n",
95 | "- etc.\n",
96 | "\n",
97 | "And a candidate can have features like:\n",
98 | "- item_description\n",
99 | "- item_image\n",
100 | "- item_price\n",
101 | "- posted_time\n",
102 | "- etc.\n",
103 | "\n",
104 | "These obviouse features can be `numerical variables`, `categorical variables`, `bitmaps` or `raw texts`. However, these low-level features are not enough for any prediction and we should extract some more abstract `latent features` from these obvious features to represent the query or the candidate as a numerical high-dimensional vector - known as `Embedding Vector`.\n",
105 | "\n",
106 | "`Matrix Factorization` (MF) is a classic collaborative filtering method to learn some `latent factors` (latent features) from `user_id`, `item_id` and `rating` features and represent **users** and **items** by latent (embedding) vectors.\n",
107 | "\n",
108 | "
\n",
109 | " \n",
110 | "
\n",
111 | "\n",
112 | "Matrix Factorization method only uses `user_id` and `candidate_id` features collaboratively to learn the `latent features`. In fact it doesn't care about other side-features like `candidate_description`, `price`, `user_comment`, etc.\n",
113 | "\n",
114 | "To involve side-features as well as ids while learning latent features (embeddings), we can use deep neural network (DNN) architectures like `softmax` or `two-tower` neural models.\n",
115 | "\n",
116 | "
\n",
117 | " \n",
118 | "
\n",
119 | "\n",
120 | "YouTube two-tower neural model uses side-features to represent queries and candidates in an abstract high-dimentional embedding vector.\n",
121 | "\n",
122 | "
\n",
123 | " \n",
124 | "
\n",
125 | "\n",
126 | "### Movielens dataset\n",
127 | "\n",
128 | "The `Movielens` dataset is a benchmark dataset in the field of recommender system research containing a set of *ratings* given to *movies* by a set of *users*, collected from the [MovieLens website](http://movielens.org/) - a movie recommendation service.\n",
129 | "\n",
130 | "There are 5 different versions of Movielens available for different purposes: \"25m\", \"latest-small\", \"100k\", \"1m\" and \"20m\". In this tutorial we are going to work with \"100k\" version. For more information about different versions visit the [official website](https://grouplens.org/datasets/movielens/).\n",
131 | "\n",
132 | "**movielens/100k-ratings**\n",
133 | "\n",
134 | "The oldest version of the MovieLens dataset containing 100,000 ratings from 943 users on 1,682 movies. Each user has rated at least 20 movies. Ratings are in whole-star increments. This dataset contains demographic data of users in addition to data on movies and ratings.\n",
135 | "\n",
136 | "**movielens/100k-movies**\n",
137 | "\n",
138 | "This dataset contains data of 1,682 movies rated in the ***movielens/100k-ratings*** dataset."
139 | ]
140 | },
141 | {
142 | "cell_type": "markdown",
143 | "metadata": {
144 | "id": "NDrq_pdbB6Pd"
145 | },
146 | "source": [
147 | "## Make an input pipeline with TensorFlow Datasets\n",
148 | "TensorFlow Datasets ([TFDS library](https://www.tensorflow.org/datasets/overview)) provides a collection of ready-to-use datasets for use with TensorFlow, Jax, and other Machine Learning frameworks.\n",
149 | "\n",
150 | "It handles downloading and preparing the data deterministically and constructing a `tf.data.Dataset` (or `np.array`) in order to enabling easy-to-use and high-performance input pipelines."
151 | ]
152 | },
153 | {
154 | "cell_type": "code",
155 | "source": [
156 | "import os\n",
157 | "os.environ['TF_USE_LEGACY_KERAS'] = '1'"
158 | ],
159 | "metadata": {
160 | "id": "NLu_As-ji2o0"
161 | },
162 | "execution_count": 1,
163 | "outputs": []
164 | },
165 | {
166 | "cell_type": "code",
167 | "metadata": {
168 | "id": "Sciomsbda9NT"
169 | },
170 | "source": [
171 | "# Make sure the latest version of TFDS library is installed.\n",
172 | "# Older versions are not supported here.\n",
173 | "!pip install -q --upgrade tensorflow-datasets\n",
174 | "\n",
175 | "import tensorflow_datasets as tfds"
176 | ],
177 | "execution_count": 2,
178 | "outputs": []
179 | },
180 | {
181 | "cell_type": "code",
182 | "metadata": {
183 | "id": "IWY6OFnhLQGx",
184 | "colab": {
185 | "base_uri": "https://localhost:8080/"
186 | },
187 | "outputId": "faeb623c-3ae3-41fd-cd61-5328c457a8f5"
188 | },
189 | "source": [
190 | "# Download the data, save them as `tfrecord` files, load the `tfrecord` files\n",
191 | "# and create the `tf.data.Dataset` object containing the dataset.\n",
192 | "ratings_dataset, ratings_dataset_info = tfds.load(\n",
193 | " name='movielens/latest-small-ratings',\n",
194 | " # MovieLens dataset is not splitted into `train` and `test` sets by default.\n",
195 | " # So TFDS has put it all into `train` split. We load it completely and split\n",
196 | " # it manually.\n",
197 | " split='train',\n",
198 | " # `with_info=True` makes the `load` function return a `tfds.core.DatasetInfo`\n",
199 | " # object containing dataset metadata like version, description, homepage,\n",
200 | " # citation, etc.\n",
201 | " with_info=True\n",
202 | ")\n",
203 | "\n",
204 | "# Calling the `tfds.load()` function in old versions of TFDS won't return an\n",
205 | "# instance of `tf.data.Dataset` type. So we can make sure about it.\n",
206 | "import tensorflow as tf\n",
207 | "assert isinstance(ratings_dataset, tf.data.Dataset)\n",
208 | "\n",
209 | "print(\n",
210 | " \"ratings_dataset size: %d\" % ratings_dataset.__len__()\n",
211 | ")\n",
212 | "\n",
213 | "# Use `tfds.as_dataframe()` to convert `tf.data.Dataset` to `pandas.DataFrame`.\n",
214 | "# Add the `tfds.core.DatasetInfo` as second argument of `tfds.as_dataframe` to\n",
215 | "# visualize images, audio, texts, videos, etc. `pandas.DataFrame` will load the\n",
216 | "# full dataset in-memory, and can be very expensive to display. So use it only\n",
217 | "# with take() function.\n",
218 | "print(\n",
219 | " tfds.as_dataframe(ratings_dataset.take(5), ratings_dataset_info)\n",
220 | ")\n",
221 | "\n",
222 | "## Feature selection\n",
223 | "ratings_dataset = ratings_dataset.map(\n",
224 | " lambda rating: {\n",
225 | " # `user_id` is useful as a user identifier.\n",
226 | " 'user_id': rating['user_id'],\n",
227 | " # `movie_id` is useful as a movie identifier.\n",
228 | " 'movie_id': rating['movie_id'],\n",
229 | " # `movie_title` is useful as a textual information about the movie.\n",
230 | " 'movie_title': rating['movie_title'],\n",
231 | " # `user_rating` shows the user's level of interest to a movie.\n",
232 | " 'user_rating': rating['user_rating'],\n",
233 | " # `timestamp` will allow us to model the effect of time.\n",
234 | " 'timestamp': rating['timestamp']\n",
235 | " }\n",
236 | ")\n",
237 | "\n",
238 | "print(\n",
239 | " tfds.as_dataframe(ratings_dataset.take(5), ratings_dataset_info)\n",
240 | ")\n",
241 | "\n",
242 | "## Split dataset randomly (80% for train and 20% for test)\n",
243 | "trainset_size = 0.8 * ratings_dataset.__len__().numpy()\n",
244 | "# In an industrial recommender system, this would most likely be done by time:\n",
245 | "# The data up to time T would be used to predict interactions after T.\n",
246 | "\n",
247 | "# set the global seed:\n",
248 | "tf.random.set_seed(42)\n",
249 | "# More info: https://www.tensorflow.org/api_docs/python/tf/random/set_seed\n",
250 | "\n",
251 | "# Shuffle the elements of the dataset randomly.\n",
252 | "ratings_dataset_shuffled = ratings_dataset.shuffle(\n",
253 | " # the new dataset will be sampled from a buffer window of first `buffer_size`\n",
254 | " # elements of the dataset\n",
255 | " buffer_size=100_000,\n",
256 | " # set the random seed that will be used to create the distribution.\n",
257 | " seed=42,\n",
258 | " # `list(dataset.as_numpy_iterator()` yields different result for each call\n",
259 | " # Because reshuffle_each_iteration defaults to True.\n",
260 | " reshuffle_each_iteration=False\n",
261 | ")\n",
262 | "# More info: https://www.tensorflow.org/api_docs/python/tf/data/Dataset#shuffle\n",
263 | "\n",
264 | "ratings_trainset = ratings_dataset_shuffled.take(trainset_size)\n",
265 | "ratings_testset = ratings_dataset_shuffled.skip(trainset_size)\n",
266 | "\n",
267 | "print(\n",
268 | " \"ratings_trainset size: %d\" % ratings_trainset.__len__()\n",
269 | ")\n",
270 | "print(\n",
271 | " \"ratings_testset size: %d\" % ratings_testset.__len__()\n",
272 | ")"
273 | ],
274 | "execution_count": 3,
275 | "outputs": [
276 | {
277 | "output_type": "stream",
278 | "name": "stdout",
279 | "text": [
280 | "ratings_dataset size: 100836\n",
281 | " movie_genres movie_id movie_title \\\n",
282 | "0 [7, 8, 13, 15] b'4874' b'K-PAX (2001)' \n",
283 | "1 [7, 18] b'527' b\"Schindler's List (1993)\" \n",
284 | "2 [5, 9] b'7943' b'Killers, The (1946)' \n",
285 | "3 [10, 13, 16] b'1644' b'I Know What You Did Last Summer (1997)' \n",
286 | "4 [1, 2, 3, 4, 12, 14] b'8360' b'Shrek 2 (2004)' \n",
287 | "\n",
288 | " timestamp user_id user_rating \n",
289 | "0 1446749868 b'105' 5.0 \n",
290 | "1 1305696664 b'17' 4.5 \n",
291 | "2 1166068511 b'309' 4.0 \n",
292 | "3 1518640852 b'111' 0.5 \n",
293 | "4 1127221149 b'182' 3.0 \n",
294 | " movie_id movie_title timestamp user_id \\\n",
295 | "0 b'4874' b'K-PAX (2001)' 1446749868 b'105' \n",
296 | "1 b'527' b\"Schindler's List (1993)\" 1305696664 b'17' \n",
297 | "2 b'7943' b'Killers, The (1946)' 1166068511 b'309' \n",
298 | "3 b'1644' b'I Know What You Did Last Summer (1997)' 1518640852 b'111' \n",
299 | "4 b'8360' b'Shrek 2 (2004)' 1127221149 b'182' \n",
300 | "\n",
301 | " user_rating \n",
302 | "0 5.0 \n",
303 | "1 4.5 \n",
304 | "2 4.0 \n",
305 | "3 0.5 \n",
306 | "4 3.0 \n",
307 | "ratings_trainset size: 80668\n",
308 | "ratings_testset size: 20168\n"
309 | ]
310 | }
311 | ]
312 | },
313 | {
314 | "cell_type": "markdown",
315 | "metadata": {
316 | "id": "2JOGVWBEWiZZ"
317 | },
318 | "source": [
319 | "To make a custom `tf.data.Dataset` object including your own dataset visit [this link](https://www.tensorflow.org/datasets/add_dataset).\n",
320 | "\n",
321 | "For more information about `tf.data.Dataset` visit [this link](https://www.tensorflow.org/api_docs/python/tf/data/Dataset)."
322 | ]
323 | },
324 | {
325 | "cell_type": "markdown",
326 | "metadata": {
327 | "id": "GnlXCP02UV8K"
328 | },
329 | "source": [
330 | "## Preprocess raw features and make embeddings with Keras preprocessing layers\n",
331 | "\n",
332 | "Raw features are usually not be immediately usable in a machine learning model and should be preprocessed in the first place.\n",
333 | "- **Numerical features** (ratings, prices, timestamps, etc) can be far away in terms of scale and need to be `normalized` so that their values lie in a small interval around 0.\n",
334 | "- **Categorical features** (ids, usernames/emails, titles, etc) are usually string features and have to be translated into `embedding vectors` (numerical feature representations) that are adjusted during training the model.\n",
335 | "- **Text features** (descriptions, comments, etc) need to be at first, `tokenized` (split into smaller parts such as individual words known as word pieces) and then translated into embeddings.\n",
336 | "\n",
337 | "[Keras preprocessing layers](https://keras.io/guides/preprocessing_layers/) let us build `end-to-end` portable models that accept raw features (raw images or raw structured data) as input; models that handle feature normalization or feature value indexing on their own."
338 | ]
339 | },
340 | {
341 | "cell_type": "markdown",
342 | "metadata": {
343 | "id": "EHvSx5Vq1dFO"
344 | },
345 | "source": [
346 | "Let's first have a look at what features we can use from the MovieLens dataset:\n"
347 | ]
348 | },
349 | {
350 | "cell_type": "code",
351 | "metadata": {
352 | "id": "iIiKVYmASodk",
353 | "colab": {
354 | "base_uri": "https://localhost:8080/"
355 | },
356 | "outputId": "d5b6473a-b387-4f91-e649-b139f41a62f4"
357 | },
358 | "source": [
359 | "from pprint import pprint\n",
360 | "\n",
361 | "for rating in ratings_trainset.take(1).as_numpy_iterator():\n",
362 | " pprint(rating)"
363 | ],
364 | "execution_count": 4,
365 | "outputs": [
366 | {
367 | "output_type": "stream",
368 | "name": "stdout",
369 | "text": [
370 | "{'movie_id': b'3421',\n",
371 | " 'movie_title': b'Animal House (1978)',\n",
372 | " 'timestamp': 975212199,\n",
373 | " 'user_id': b'216',\n",
374 | " 'user_rating': 5.0}\n"
375 | ]
376 | }
377 | ]
378 | },
379 | {
380 | "cell_type": "markdown",
381 | "metadata": {
382 | "id": "vAIqWlEjsvVO"
383 | },
384 | "source": [
385 | "For more information about each field visit [this link](https://www.tensorflow.org/datasets/catalog/movielens)."
386 | ]
387 | },
388 | {
389 | "cell_type": "markdown",
390 | "metadata": {
391 | "id": "82VLAC1qJHMD"
392 | },
393 | "source": [
394 | "### Normalize numerical features\n",
395 | "`timestamp` values are far too large to be used directly in a machine learning model. However, it can be normalized in a small interval around 0. Standardization (`Z-score Normalization`) is a common preprocessing transformation that rescales features to normalize their range by subtracting the feature's `mean` and dividing by its `standard deviation`.\n",
396 | "\n"
397 | ]
398 | },
399 | {
400 | "cell_type": "code",
401 | "metadata": {
402 | "id": "Ga9_p0_1GpUC",
403 | "colab": {
404 | "base_uri": "https://localhost:8080/"
405 | },
406 | "outputId": "fdea8df1-9f6a-47c7-a1ae-ed70342b6201"
407 | },
408 | "source": [
409 | "# Make a Keras Normalization layer to standardize a numerical feature.\n",
410 | "timestamp_normalization_layer = \\\n",
411 | " tf.keras.layers.Normalization(axis=None)\n",
412 | "\n",
413 | "# Normalization layer is a non-trainable layer and its state (mean and std of\n",
414 | "# feature set) must be set before training in a step called \"adaptation\".\n",
415 | "timestamp_normalization_layer.adapt(\n",
416 | " ratings_trainset.map(\n",
417 | " lambda x: x['timestamp']\n",
418 | " )\n",
419 | ")\n",
420 | "\n",
421 | "for rating in ratings_trainset.take(3).as_numpy_iterator():\n",
422 | " print(\n",
423 | " f\"Raw timestamp: {rating['timestamp']} ->\",\n",
424 | " f\"Normalized timestamp: {timestamp_normalization_layer(rating['timestamp'])}\"\n",
425 | " )"
426 | ],
427 | "execution_count": 5,
428 | "outputs": [
429 | {
430 | "output_type": "stream",
431 | "name": "stdout",
432 | "text": [
433 | "Raw timestamp: 975212199 -> Normalized timestamp: -1.0651628971099854\n",
434 | "Raw timestamp: 1167543227 -> Normalized timestamp: -0.1764659434556961\n",
435 | "Raw timestamp: 958881789 -> Normalized timestamp: -1.1406203508377075\n"
436 | ]
437 | }
438 | ]
439 | },
440 | {
441 | "cell_type": "markdown",
442 | "metadata": {
443 | "id": "Q_2VmQ4KpOpo"
444 | },
445 | "source": [
446 | "### Turning categorical features into embeddings\n",
447 | "\n",
448 | "A categorical feature is a feature that does not express a continuous quantity, but rather takes on one of a set of fixed values. Most deep learning models express these feature by turning them into high-dimensional embedding vectors which will be adjusted during model training.\n",
449 | "\n",
450 | "Here we represent each user and each movie by an embedding vector. Initially, these embeddings will take on random values, but during training, we will adjust them so that embeddings of users and the movies they watch end up closer together.\n",
451 | "\n",
452 | "Taking raw categorical features and turning them into embeddings is normally a two-step process:\n",
453 | "\n",
454 | "\n",
455 | "1. Build a mapping (called a `\"vocabulary\"`) that maps each raw values for example \"Postman, The (1997)\" to unique integers (say, 15).\n",
456 | "2. Turn these integers into embedding vectors.\n",
457 | "\n"
458 | ]
459 | },
460 | {
461 | "cell_type": "code",
462 | "metadata": {
463 | "id": "2yfWHKFfpPSx",
464 | "colab": {
465 | "base_uri": "https://localhost:8080/"
466 | },
467 | "outputId": "abb36d70-c592-48bf-a40a-0bcef167b7c3"
468 | },
469 | "source": [
470 | "# Make a Keras StringLookup layer as the mapping (lookup)\n",
471 | "user_id_lookup_layer = \\\n",
472 | " tf.keras.layers.StringLookup(mask_token=None)\n",
473 | "\n",
474 | "# StringLookup layer is a non-trainable layer and its state (the vocabulary)\n",
475 | "# must be constructed and set before training in a step called \"adaptation\".\n",
476 | "user_id_lookup_layer.adapt(\n",
477 | " ratings_trainset.map(\n",
478 | " lambda x: x['user_id']\n",
479 | " )\n",
480 | ")\n",
481 | "\n",
482 | "print(\n",
483 | " f\"Vocabulary[:10] -> {user_id_lookup_layer.get_vocabulary()[:10]}\"\n",
484 | " # Vocabulary: ['[UNK]', '405', '655', '13', ...]\n",
485 | " # The vocabulary includes one (or more!) unknown (or \"out of vocabulary\", OOV)\n",
486 | " # tokens. So the layer can handle categorical values that are not in the\n",
487 | " # vocabulary and the model can continue to learn about and make\n",
488 | " # recommendations even using features that have not been seen during\n",
489 | " # vocabulary construction.\n",
490 | ")\n",
491 | "\n",
492 | "print(\n",
493 | " \"Mapped integer for user ids: ['-2', '13', '655', 'xxx']\\n\",\n",
494 | " user_id_lookup_layer(\n",
495 | " ['-2', '13', '655', 'xxx']\n",
496 | " )\n",
497 | ")\n",
498 | "\n",
499 | "user_id_embedding_dim = 32\n",
500 | "# The larger it is, the higher the capacity of the model, but the slower it is\n",
501 | "# to fit and serve and more prone to overfitting.\n",
502 | "\n",
503 | "user_id_embedding_layer = tf.keras.layers.Embedding(\n",
504 | " # Size of the vocabulary\n",
505 | " input_dim=user_id_lookup_layer.vocabulary_size(),\n",
506 | " # Dimension of the dense embedding\n",
507 | " output_dim=user_id_embedding_dim\n",
508 | ")\n",
509 | "\n",
510 | "# A model that takes raw string feature values (user_id) in and yields embeddings\n",
511 | "user_id_model = tf.keras.Sequential(\n",
512 | " [\n",
513 | " user_id_lookup_layer,\n",
514 | " user_id_embedding_layer\n",
515 | " ]\n",
516 | ")\n",
517 | "\n",
518 | "print(\n",
519 | " \"Embeddings for user ids: ['-2', '13', '655', 'xxx']\\n\",\n",
520 | " user_id_model(\n",
521 | " tf.convert_to_tensor(['-2', '13', '655', 'xxx'])\n",
522 | " )\n",
523 | ")"
524 | ],
525 | "execution_count": 6,
526 | "outputs": [
527 | {
528 | "output_type": "stream",
529 | "name": "stdout",
530 | "text": [
531 | "Vocabulary[:10] -> ['[UNK]', '414', '599', '474', '448', '274', '610', '68', '380', '606']\n",
532 | "Mapped integer for user ids: ['-2', '13', '655', 'xxx']\n",
533 | " tf.Tensor([ 0 493 0 0], shape=(4,), dtype=int64)\n",
534 | "Embeddings for user ids: ['-2', '13', '655', 'xxx']\n",
535 | " tf.Tensor(\n",
536 | "[[ 0.01478275 -0.03399806 -0.04737892 -0.01455851 0.02260116 0.03772945\n",
537 | " -0.04947149 -0.03576241 0.01529941 -0.01436429 0.03431726 0.00177632\n",
538 | " 0.00778866 0.00945215 -0.02211686 0.04326234 0.00865857 0.03611586\n",
539 | " -0.00819879 -0.01737251 -0.03739591 0.04867731 -0.00277033 0.02783228\n",
540 | " -0.03756921 0.04588716 -0.00360299 -0.00270822 -0.01879736 0.00096357\n",
541 | " 0.04194726 -0.03343551]\n",
542 | " [-0.00042721 0.01578123 0.03790379 0.03472162 -0.04650879 0.02481607\n",
543 | " -0.0494769 0.01245934 0.04805774 0.0247086 0.03865222 0.00027167\n",
544 | " 0.00802004 0.00562286 0.02058078 -0.03171451 0.0408157 0.00758809\n",
545 | " 0.04206653 0.04217685 0.01028284 0.02889507 0.03656546 0.00334147\n",
546 | " -0.00780983 -0.02825767 0.00996434 0.02749718 -0.03737084 -0.0091005\n",
547 | " 0.03157964 0.01717177]\n",
548 | " [ 0.01478275 -0.03399806 -0.04737892 -0.01455851 0.02260116 0.03772945\n",
549 | " -0.04947149 -0.03576241 0.01529941 -0.01436429 0.03431726 0.00177632\n",
550 | " 0.00778866 0.00945215 -0.02211686 0.04326234 0.00865857 0.03611586\n",
551 | " -0.00819879 -0.01737251 -0.03739591 0.04867731 -0.00277033 0.02783228\n",
552 | " -0.03756921 0.04588716 -0.00360299 -0.00270822 -0.01879736 0.00096357\n",
553 | " 0.04194726 -0.03343551]\n",
554 | " [ 0.01478275 -0.03399806 -0.04737892 -0.01455851 0.02260116 0.03772945\n",
555 | " -0.04947149 -0.03576241 0.01529941 -0.01436429 0.03431726 0.00177632\n",
556 | " 0.00778866 0.00945215 -0.02211686 0.04326234 0.00865857 0.03611586\n",
557 | " -0.00819879 -0.01737251 -0.03739591 0.04867731 -0.00277033 0.02783228\n",
558 | " -0.03756921 0.04588716 -0.00360299 -0.00270822 -0.01879736 0.00096357\n",
559 | " 0.04194726 -0.03343551]], shape=(4, 32), dtype=float32)\n"
560 | ]
561 | }
562 | ]
563 | },
564 | {
565 | "cell_type": "code",
566 | "source": [
567 | "movie_id_lookup_layer = \\\n",
568 | " tf.keras.layers.StringLookup(mask_token=None)\n",
569 | "\n",
570 | "movie_id_lookup_layer.adapt(\n",
571 | " ratings_trainset.map(\n",
572 | " lambda x: x['movie_id']\n",
573 | " )\n",
574 | ")\n",
575 | "\n",
576 | "# Same as user_id_embedding_dim to be able to measure the similarity\n",
577 | "movie_id_embedding_dim = 32\n",
578 | "\n",
579 | "movie_id_embedding_layer = tf.keras.layers.Embedding(\n",
580 | " input_dim=movie_id_lookup_layer.vocabulary_size(),\n",
581 | " output_dim=movie_id_embedding_dim\n",
582 | ")\n",
583 | "\n",
584 | "movie_id_model = tf.keras.Sequential(\n",
585 | " [\n",
586 | " movie_id_lookup_layer,\n",
587 | " movie_id_embedding_layer\n",
588 | " ]\n",
589 | ")\n",
590 | "\n",
591 | "print(\n",
592 | " f\"Embedding for the movie 898:\\n {movie_id_model(tf.convert_to_tensor(['898']))}\"\n",
593 | ")"
594 | ],
595 | "metadata": {
596 | "id": "O-Fw4HAuO1kh",
597 | "outputId": "be83a6c3-31f7-4cd7-fc1e-83f94074d213",
598 | "colab": {
599 | "base_uri": "https://localhost:8080/"
600 | }
601 | },
602 | "execution_count": 7,
603 | "outputs": [
604 | {
605 | "output_type": "stream",
606 | "name": "stdout",
607 | "text": [
608 | "Embedding for the movie 898:\n",
609 | " [[ 0.02420554 -0.02993221 -0.03578611 -0.01940017 0.02106949 -0.01936241\n",
610 | " -0.00158745 0.00174867 0.02626561 -0.0172717 0.02571883 -0.03316146\n",
611 | " 0.03160522 0.04094924 -0.01908119 0.01257378 -0.00210525 -0.013066\n",
612 | " -0.03793862 0.01088411 0.02089325 -0.04121248 0.03619212 0.01857083\n",
613 | " 0.02743815 0.03726472 -0.03322572 0.03820366 0.01855929 0.03005471\n",
614 | " -0.01181834 -0.00866457]]\n"
615 | ]
616 | }
617 | ]
618 | },
619 | {
620 | "cell_type": "markdown",
621 | "metadata": {
622 | "id": "U3hdi7pZS2oC"
623 | },
624 | "source": [
625 | "### Tokenize textual features and translate them into embeddings\n",
626 | "Candidates textual description and users' reviews can be useful especially in a `cold-start` or `long-tail` scenario.\n",
627 | "\n",
628 | "While the MovieLens dataset does not give us rich textual features, we can still use movie titles. This may help us capture the fact that movies with very similar titles are likely to belong to the same series (for example \"Harry Potter and the Philosopher's Stone\" and \"Harry Potter and the Chamber of Secrets\")."
629 | ]
630 | },
631 | {
632 | "cell_type": "code",
633 | "metadata": {
634 | "id": "mGF8c2ZKOtuG",
635 | "colab": {
636 | "base_uri": "https://localhost:8080/"
637 | },
638 | "outputId": "a4d65258-0200-4986-8653-fdf07ba98eb7"
639 | },
640 | "source": [
641 | "# Keras TextVectorization layer transforms the raw texts into `word pieces` and\n",
642 | "# map these pieces into tokens.\n",
643 | "movie_title_vectorization_layer = \\\n",
644 | " tf.keras.layers.TextVectorization()\n",
645 | "movie_title_vectorization_layer.adapt(\n",
646 | " ratings_trainset.map(\n",
647 | " lambda rating: rating['movie_title']\n",
648 | " )\n",
649 | ")\n",
650 | "\n",
651 | "# Verify that the tokenization is done correctly\n",
652 | "print(\n",
653 | " \"Vocabulary[40:50] -> \",\n",
654 | " movie_title_vectorization_layer.get_vocabulary()[40:50]\n",
655 | ")\n",
656 | "\n",
657 | "print(\n",
658 | " \"Vectorized title for 'Postman, The (1997)'\\n\",\n",
659 | " movie_title_vectorization_layer('Postman, The (1997)')\n",
660 | ")\n",
661 | "\n",
662 | "movie_title_embedding_dim = 32\n",
663 | "movie_title_embedding_layer = tf.keras.layers.Embedding(\n",
664 | " input_dim=len(movie_title_vectorization_layer.get_vocabulary()),\n",
665 | " output_dim=movie_title_embedding_dim,\n",
666 | " # Whether or not the input value 0 is a MASK token.\n",
667 | " # Keras TextVectorization layer builds the vocabulary with MASK token.\n",
668 | " mask_zero=True\n",
669 | ")\n",
670 | "\n",
671 | "movie_title_model = tf.keras.Sequential(\n",
672 | " [\n",
673 | " movie_title_vectorization_layer,\n",
674 | " movie_title_embedding_layer,\n",
675 | " # each title contains multiple words, so we will get multiple embeddings\n",
676 | " # for each title that should be compressed into a single embedding for\n",
677 | " # the text. Models like RNNs, Transformers or Attentions are useful here.\n",
678 | " # However, averaging all the words' embeddings together is also a good\n",
679 | " # starting point.\n",
680 | " tf.keras.layers.GlobalAveragePooling1D()\n",
681 | " ]\n",
682 | ")"
683 | ],
684 | "execution_count": 8,
685 | "outputs": [
686 | {
687 | "output_type": "stream",
688 | "name": "stdout",
689 | "text": [
690 | "Vocabulary[40:50] -> ['2014', 'ii', '1985', '2013', 'day', 'wars', '2015', '1982', 'for', 'episode']\n",
691 | "Vectorized title for 'Postman, The (1997)'\n",
692 | " tf.Tensor([1119 2 12], shape=(3,), dtype=int64)\n"
693 | ]
694 | }
695 | ]
696 | },
697 | {
698 | "cell_type": "markdown",
699 | "metadata": {
700 | "id": "PEmOyPxuOcHx"
701 | },
702 | "source": [
703 | "For more information about feature preprocessing visit [this link](https://www.tensorflow.org/recommenders/examples/featurization?hl=lt)."
704 | ]
705 | },
706 | {
707 | "cell_type": "markdown",
708 | "metadata": {
709 | "id": "UChreol5-Aj2"
710 | },
711 | "source": [
712 | "## Query and Candidate representation\n",
713 | "We are building a [two-tower retrieval model](https://research.google/pubs/pub48840/), a model including two seperate models (towers) one for transforming query raw features to query representation (query tower) and one another for transforming candidate raw features to the same dimensionality candidate representation.\n",
714 | "\n",
715 | "The output tensors of the two models will multiply together (inner product) to give a query-candidate `affinity score` (similarity measure). Higher scores express a better match between the candidate and the query."
716 | ]
717 | },
718 | {
719 | "cell_type": "code",
720 | "metadata": {
721 | "id": "H1jx9RDfoJIq"
722 | },
723 | "source": [
724 | "# Query tower\n",
725 | "query_model = user_id_model\n",
726 | "\n",
727 | "# Candidate tower\n",
728 | "candidate_model = movie_id_model\n",
729 | "\n",
730 | "\n",
731 | "# Here we only used query and candidate identifiers to buid the towers. This\n",
732 | "# corresponds exactly to a classic matrix factorization approach.\n",
733 | "# https://ieeexplore.ieee.org/abstract/document/4781121\n",
734 | "# However, we can extend `tf.keras.Model` class to an arbitrarily complex model\n",
735 | "# including other features and return the final embedding vector at the end.\n",
736 | "# For example, by using movie metadata in the candidate tower, we can alleviate\n",
737 | "# cold-start problem.\n",
738 | "# return tf.concat([\n",
739 | "# self.user_embedding(inputs[\"user_id\"]),\n",
740 | "# self.timestamp_embedding(inputs[\"timestamp\"]),\n",
741 | "# self.normalized_timestamp(inputs[\"timestamp\"])\n",
742 | "# ], axis=1)"
743 | ],
744 | "execution_count": 9,
745 | "outputs": []
746 | },
747 | {
748 | "cell_type": "markdown",
749 | "metadata": {
750 | "id": "-X0nS9g18q4v"
751 | },
752 | "source": [
753 | "## Build the Retrieval (Candidate Generation) task\n",
754 | "\n",
755 | "It is about selecting an initial set of hundreds of candidates from all possible candidates. The main objective of this model is to efficiently weed out all candidates that the user is not interested in. Because the retrieval model may be dealing with millions of candidates, **it has to be computationally efficient**.\n",
756 | "\n",
757 | "A retrieval system is a model that predicts a set of movies from the catalogue that the user is likely to watch. So the train set should be expressesing which movies the users watched, and which they did not. for example:\n",
758 | "```\n",
759 | "[\n",
760 | " (('user1', 'Star Wars'), POSITIVE),\n",
761 | " (('user1', 'Harry Potter'), NEGATIVE),\n",
762 | " ...\n",
763 | "]\n",
764 | "```\n",
765 | "So we treat Movielens as an `implicit feedback dataset`, where users' watches tell us which things they prefer to see and which they'd rather not see. This means that every movie a user rated (so watched!), no matter the given rating, is an **implicit positive example**, and every movie they have not rated (not seen!) is an **implicit negative example**."
766 | ]
767 | },
768 | {
769 | "cell_type": "code",
770 | "metadata": {
771 | "id": "QP5VaGW69c2B"
772 | },
773 | "source": [
774 | "# We don't need rating field for the retrieval task\n",
775 | "retrieval_ratings_trainset = ratings_trainset.map(\n",
776 | " lambda rating: {\n",
777 | " 'user_id': rating['user_id'],\n",
778 | " 'movie_id': rating['movie_id'],\n",
779 | " }\n",
780 | ")\n",
781 | "\n",
782 | "retrieval_ratings_testset = ratings_testset.map(\n",
783 | " lambda rating: {\n",
784 | " 'user_id': rating['user_id'],\n",
785 | " 'movie_id': rating['movie_id'],\n",
786 | " }\n",
787 | ")"
788 | ],
789 | "execution_count": 10,
790 | "outputs": []
791 | },
792 | {
793 | "cell_type": "markdown",
794 | "metadata": {
795 | "id": "SqeFh9s99bLw"
796 | },
797 | "source": [
798 | "The similarity between the query representation (query embedding vector) and the candiate representation (candidate embedding vector) a.k.a. `affinity score` can be calcualted by dot-product (or other similarity measures). The K-nearest candidates (candidates with higher affinity scores) will be chosen for the final list.\n",
799 | "\n",
800 | "In our training data we only have positive (user, movie) pairs. To figure out how good our model is, we need to compare the affinity score that the model calculates for this positive pair to the scores of all the other possible candidates: if the score for the positive pair is higher than for all other possible candidates, our model is highly accurate.\n",
801 | "\n",
802 | "To measure the performance of a retrieval task, `factorized top-K categorical accuracy` metrics over a corpus of candidates can be used. These metrics measure how good the model is at picking the true candidate out of **all possible candidates** in the system.\n",
803 | "\n",
804 | "For example, a `top-5 categorical accuracy` metric of `0.2` would tell us that, on average, the true positive is in the top 5 retrieved items 20% of the time."
805 | ]
806 | },
807 | {
808 | "cell_type": "code",
809 | "metadata": {
810 | "id": "DgwkIUn71_0y",
811 | "colab": {
812 | "base_uri": "https://localhost:8080/"
813 | },
814 | "outputId": "f868ba6f-c3ae-4536-a977-1557ef45f115"
815 | },
816 | "source": [
817 | "# To calculate the factorized top-k categorical accuracy we need the dataset of\n",
818 | "# all possible candidates that are used as implicit negatives for evaluation.\n",
819 | "movies_dataset, movies_dataset_info = tfds.load(\n",
820 | " name='movielens/latest-small-movies',\n",
821 | " split='train',\n",
822 | " with_info=True\n",
823 | ")\n",
824 | "\n",
825 | "print(\n",
826 | " tfds.as_dataframe(movies_dataset.take(5), movies_dataset_info)\n",
827 | ")\n",
828 | "\n",
829 | "# We are using just `movie_id` feature for making the candidates representation\n",
830 | "candidates_corpus_dataset = movies_dataset.map(\n",
831 | " lambda movie: movie['movie_id']\n",
832 | ")"
833 | ],
834 | "execution_count": 11,
835 | "outputs": [
836 | {
837 | "output_type": "stream",
838 | "name": "stdout",
839 | "text": [
840 | " movie_genres movie_id movie_title\n",
841 | "0 [4] b'2261' b'One Crazy Summer (1986)'\n",
842 | "1 [10] b'1979' b'Friday the 13th Part VI: Jason Lives (1986)'\n",
843 | "2 [4, 5] b'6143' b'Trail of the Pink Panther (1982)'\n",
844 | "3 [4, 7, 14] b'5856' b'Do You Remember Dolly Bell? (Sjecas li se, D...\n",
845 | "4 [0, 4, 7, 16] b'70728' b'Bronson (2009)'\n"
846 | ]
847 | }
848 | ]
849 | },
850 | {
851 | "cell_type": "markdown",
852 | "metadata": {
853 | "id": "P0aPHewDuQZf"
854 | },
855 | "source": [
856 | "[TensorFlow Recommenders](https://www.tensorflow.org/recommenders) (TFRS) is a library to facilitate building and evaluating flexible recommendation models.\n",
857 | "\n",
858 | "It can calculate the `factorized top-k categorical accuracy` through `FactorizedTopK` metrics using dataset of all possible candidate embeddings."
859 | ]
860 | },
861 | {
862 | "cell_type": "code",
863 | "metadata": {
864 | "id": "YPO73Ihx_xQS"
865 | },
866 | "source": [
867 | "!pip install -q scann tensorflow-recommenders\n",
868 | "# We also installed `ScaNN` package as a dependency for TFRS library\n",
869 | "# We will describe ScaNN in future but it has to be installed before importing\n",
870 | "# the TFRS\n",
871 | "import tensorflow_recommenders as tfrs\n",
872 | "\n",
873 | "factorized_top_k_metrics = tfrs.metrics.FactorizedTopK(\n",
874 | " # dataset of candidate embeddings from which candidates should be retrieved\n",
875 | " candidates=candidates_corpus_dataset.batch(128).map(\n",
876 | " candidate_model\n",
877 | " )\n",
878 | ")"
879 | ],
880 | "execution_count": 12,
881 | "outputs": []
882 | },
883 | {
884 | "cell_type": "markdown",
885 | "metadata": {
886 | "id": "Ie_v22Od-zY8"
887 | },
888 | "source": [
889 | "`in-batch softmax loss` can be used as a `loss function` in order to train the system.\n",
890 | "\n",
891 | "TFRS proposes a Keras layer named `tfrs.tasks.Retrieval` that takes the query and candidate embeddings as arguments, and returns the computed loss."
892 | ]
893 | },
894 | {
895 | "cell_type": "code",
896 | "metadata": {
897 | "id": "4E8eOyErNqHY"
898 | },
899 | "source": [
900 | "retrieval_task_layer = tfrs.tasks.Retrieval(\n",
901 | " metrics=factorized_top_k_metrics\n",
902 | ")\n",
903 | "\n",
904 | "# The task computes the metrics and return the in-batch softmax loss.\n",
905 | "# Because the metrics range over the entire candidate set, they are usually much\n",
906 | "# slower to compute. Consider setting `compute_metrics=False` in Retrieval\n",
907 | "# costructor during training to save the time in computing the metrics."
908 | ],
909 | "execution_count": 13,
910 | "outputs": []
911 | },
912 | {
913 | "cell_type": "markdown",
914 | "metadata": {
915 | "id": "lVFXh69bPYL9"
916 | },
917 | "source": [
918 | "### Create the training loop\n",
919 | "To create an appropriate training loop and train the models we can extend the class `tf.keras.Model` and override the `train_step` and `test_step` functions. [See how](https://keras.io/guides/customizing_what_happens_in_fit/).\n",
920 | "\n",
921 | "However, to keep the focus on modelling and abstract away some of the boilerplate, TFRS exposes `tfrs.models.Model` base class which allows us to compute both training and test losses using the same method. All we need to do is to set up the components in the `__init__` method, and implement the `compute_loss` method, taking in the raw features and returning a loss value. The base model will then take care of creating the appropriate training loop to fit the model."
922 | ]
923 | },
924 | {
925 | "cell_type": "code",
926 | "metadata": {
927 | "id": "EGN_rnLNN07m"
928 | },
929 | "source": [
930 | "class RetrievalModel(tfrs.models.Model):\n",
931 | " \"\"\"MovieLens candidate generation model\"\"\"\n",
932 | "\n",
933 | " def __init__(self, query_model, candidate_model, retrieval_task_layer):\n",
934 | " super().__init__()\n",
935 | " self.query_model: tf.keras.Model = query_model\n",
936 | " self.candidate_model: tf.keras.Model = candidate_model\n",
937 | " self.retrieval_task_layer: tf.keras.layers.Layer = retrieval_task_layer\n",
938 | "\n",
939 | " #def compute_loss(self, features: Dict[Text, tf.Tensor], training=False):\n",
940 | " def compute_loss(self, features, training=False) -> tf.Tensor:\n",
941 | " query_embeddings = self.query_model(features['user_id'])\n",
942 | " positive_candidate_embeddings = self.candidate_model(features[\"movie_id\"])\n",
943 | "\n",
944 | " loss = self.retrieval_task_layer(\n",
945 | " query_embeddings,\n",
946 | " positive_candidate_embeddings\n",
947 | " # ,compute_metrics=not training # To speed up training\n",
948 | " )\n",
949 | " return loss"
950 | ],
951 | "execution_count": 14,
952 | "outputs": []
953 | },
954 | {
955 | "cell_type": "markdown",
956 | "metadata": {
957 | "id": "ylJqYSN8GH68"
958 | },
959 | "source": [
960 | "### Fit the model using standard Keras routine"
961 | ]
962 | },
963 | {
964 | "cell_type": "code",
965 | "metadata": {
966 | "id": "HoyYTDhB9CAP"
967 | },
968 | "source": [
969 | "movielens_retrieval_model = RetrievalModel(\n",
970 | " query_model,\n",
971 | " candidate_model,\n",
972 | " retrieval_task_layer\n",
973 | ")\n",
974 | "\n",
975 | "optimizer_step_size = 0.1\n",
976 | "movielens_retrieval_model.compile(\n",
977 | " optimizer=tf.keras.optimizers.Adagrad(\n",
978 | " learning_rate=optimizer_step_size\n",
979 | " )\n",
980 | ")"
981 | ],
982 | "execution_count": 15,
983 | "outputs": []
984 | },
985 | {
986 | "cell_type": "code",
987 | "metadata": {
988 | "id": "ZNw02KWxE3bI",
989 | "colab": {
990 | "base_uri": "https://localhost:8080/"
991 | },
992 | "outputId": "f17923c4-38d2-4cda-dff0-1242bfff9eae"
993 | },
994 | "source": [
995 | "# Shuffle the training data for each epoch.\n",
996 | "# Batch and cache both the training and evaluation data.\n",
997 | "# `cache()` method caches the elements in the dataset in memory. To caches data\n",
998 | "# in a file pass the `filename` argument to the method: cache(filename='')\n",
999 | "# The first time the dataset is iterated over, its elements will be cached\n",
1000 | "# either in the specified file or in memory. Subsequent iterations will use the\n",
1001 | "# cached data.\n",
1002 | "retrieval_cached_ratings_trainset = \\\n",
1003 | " retrieval_ratings_trainset.shuffle(100_000).batch(8192).cache()\n",
1004 | "retrieval_cached_ratings_testset = \\\n",
1005 | " retrieval_ratings_testset.batch(4096).cache()\n",
1006 | "\n",
1007 | "num_epochs = 5\n",
1008 | "history = movielens_retrieval_model.fit(\n",
1009 | " retrieval_cached_ratings_trainset,\n",
1010 | " validation_data=retrieval_cached_ratings_testset,\n",
1011 | " validation_freq=1,\n",
1012 | " epochs=num_epochs\n",
1013 | ")"
1014 | ],
1015 | "execution_count": 16,
1016 | "outputs": [
1017 | {
1018 | "output_type": "stream",
1019 | "name": "stdout",
1020 | "text": [
1021 | "Epoch 1/5\n",
1022 | "10/10 [==============================] - 169s 15s/step - factorized_top_k/top_1_categorical_accuracy: 6.1982e-05 - factorized_top_k/top_5_categorical_accuracy: 0.0013 - factorized_top_k/top_10_categorical_accuracy: 0.0031 - factorized_top_k/top_50_categorical_accuracy: 0.0262 - factorized_top_k/top_100_categorical_accuracy: 0.0574 - loss: 71061.1591 - regularization_loss: 0.0000e+00 - total_loss: 71061.1591 - val_factorized_top_k/top_1_categorical_accuracy: 0.0017 - val_factorized_top_k/top_5_categorical_accuracy: 0.0090 - val_factorized_top_k/top_10_categorical_accuracy: 0.0175 - val_factorized_top_k/top_50_categorical_accuracy: 0.0799 - val_factorized_top_k/top_100_categorical_accuracy: 0.1389 - val_loss: 30403.5078 - val_regularization_loss: 0.0000e+00 - val_total_loss: 30403.5078\n",
1023 | "Epoch 2/5\n",
1024 | "10/10 [==============================] - 128s 13s/step - factorized_top_k/top_1_categorical_accuracy: 8.4296e-04 - factorized_top_k/top_5_categorical_accuracy: 0.0079 - factorized_top_k/top_10_categorical_accuracy: 0.0174 - factorized_top_k/top_50_categorical_accuracy: 0.0816 - factorized_top_k/top_100_categorical_accuracy: 0.1432 - loss: 68374.8374 - regularization_loss: 0.0000e+00 - total_loss: 68374.8374 - val_factorized_top_k/top_1_categorical_accuracy: 7.9334e-04 - val_factorized_top_k/top_5_categorical_accuracy: 0.0065 - val_factorized_top_k/top_10_categorical_accuracy: 0.0140 - val_factorized_top_k/top_50_categorical_accuracy: 0.0683 - val_factorized_top_k/top_100_categorical_accuracy: 0.1245 - val_loss: 29863.0508 - val_regularization_loss: 0.0000e+00 - val_total_loss: 29863.0508\n",
1025 | "Epoch 3/5\n",
1026 | "10/10 [==============================] - 128s 13s/step - factorized_top_k/top_1_categorical_accuracy: 0.0012 - factorized_top_k/top_5_categorical_accuracy: 0.0118 - factorized_top_k/top_10_categorical_accuracy: 0.0232 - factorized_top_k/top_50_categorical_accuracy: 0.1039 - factorized_top_k/top_100_categorical_accuracy: 0.1754 - loss: 66258.5078 - regularization_loss: 0.0000e+00 - total_loss: 66258.5078 - val_factorized_top_k/top_1_categorical_accuracy: 4.9583e-04 - val_factorized_top_k/top_5_categorical_accuracy: 0.0050 - val_factorized_top_k/top_10_categorical_accuracy: 0.0111 - val_factorized_top_k/top_50_categorical_accuracy: 0.0597 - val_factorized_top_k/top_100_categorical_accuracy: 0.1075 - val_loss: 29832.7773 - val_regularization_loss: 0.0000e+00 - val_total_loss: 29832.7773\n",
1027 | "Epoch 4/5\n",
1028 | "10/10 [==============================] - 128s 13s/step - factorized_top_k/top_1_categorical_accuracy: 0.0012 - factorized_top_k/top_5_categorical_accuracy: 0.0129 - factorized_top_k/top_10_categorical_accuracy: 0.0270 - factorized_top_k/top_50_categorical_accuracy: 0.1152 - factorized_top_k/top_100_categorical_accuracy: 0.1959 - loss: 64616.1264 - regularization_loss: 0.0000e+00 - total_loss: 64616.1264 - val_factorized_top_k/top_1_categorical_accuracy: 4.4625e-04 - val_factorized_top_k/top_5_categorical_accuracy: 0.0035 - val_factorized_top_k/top_10_categorical_accuracy: 0.0085 - val_factorized_top_k/top_50_categorical_accuracy: 0.0501 - val_factorized_top_k/top_100_categorical_accuracy: 0.0933 - val_loss: 30017.0508 - val_regularization_loss: 0.0000e+00 - val_total_loss: 30017.0508\n",
1029 | "Epoch 5/5\n",
1030 | "10/10 [==============================] - 128s 13s/step - factorized_top_k/top_1_categorical_accuracy: 0.0010 - factorized_top_k/top_5_categorical_accuracy: 0.0135 - factorized_top_k/top_10_categorical_accuracy: 0.0288 - factorized_top_k/top_50_categorical_accuracy: 0.1269 - factorized_top_k/top_100_categorical_accuracy: 0.2194 - loss: 63291.1729 - regularization_loss: 0.0000e+00 - total_loss: 63291.1729 - val_factorized_top_k/top_1_categorical_accuracy: 1.4875e-04 - val_factorized_top_k/top_5_categorical_accuracy: 0.0030 - val_factorized_top_k/top_10_categorical_accuracy: 0.0073 - val_factorized_top_k/top_50_categorical_accuracy: 0.0445 - val_factorized_top_k/top_100_categorical_accuracy: 0.0849 - val_loss: 30291.8203 - val_regularization_loss: 0.0000e+00 - val_total_loss: 30291.8203\n"
1031 | ]
1032 | }
1033 | ]
1034 | },
1035 | {
1036 | "cell_type": "markdown",
1037 | "metadata": {
1038 | "id": "lvDnN_hQJ5fi"
1039 | },
1040 | "source": [
1041 | "`factorized_top_k/top_10_categorical_accuracy: 0.0538` would tell us that, on average, the true positive is in the top 10 retrieved items from the entire candidate set 5% of the time.\n",
1042 | "\n",
1043 | "If the candidate set is a large set, turn metric calculation off in training, and only run it in evaluation. Because this can be quite slow!"
1044 | ]
1045 | },
1046 | {
1047 | "cell_type": "code",
1048 | "metadata": {
1049 | "id": "3djrQCxhBZCV",
1050 | "colab": {
1051 | "base_uri": "https://localhost:8080/",
1052 | "height": 472
1053 | },
1054 | "outputId": "fb0af53f-81de-4fc6-fa68-e8a01649b8b9"
1055 | },
1056 | "source": [
1057 | "# Plot changes in model loss during training\n",
1058 | "import matplotlib.pyplot as plt\n",
1059 | "\n",
1060 | "plt.plot(history.history[\"loss\"])\n",
1061 | "plt.plot(history.history[\"val_loss\"])\n",
1062 | "plt.title(\"Model losses during training\")\n",
1063 | "plt.xlabel(\"epoch\")\n",
1064 | "plt.ylabel(\"loss\")\n",
1065 | "plt.legend([\"train\", \"test\"], loc=\"upper right\")\n",
1066 | "plt.show()"
1067 | ],
1068 | "execution_count": 17,
1069 | "outputs": [
1070 | {
1071 | "output_type": "display_data",
1072 | "data": {
1073 | "text/plain": [
1074 | ""
1075 | ],
1076 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAk0AAAHHCAYAAACiOWx7AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAABeWUlEQVR4nO3deXhTVcIG8DdNm6RbutDdbiwKlK0sUgOOwFCpUFEc/FRktAiIIKBQBMSFTccijiMoCI6O4Iw4ICo4Y2UpZXHEIlCoslaWQtnSFmiTrkmbnO+PtJeGLtyWlrT4/p4nT5N7zz05J6HNy7nnniiEEAJEREREVC8nRzeAiIiIqDVgaCIiIiKSgaGJiIiISAaGJiIiIiIZGJqIiIiIZGBoIiIiIpKBoYmIiIhIBoYmIiIiIhkYmoiIiIhkYGgiug0pFArMnz+/wcedOXMGCoUCq1evrrfczp07oVAosHPnzka1rzWLjIzEmDFjWk29jjZ//nwoFIpGHbt69WooFAqcOXOmaRtF1EgMTUTNpOoPvkKhwI8//lhjvxACYWFhUCgUePDBBx3QQiKgpKQE8+fP/10GYKKGYmgiamYajQZffPFFje27du3C+fPnoVarHdAqamkyMzPx8ccf3/LnLSkpwYIFC5otNL322msoLS1t1LFPPfUUSktLERER0cStImochiaiZjZs2DCsX78eFRUVdtu/+OIL9O7dG0FBQQ5qGTmaEEIKFGq1Gi4uLg5u0Y0VFxc3qLyzszM0Gk2jnkupVEKj0TT69B5RU2NoImpmo0aNwpUrV5CSkiJtM5vN+Oqrr/Dkk0/WekxxcTFmzJiBsLAwqNVqdOzYEX/9618hhLArZzKZMH36dPj7+8PT0xMPPfQQzp8/X2udFy5cwNixYxEYGAi1Wo0uXbrg008/bbqOAli/fj169+4NV1dX+Pn54c9//jMuXLhgV0av1+OZZ55BaGgo1Go1goOD8fDDD9vNW9m/fz/i4uLg5+cHV1dXtG3bFmPHjrWrx2q1YsmSJejSpQs0Gg0CAwPx3HPPIT8/366cnLpqI4TAm2++idDQULi5uWHQoEE4cuRIjXJ1zdmpbT5OZGQkHnzwQWzZsgV9+vSBq6srPvroI2lf9TlNVcfv3r0biYmJ8Pf3h7u7Ox555BHk5eXVeC3mz5+PkJAQqa1Hjx694TypM2fOwN/fHwCwYMEC6XRy1Xy4MWPGwMPDA6dOncKwYcPg6emJ0aNHAwD+97//4f/+7/8QHh4OtVqNsLAwTJ8+vcaoUm2vj0KhwJQpU7Bx40Z07dpV+ve4efNm2a/hjz/+iL59+0Kj0aBdu3b45z//WaN/v/76KwYMGABXV1eEhobizTffxKpVqzhPihrN2dENILrdRUZGQqfT4d///jeGDh0KANi0aRMMBgOeeOIJvP/++3blhRB46KGHsGPHDowbNw7R0dHYsmULZs6ciQsXLuC9996Tyo4fPx6ff/45nnzySfTr1w/bt29HfHx8jTbk5OTgnnvukT6s/P39sWnTJowbNw5GoxHTpk276X6uXr0azzzzDO6++24kJSUhJycHS5cuxe7du3Hw4EF4e3sDAEaOHIkjR45g6tSpiIyMRG5uLlJSUpCdnS09HjJkCPz9/fHyyy/D29sbZ86cwTfffGP3fM8995z0nC+88AKysrKwbNkyHDx4ELt374aLi4vsumozd+5cvPnmmxg2bBiGDRuGAwcOYMiQITCbzTf1OmVmZmLUqFF47rnn8Oyzz6Jjx471lp86dSp8fHwwb948nDlzBkuWLMGUKVOwbt06qcycOXOwePFiDB8+HHFxcfjll18QFxeHsrKyeuv29/fHihUrMGnSJDzyyCP405/+BADo3r27VKaiogJxcXG499578de//hVubm4AbAG5pKQEkyZNQps2bbB371588MEHOH/+PNavX3/D1+HHH3/EN998g+effx6enp54//33MXLkSGRnZ6NNmzb1Hnvy5Ek8+uijGDduHBISEvDpp59izJgx6N27N7p06QLA9p+EQYMGQaFQYM6cOXB3d8cnn3zC0+F0cwQRNYtVq1YJAGLfvn1i2bJlwtPTU5SUlAghhPi///s/MWjQICGEEBERESI+Pl46buPGjQKAePPNN+3qe/TRR4VCoRAnT54UQgiRkZEhAIjnn3/ertyTTz4pAIh58+ZJ28aNGyeCg4PF5cuX7co+8cQTwsvLS2pXVlaWACBWrVpVb9927NghAIgdO3YIIYQwm80iICBAdO3aVZSWlkrlvvvuOwFAzJ07VwghRH5+vgAg3nnnnTrr3rBhg/S61eV///ufACDWrFljt33z5s122+XUVZvc3FyhUqlEfHy8sFqt0vZXXnlFABAJCQnStnnz5ona/pRWvf9ZWVnStoiICAFAbN68uUb5iIgIu3qrjo+NjbVrw/Tp04VSqRQFBQVCCCH0er1wdnYWI0aMsKtv/vz5Ndpam7y8vBr/XqokJCQIAOLll1+usa/q30x1SUlJQqFQiLNnz0rbant9AAiVSiX9WxZCiF9++UUAEB988EGN16C21/CHH36QtuXm5gq1Wi1mzJghbZs6dapQKBTi4MGD0rYrV64IX1/fGnUSycXTc0S3wGOPPYbS0lJ89913KCwsxHfffVfnqbnvv/8eSqUSL7zwgt32GTNmQAiBTZs2SeUA1Ch3/aiREAJff/01hg8fDiEELl++LN3i4uJgMBhw4MCBm+rf/v37kZubi+eff95u/kp8fDw6deqE5ORkAICrqytUKhV27txZ4zRalaoRqe+++w7l5eW1llm/fj28vLxw//332/Wnd+/e8PDwwI4dO2TXVZtt27bBbDZj6tSpdqeWmmJErm3btoiLi5NdfsKECXZt+MMf/gCLxYKzZ88CAFJTU1FRUYHnn3/e7ripU6fedFurTJo0qcY2V1dX6X5xcTEuX76Mfv36QQiBgwcP3rDO2NhYtG/fXnrcvXt3aLVanD59+obHRkVF4Q9/+IP02N/fHx07drQ7dvPmzdDpdIiOjpa2+fr6SqcXiRqDoYnoFvD390dsbCy++OILfPPNN7BYLHj00UdrLXv27FmEhITA09PTbnvnzp2l/VU/nZyc7D54ANQ43ZOXl4eCggL8/e9/h7+/v93tmWeeAQDk5ubeVP+q2lTbqaZOnTpJ+9VqNd5++21s2rQJgYGBuO+++7B48WLo9Xqp/IABAzBy5EgsWLAAfn5+ePjhh7Fq1SqYTCapzIkTJ2AwGBAQEFCjT0VFRVJ/5NRVX3/uvPNOu+3+/v7w8fFpxCt0Tdu2bRtUPjw83O5x1fNXhc6qtnbo0MGunK+v7023FbBN5A4NDa2xPTs7G2PGjIGvry88PDzg7++PAQMGAAAMBsMN672+X4Ctb3WF6YYee/bs2RqvCVDzdSJqCM5pIrpFnnzySTz77LPQ6/UYOnSoNArS3KxWKwDgz3/+MxISEmotU30OS3ObNm0ahg8fjo0bN2LLli14/fXXkZSUhO3bt6Nnz55QKBT46quvsGfPHvz3v//Fli1bMHbsWLz77rvYs2cPPDw8YLVaERAQgDVr1tT6HFWTm+XUdbPqurLLYrHUur36CI0cSqWy1u3iuosCmotarYaTk/3/ry0WC+6//35cvXoVs2fPRqdOneDu7o4LFy5gzJgx0r+5+txMvxz9mtDvF0MT0S3yyCOP4LnnnsOePXvsJvFeLyIiAtu2bUNhYaHdaNPx48el/VU/rVYrTp06ZTfCk5mZaVdf1ZV1FosFsbGxTdkluzZXPfcf//hHu32ZmZk11tlp3749ZsyYgRkzZuDEiROIjo7Gu+++i88//1wqc8899+Cee+7BX/7yF3zxxRcYPXo01q5di/Hjx6N9+/bYtm0b+vfvLyuE1FdXff05ceIE2rVrJ23Py8urMRJSNZpTUFBgF4SrRoCaW1VbT548aTeKdeXKFVmjNo25nP/QoUP47bff8Nlnn+Hpp5+Wtle/QtTRIiIicPLkyRrba9tGJBdPzxHdIh4eHlixYgXmz5+P4cOH11lu2LBhsFgsWLZsmd329957DwqFQroCr+rn9VffLVmyxO6xUqnEyJEj8fXXX+Pw4cM1nu/6y9cbo0+fPggICMDKlSvtTn1t2rQJx44dk67oKykpqXFFV/v27eHp6Skdl5+fX2PEoGpeSlWZxx57DBaLBW+88UaNtlRUVKCgoEB2XbWJjY2Fi4sLPvjgA7vjr39tq9oPAD/88IO0rbi4GJ999lmd9TelwYMHw9nZGStWrLDbfv2/n7pUXQ1X9ZrJUTXSU/21EUJg6dKlsutobnFxcUhLS0NGRoa07erVq3WOThLJwZEmoluortNj1Q0fPhyDBg3Cq6++ijNnzqBHjx7YunUrvv32W0ybNk36kI6OjsaoUaPw4YcfwmAwoF+/fkhNTa31f9KLFi3Cjh07EBMTg2effRZRUVG4evUqDhw4gG3btuHq1as31S8XFxe8/fbbeOaZZzBgwACMGjVKWnIgMjIS06dPBwD89ttvGDx4MB577DFERUXB2dkZGzZsQE5ODp544gkAwGeffYYPP/wQjzzyCNq3b4/CwkJ8/PHH0Gq1GDZsGADbXKXnnnsOSUlJyMjIwJAhQ+Di4oITJ05g/fr1WLp0KR599FFZddXG398fL730EpKSkvDggw9i2LBhOHjwIDZt2gQ/Pz+7skOGDEF4eDjGjRuHmTNnQqlU4tNPP4W/vz+ys7Nv6nWVIzAwEC+++CLeffddPPTQQ3jggQfwyy+/SG290UiSq6sroqKisG7dOtx1113w9fVF165d0bVr1zqP6dSpE9q3b4+XXnoJFy5cgFarxddffy1rZOtWmTVrFj7//HPcf//9mDp1qrTkQHh4OK5evcoFM6lRGJqIWhgnJyf85z//wdy5c7Fu3TqsWrUKkZGReOeddzBjxgy7slUfzmvWrMHGjRvxxz/+EcnJyQgLC7MrFxgYiL1792LhwoX45ptv8OGHH6JNmzbo0qUL3n777SZp95gxY+Dm5oZFixZh9uzZ0kKMb7/9tnTaKiwsDKNGjUJqair+9a9/wdnZGZ06dcKXX36JkSNHArAFor1792Lt2rXIycmBl5cX+vbtizVr1tidflq5ciV69+6Njz76CK+88gqcnZ0RGRmJP//5z+jfv3+D6qrNm2++CY1Gg5UrV0qBc+vWrTXWwXJxccGGDRvw/PPP4/XXX0dQUBCmTZsGHx8faaJ9c3v77bfh5uaGjz/+GNu2bYNOp8PWrVtx7733ylqN+5NPPsHUqVMxffp0mM1mzJs3r97Q5OLigv/+97944YUXkJSUBI1Gg0ceeQRTpkxBjx49mrJrjRYWFoYdO3bghRdewFtvvQV/f39MnjwZ7u7ueOGFFxq9Sjn9vikEZ84REd12CgoK4OPjgzfffBOvvvqqo5vTYkybNg0fffQRioqK6pxQTlQXzmkiImrlavtC3Kr5VwMHDry1jWlBrn9drly5gn/961+49957GZioUXh6joiolVu3bh1Wr16NYcOGwcPDAz/++CP+/e9/Y8iQIdKpyt8jnU6HgQMHonPnzsjJycE//vEPGI1GvP76645uGrVSDE1ERK1c9+7d4ezsjMWLF8NoNEqTw998801HN82hhg0bhq+++gp///vfoVAo0KtXL/zjH//Afffd5+imUSvFOU1EREREMnBOExEREZEMDE1EREREMnBOUxOxWq24ePEiPD09uWgaERFRKyGEQGFhIUJCQmp8z+L1GJqayMWLF2ssKEhEREStw7lz5xAaGlpvGYamJlL1xarnzp2DVqt1cGuIiIhIDqPRiLCwMLsvSK8LQ1MTqTolp9VqGZqIiIhaGTlTazgRnIiIiEgGhiYiIiIiGRiaiIiIiGTgnCYiIqIWzmq1wmw2O7oZrZKLi0uTfUEzQxMREVELZjabkZWVBavV6uimtFre3t4ICgq66XUUGZqIiIhaKCEELl26BKVSibCwsBsuvkj2hBAoKSlBbm4uACA4OPim6mNoIiIiaqEqKipQUlKCkJAQuLm5Obo5rZKrqysAIDc3FwEBATd1qo6RlYiIqIWyWCwAAJVK5eCWtG5VgbO8vPym6mFoIiIiauH4naY3p6leP4YmIiIiIhkYmoiIiKhFi4yMxJIlSxzdDMeHpgsXLuDPf/4z2rRpA1dXV3Tr1g379++X9gshMHfuXAQHB8PV1RWxsbE4ceKEXR1Xr17F6NGjodVq4e3tjXHjxqGoqMiuzK+//oo//OEP0Gg0CAsLw+LFi2u0Zf369ejUqRM0Gg26deuG77//vnk6TUREdJsbOHAgpk2b1iR17du3DxMmTGiSum6GQ0NTfn4++vfvDxcXF2zatAlHjx7Fu+++Cx8fH6nM4sWL8f7772PlypX4+eef4e7ujri4OJSVlUllRo8ejSNHjiAlJQXfffcdfvjhB7sX12g0YsiQIYiIiEB6ejreeecdzJ8/H3//+9+lMj/99BNGjRqFcePG4eDBgxgxYgRGjBiBw4cP35oXow4l5gocvmBAhYXrcxAR0e1DCIGKigpZZf39/VvG1YPCgWbPni3uvffeOvdbrVYRFBQk3nnnHWlbQUGBUKvV4t///rcQQoijR48KAGLfvn1SmU2bNgmFQiEuXLgghBDiww8/FD4+PsJkMtk9d8eOHaXHjz32mIiPj7d7/piYGPHcc8/J6ovBYBAAhMFgkFVerh3Hc0TE7O9E59c3icc/+km8vemYSDmiF1eKTDc+mIiIWrXS0lJx9OhRUVpa6uimNEhCQoIAYHdbtWqVACC+//570atXL+Hi4iJ27NghTp48KR566CEREBAg3N3dRZ8+fURKSopdfREREeK9996THgMQH3/8sRgxYoRwdXUVHTp0EN9++22d7anvdWzI57dDR5r+85//oE+fPvi///s/BAQEoGfPnvj444+l/VlZWdDr9YiNjZW2eXl5ISYmBmlpaQCAtLQ0eHt7o0+fPlKZ2NhYODk54eeff5bK3HfffXaXbMbFxSEzMxP5+flSmerPU1Wm6nmuZzKZYDQa7W7N4UqRGZ4aZ5SYLdhz+io+3HkK4/+5H73eSMHAd3YgcV0GPt9zFkcvGmGximZpAxERtQxCCJSYKxxys2UVeZYuXQqdTodnn30Wly5dwqVLlxAWFgYAePnll7Fo0SIcO3YM3bt3R1FREYYNG4bU1FQcPHgQDzzwAIYPH47s7Ox6n2PBggV47LHH8Ouvv2LYsGEYPXo0rl69elOv7404dHHL06dPY8WKFUhMTMQrr7yCffv24YUXXoBKpUJCQgL0ej0AIDAw0O64wMBAaZ9er0dAQIDdfmdnZ/j6+tqVadu2bY06qvb5+PhAr9fX+zzXS0pKwoIFCxrZc/lG9g7FIz3vwMm8Ihw4m48D2fk4kF2Ak7lFOHOlBGeulOCbgxcAAO4qJXqEeaNXuA96RXijZ5gPfNy5tgcR0e2itNyCqLlbHPLcRxfGwU0lLzZ4eXlBpVLBzc0NQUFBAIDjx48DABYuXIj7779fKuvr64sePXpIj9944w1s2LAB//nPfzBlypQ6n2PMmDEYNWoUAOCtt97C+++/j7179+KBBx5ocN/kcmhoslqt6NOnD9566y0AQM+ePXH48GGsXLkSCQkJjmzaDc2ZMweJiYnSY6PRKKXopubkpMBdgZ64K9ATT/QNBwAYSspx8JwtQB3MzkdGdgEKTRX46dQV/HTqinRsOz939KwMUb3CfXBXoCeUTlzvg4iIHKP6mSEAKCoqwvz585GcnIxLly6hoqICpaWlNxxp6t69u3Tf3d0dWq1W+rqU5uLQ0BQcHIyoqCi7bZ07d8bXX38NAFI6zcnJsfu+mJycHERHR0tlrn+RKioqcPXqVen4oKAg5OTk2JWpenyjMlX7r6dWq6FWq2X3tal5ublgYMcADOxoG2WzWAVO5BbiwNmCytGofJzOK8bpy7bb1wfOAwA81M7oEeZlG40K90HPcG94u3E0ioioNXB1UeLowjiHPXdTcHd3t3v80ksvISUlBX/961/RoUMHuLq64tFHH4XZbK63HhcXF7vHCoWi2b/U2KGhqX///sjMzLTb9ttvvyEiIgIA0LZtWwQFBSE1NVUKSUajET///DMmTZoEANDpdCgoKEB6ejp69+4NANi+fTusVitiYmKkMq+++irKy8ulFzklJQUdO3aUrtTT6XRITU21uzwyJSUFOp2u2frflJROCnQK0qJTkBZPxthGowpKzDiYfS1EZWQXoMhUgd0nr2D3yWujUe393StP6dmC1J0BHnDiaBQRUYujUChknyJzNJVKJX0NTH12796NMWPG4JFHHgFgG3k6c+ZMM7eucRz6yk+fPh39+vXDW2+9hcceewx79+7F3//+d2kpAIVCgWnTpuHNN9/EnXfeibZt2+L1119HSEgIRowYAcA2MvXAAw/g2WefxcqVK1FeXo4pU6bgiSeeQEhICADgySefxIIFCzBu3DjMnj0bhw8fxtKlS/Hee+9JbXnxxRcxYMAAvPvuu4iPj8fatWuxf/9+u2UJWhtvNxUGdQrAoE7XRqN+yym0haizttN6py8X41Se7bY+3TYa5al2RnS4t+20XrhtbpSXm0t9T0VERGQnMjISP//8M86cOQMPD486R4HuvPNOfPPNNxg+fDgUCgVef/31Zh8xaiyHhqa7774bGzZswJw5c7Bw4UK0bdsWS5YswejRo6Uys2bNQnFxMSZMmICCggLce++92Lx5MzQajVRmzZo1mDJlCgYPHgwnJyeMHDkS77//vrTfy8sLW7duxeTJk9G7d2/4+flh7ty5dms59evXD1988QVee+01vPLKK7jzzjuxceNGdO3a9da8GLeA0kmBzsFadA7WYnSMbTTvarEZBytHog6cLcAv521zo/534jL+d+KydGyHAA/0CveWRqQ6+HM0ioiI6vbSSy8hISEBUVFRKC0txapVq2ot97e//Q1jx45Fv3794Ofnh9mzZzfbFek3SyEacg0h1cloNMLLywsGgwFardbRzWm0CosVmTmFtgnmlVfrnblSUqOcp8YZ0ZVX6vWO8EF0uDe0Go5GERE1pbKyMmRlZaFt27Z2gwXUMPW9jg35/G4dJ0bplnFWOqFLiBe6hHjhqXtso1FXikx2c6N+OWdAYZn9aJRCAdwZ4CFNMO8V4Y12fhyNIiKi2wdDE91QGw81YqMCERtlW8eqwmLFcX1h5Wm9AqSfzUf21RL8llOE33KKsHbfOQCAVuNcOS/KFqKiw7zhydEoIiJqpRiaqMGclU7oeocXut7hhacqLy68XGSqXHzTNiL16/kCGMsqsOu3POz6LQ+AbTTqrgBP28KblWGqvb87FAqORhERUcvH0ERNws9DjSFdgjCki21dq3KLFccvFUqn9A5k5+Pc1VJk5hQiM6cQ/95rG43ydnNBz7BrE8x7hHnDQ81/lkRE1PLw04mahYvSCd1CvdAt1AsJ/SIBALmFZdLcqIOVV+oVlJRjR2YedmTaRqOcFMBdgZ7SmlG9wr3R1o+jUURE5HgMTXTLBHhqENclCHGVo1HmCiuOXTJK36d34Gw+LhSU4ri+EMf1hfjiZ9sS+j5uLtKaUb3CbaNR7hyNIiKiW4yfPOQwKmcn9AjzRo8wbzzT37Yt11hmF6J+vWBAfkk5th/Pxfbjtq/LcVIAHYO06F35fXq9wn0Q0caNo1FERNSsGJqoRQnQavBA12A80NX2XYPmCiuOXjJWTjLPx8HsAlwoKMWxS0Ycu2TE53tso1G+7irb6uWVIapHmFer+aoBIiJqHfipQi2aytkJ0WG25QrGoi0AQG+oHI2qDFKHLxhxtdiMbcdyse2YbTTK9l18ntJyB73CfRDuy9EoIiJqPIYmanWCvDQY1i0Yw7rZRqNMFRYcuWgbjaqaaH7JUIYjF404ctGIf+05CwDw81AhOuxaiOoR6g1XVdN8azcREd3+GJqo1VM7K6W5TVUuGUpx4Oy1VcyPXDDicpEZ247lYNuxHACAc+V38fUK95au1gv1ceVoFBFRExg4cCCio6OxZMmSJqlvzJgxKCgowMaNG5ukvsZgaKLbUrCXK+K7uyK+u200qqzcNhpV9eXE6WfzkWM04dAFAw5dMOCztKrRKLVdiOoe6gWNC0ejiIiIX9jbZG6XL+z9vRBC4KKhTJoXdSC7AEcvGlBusf91cHZSICpEi17hPuhZueQBR6OI6FZprV/YO2bMGHz22Wd227KyslBUVISZM2fif//7H9zd3TFkyBC899578PPzAwB89dVXWLBgAU6ePAk3Nzf07NkT3377Ld555x0sWLDArr4dO3Zg4MCBstrTVF/Yy9DURBiaWr+ycgsOXzBUTjK3ndrLLTTVKOfvaRuN6l05GtX1Do5GEVHzqPFhLwRQXuKYxri42b4PSwaDwYChQ4eia9euWLhwoe1wFxd07twZ48ePx9NPP43S0lLMnj0bFRUV2L59Oy5duoTw8HAsXrwYjzzyCAoLC/G///0PTz/9NABg3LhxMBqNWLVqFQDA19cXKpVKVnuaKjTx9BxRJY2LEn0ifdEn0heAbTTqQkGptGbUwex8HLloRF6hCVuO5GDLEdvcKBelAlEhXtLim70ifBDipeFoFBE1vfIS4K0Qxzz3KxcBlbusol5eXlCpVHBzc0NQkG1B4zfffBM9e/bEW2+9JZX79NNPERYWht9++w1FRUWoqKjAn/70J0RERAAAunXrJpV1dXWFyWSS6nMEhiaiOigUCoT6uCHUxw0P9bD9kSort+DX84ZqSx4U4HKRCb+cK8Av5wqwavcZAECgVi1NTu8V4Y0uIRyNIqLft19++QU7duyAh4dHjX2nTp3CkCFDMHjwYHTr1g1xcXEYMmQIHn30Ufj4+NRSm2MwNBE1gMZFib5tfdG37bXRqPP5pXYh6uglI3KMJmw6rMemw3oAtrlRkX7uaO/vjvb+HrZbgAfa+btDq3FxZJeIqDVxcbON+DjquW9CUVERhg8fjrfffrvGvuDgYCiVSqSkpOCnn37C1q1b8cEHH+DVV1/Fzz//jLZt297UczcVhiaim6BQKBDm64YwXzc8HH0HAKDUbMGv5wtsp/Wybaf1LheZcTK3CCdziwDk2NURqFVfC1L+7mgfYLsfzFN8RHQ9hUL2KTJHU6lUsFgs0uNevXrh66+/RmRkJJyda48fCoUC/fv3R//+/TF37lxERERgw4YNSExMrFGfIzA0ETUxV5USMe3aIKZdGwC20ahLhjKczC3CqbzKW24xTuUVIbfQhByj7fbTqSt29biplGjn744O1Uam2vt7INLPDWpnnuojopYtMjISP//8M86cOQMPDw9MnjwZH3/8MUaNGoVZs2bB19cXJ0+exNq1a/HJJ59g//79SE1NxZAhQxAQEICff/4ZeXl56Ny5s1Tfli1bkJmZiTZt2sDLywsuLrd2pJ6hiaiZKRQKhHi7IsTbFffd5W+3z1hWjtN5xdcCVeXPs1dKUGK24PAFIw5fMNod46QAwnzd0N7fAx0CPOxO+fm4y7uShIioub300ktISEhAVFQUSktLkZWVhd27d2P27NkYMmQITCYTIiIi8MADD8DJyQlarRY//PADlixZAqPRiIiICLz77rsYOnQoAODZZ5/Fzp070adPHxQVFTVoyYGmwiUHmgiXHKCmVG6xIvtqSWWIKrYbpSosq6jzOF93lV2I6lA5OnWHjyuUTjzVR9TatNZ1mloaLjlAdBtzUTpJwac6IQTyikzS6b1TebZ5UqfzinGhoBRXi824WmzGvjP5dsepnJ3Qzs+9xrypdv7ucFPxzwARkRz8a0nUiigUCgR4ahDgqYGufRu7fSXmCpzOqwpTxdLpvtOXi2GusOK4vhDH9YU16rzD2xXtqkanKk/3dfD3gL+nmhPRiYiqYWgiuk24qZzR9Q4vdL3Dy267xSpwIb/02iT0ahPRrxSbcaGgFBcKSvG/E5ftjvPUOFdbHuHaKb+INm5wUTrdyq4REbUIDE1EtzmlkwLhbdwQ3sYNgzoF2O3LLzZXC1PF0kT07KslKCyrQMa5AmScK7A7xtlJgYg2bnZX9FWd8uOaU0R0O2NoIvod83FXoY/7ta+OqWKqsODslRLbBPTcIrtTfiVmS+X9YuCo/ZpT/p5q2+k9KUzZglWwVgMnTkQnajRes3Vzmur1Y2giohrUzkrcFeiJuwI97bYLIaA3luFUbjFO5hZemzuVV4Qcowl5hbbbntNX7Y5zdVFemzdVdVVfgDsi27jz62WI6qFU2n4/zGYzXF1dHdya1qukxPYlxze7rhNDExHJplAoEOzlimAvV9x7p5/dvsLKNaeqX9V3Kq8YZy4Xo7TcgiMXjThy0XhdfUCYj9u1ZRICri2T4Ms1p4jg7OwMNzc35OXlwcXFBU5OnE/YEEIIlJSUIDc3F97e3lIIbSyu09REuE4TUe3KLVacu1pid0XfycqfxnrWnPJxc6l1InqYrxvXnKLfFbPZjKysLFitVkc3pdXy9vZGUFBQrVcEN+Tzm6GpiTA0ETWMEAKXi8w1rug7lVeE8/mldR6nUjqhrZ+7XZCqWnPKXc3Bc7o9Wa1WmM1mRzejVXJxcal3hImhyQEYmoiaTqnZgtOX7a/oO5VXjNN5RTBV1P2/7RAvjf0VfZWn/AK45hQR1YGhyQEYmoian9UqcKGgVDq9V3XK73ReES4X1f2/cA+1s91K6LbJ6O4I93WHyplzRIh+zxiaHIChicixCkrM141M2ULV2SvFsNbxV07ppECErxvaVf/y48pg5eXKNaeIfg8YmhyAoYmoZTJVWJB9pUQKUdKXH+cWodhsqfM4Pw+1/ff0+bkjzNcVoT5uXCaB6DbCL+wlIqqkdlbizkBP3FnLmlM5RlO1iehVV/UVQ28sw+UiEy4XmfBz1tUadfp7qhHq44owHzcpSFXdD/Zy5Sk/otsUR5qaCEeaiG4fRaYKnL7uqr6sy8U4n1+KIlPdyyQAgJMCCNJqEOrrVi1YVd73dUOQVsMlE4haEJ6ecwCGJqLbnxAChtJynLtainP5JTifX1LtfinO55egrLz+tXScnRQI8Xa1jVB520anwqoFLH9e6Ud0S/H0HBFRM1AoFPB2U8HbTYVuoV419gshkFdkwvn8Upy7ei1Inbtq+3mhoBTlFoHsqyXIvloC4EqNOtTOTrijjlN/oT5u8HFzYagichCGJiKiJqJQKBDgqUGApwa9wn1q7LdYBXKMZVKoqhqhqgpYlwylMFVYcTqvGKfzimt9DneVUhqZCq1+6q8yWHlqeNUfUXPh6bkmwtNzRHSzyi1WXCooq/XU37mrJcgtNN2wDi9XlzpP/YX6uMFVxSv/iKrj6TkiolbIRemE8DZuCG/jVuv+snILLhRUjVLZTvmdrxasrhabYSgth+FCOQ5fMNZah5+HqtYRqlAfN9zhzSv/iOrD0ERE1EpoXJTSiua1KTJV4EItp/7O5Zfi/NUSFJoqcLnIjMtFZmScK6hxvKLqyr+qkanrglWQVgNnJUMV/X7x9FwT4ek5ImrpDCXldZ76Oyfzyr9gb03lqb6qYHVtWQV/DzWcuJwCtTI8PUdERDV4ubnAy80LXe+o/cq/K8Vmu1N/VVf9nc8vxYX8UpgtVlvYulpaa/0qZyeEervWGKGyXQHoCl93Fa/8o1bNoaFp/vz5WLBggd22jh074vjx4wCAgQMHYteuXXb7n3vuOaxcuVJ6nJ2djUmTJmHHjh3w8PBAQkICkpKS4Ox8rWs7d+5EYmIijhw5grCwMLz22msYM2aMXb3Lly/HO++8A71ejx49euCDDz5A3759m7jHREQtk0KhgJ+HGn4eavSs5co/q1Ugp7DM7mq/6qcBLxnKYK6w4vTlYpy+XPuVf24qZY0FP0OrBSt+3x+1dA4faerSpQu2bdsmPa4edgDg2WefxcKFC6XHbm7XJkhaLBbEx8cjKCgIP/30Ey5duoSnn34aLi4ueOuttwAAWVlZiI+Px8SJE7FmzRqkpqZi/PjxCA4ORlxcHABg3bp1SExMxMqVKxETE4MlS5YgLi4OmZmZCAgIaM7uExG1Ck5OCgR72b4m5u5I3xr7yy1W6A2VV/7Vcuovx2hCidmC33KK8FtOUa3PodU4XzdB/dpK6qE+rnBTOfwji37nHDqnaf78+di4cSMyMjJq3T9w4EBER0djyZIlte7ftGkTHnzwQVy8eBGBgYEAgJUrV2L27NnIy8uDSqXC7NmzkZycjMOHD0vHPfHEEygoKMDmzZsBADExMbj77ruxbNkyAIDVakVYWBimTp2Kl19+WVZfOKeJiKhuZeUWXCwotTv1J62kfrUEV4rNN6yjjbuqzlN/d/i4Qu3M5RSo4VrVnKYTJ04gJCQEGo0GOp0OSUlJCA8Pl/avWbMGn3/+OYKCgjB8+HC8/vrr0mhTWloaunXrJgUmAIiLi8OkSZNw5MgR9OzZE2lpaYiNjbV7zri4OEybNg0AYDabkZ6ejjlz5kj7nZycEBsbi7S0tDrbbTKZYDJdWzPFaKz98l4iIrJd+dfO3wPt6rjyr9hUcW05harTf9XmVRnLKnCl2IwrxWb8UsuVfwAQqFXbj1BVm6ge7MUr/+jmOTQ0xcTEYPXq1ejYsSMuXbqEBQsW4A9/+AMOHz4MT09PPPnkk4iIiEBISAh+/fVXzJ49G5mZmfjmm28AAHq93i4wAZAe6/X6essYjUaUlpYiPz8fFoul1jJVc6tqk5SUVGM+FhERNY672hl3BXrirkDPWvfbvvOv+lfT2Aer0nILcowm5BhN2H82v8bxSicFAj3VCNBqEKhVI1CrQaBWgwBPNYK8bPcDPTXQujpzsjrVyaGhaejQodL97t27IyYmBhEREfjyyy8xbtw4TJgwQdrfrVs3BAcHY/DgwTh16hTat2/viCZL5syZg8TEROmx0WhEWFiYA1tERHT78nJ1gdcddV/5d7XYXOepv/OVV/5dNJThoqGs3udROzshUKtBkFaDAClcXQtZVY85v+r3qUW9697e3rjrrrtw8uTJWvfHxMQAAE6ePIn27dsjKCgIe/futSuTk5MDAAgKCpJ+Vm2rXkar1cLV1RVKpRJKpbLWMlV11EatVkOtVjesg0RE1OQUCgXaeKjRxkON6DDvGvutVoHcQhP0xjLkGMuQayyrvG+qfGxCTmEZCkrKYaqwVvtC5bp5qp0RoK0cpfLUXDeCZfvp76nmPKvbTIsKTUVFRTh16hSeeuqpWvdXTRgPDg4GAOh0OvzlL39Bbm6udJVbSkoKtFotoqKipDLff/+9XT0pKSnQ6XQAAJVKhd69eyM1NRUjRowAYJsInpqaiilTpjR1F4mI6BZzclIgyEuDIC9NveXKyi1SgMqpDFXXApYtXOmNZSgxW1BoqkBhXgVO1fHFylV83VUI8FRLo1eB2qpThLb7QVoN2niooeSioK2CQ6+ee+mllzB8+HBERETg4sWLmDdvHjIyMnD06FEYjUZ88cUXGDZsGNq0aYNff/0V06dPR2hoqLR2k8ViQXR0NEJCQrB48WLo9Xo89dRTGD9+vN2SA127dsXkyZMxduxYbN++HS+88AKSk5PtlhxISEjARx99hL59+2LJkiX48ssvcfz48RpznerCq+eIiH4fikwVtlBlKKsMWCa7UFV132ypf4X1Kk4KwL8yWAV4ahDkpUagZ+Wcq2qnBn3cXDjfqhm0mqvnzp8/j1GjRuHKlSvw9/fHvffeiz179sDf3x9lZWXYtm0blixZguLiYoSFhWHkyJF47bXXpOOVSiW+++47TJo0CTqdDu7u7khISLBb16lt27ZITk7G9OnTsXTpUoSGhuKTTz6RAhMAPP7448jLy8PcuXOh1+sRHR2NzZs3yw5MRET0++GhdoZHPd8BCNjmWRWUlF8LVYbK0atC+xGsvEITrALSJHbAUGedKqWT3TwrW8CqPB1Y7RShp4aLhDYXfvdcE+FIExERNZTFKnClyCSNVukr51zlSKcJbQFLzjpWVdxVyhqjVNdfJRigVUPjwvlWQCsaaSIiIvo9UzopEKC1jRJ1Q80rA6uYKizIKzRJIcoWsCrvVztFWFhWgWKzpd6vs6ni5epyw6sE/TzUcOH6VhKGJiIiohZO7axEqI8bQn3c6i1XYq6QApR0ZaA0gmUbvdIbymCqsMJQWg5DaTkycwrrrE+hANq4q6VJ67VdJRio1cDXTQWn38FkdoYmIiKi24Sbyhlt/ZzR1s+9zjJCCBjLKmpZesF2v+oUYW6hCRVWgctFJlwuMuHIxbq/+cLZSYGAysVDa7tKsCpcaTWte/FQhiYiIqLfEYVCYVss1NUFd9axAjtgW9/qaom5xpWBOdedFrxcZAtXchYP1bg42c2rCtLWvEqwJS8e2jJbRURERA7l5KSAn4dtXlOXkLrLlVusuFxtMnt9i4eWlVtx9koJzl65weKhGudro1SeGgR6aRDoqUbHIC107ds0cU/lY2giIiKiRnNROiHYyxXBXq71lmvQ4qFlFSgsK8LJ3CK7OoZ1C2JoIiIiotubxkWJ8DZuCG9T/2T2+hYP7R3hc4taWzuGJiIiImox5Cwe6ihcfIGIiIhIBoYmIiIiIhkYmoiIiIhkYGgiIiIikoGhiYiIiEgGhiYiIiIiGRiaiIiIiGRgaCIiIiKSgaGJiIiISAaGJiIiIiIZGJqIiIiIZGBoIiIiIpKBoYmIiIhIBoYmIiIiIhkYmoiIiIhkYGgiIiIikoGhiYiIiEgGhiYiIiIiGRiaiIiIiGRgaCIiIiKSgaGJiIiISAaGJiIiIiIZGJqIiIiIZGBoIiIiIpKBoYmIiIhIBoYmIiIiIhkYmoiIiIhkYGgiIiIikoGhiYiIiEgGhiYiIiIiGRiaiIiIiGRgaCIiIiKSgaGJiIiISAaGJiIiIiIZGJqIiIiIZGBoIiIiIpKBoYmIiIhIBoeGpvnz50OhUNjdOnXqJO0vKyvD5MmT0aZNG3h4eGDkyJHIycmxqyM7Oxvx8fFwc3NDQEAAZs6ciYqKCrsyO3fuRK9evaBWq9GhQwesXr26RluWL1+OyMhIaDQaxMTEYO/evc3SZyIiImqdHD7S1KVLF1y6dEm6/fjjj9K+6dOn47///S/Wr1+PXbt24eLFi/jTn/4k7bdYLIiPj4fZbMZPP/2Ezz77DKtXr8bcuXOlMllZWYiPj8egQYOQkZGBadOmYfz48diyZYtUZt26dUhMTMS8efNw4MAB9OjRA3FxccjNzb01LwIRERG1fMKB5s2bJ3r06FHrvoKCAuHi4iLWr18vbTt27JgAINLS0oQQQnz//ffCyclJ6PV6qcyKFSuEVqsVJpNJCCHErFmzRJcuXezqfvzxx0VcXJz0uG/fvmLy5MnSY4vFIkJCQkRSUpLsvhgMBgFAGAwG2ccQERGRYzXk89vhI00nTpxASEgI2rVrh9GjRyM7OxsAkJ6ejvLycsTGxkplO3XqhPDwcKSlpQEA0tLS0K1bNwQGBkpl4uLiYDQaceTIEalM9TqqylTVYTabkZ6eblfGyckJsbGxUpnamEwmGI1GuxsRERHdvhwammJiYrB69Wps3rwZK1asQFZWFv7whz+gsLAQer0eKpUK3t7edscEBgZCr9cDAPR6vV1gqtpfta++MkajEaWlpbh8+TIsFkutZarqqE1SUhK8vLykW1hYWKNeAyIiImodnB355EOHDpXud+/eHTExMYiIiMCXX34JV1dXB7bsxubMmYPExETpsdFoZHAiIiK6jTn89Fx13t7euOuuu3Dy5EkEBQXBbDajoKDArkxOTg6CgoIAAEFBQTWupqt6fKMyWq0Wrq6u8PPzg1KprLVMVR21UavV0Gq1djciIiK6fbWo0FRUVIRTp04hODgYvXv3houLC1JTU6X9mZmZyM7Ohk6nAwDodDocOnTI7iq3lJQUaLVaREVFSWWq11FVpqoOlUqF3r1725WxWq1ITU2VyhARERE59Oq5GTNmiJ07d4qsrCyxe/duERsbK/z8/ERubq4QQoiJEyeK8PBwsX37drF//36h0+mETqeTjq+oqBBdu3YVQ4YMERkZGWLz5s3C399fzJkzRypz+vRp4ebmJmbOnCmOHTsmli9fLpRKpdi8ebNUZu3atUKtVovVq1eLo0ePigkTJghvb2+7q/JuhFfPERERtT4N+fx26Jym8+fPY9SoUbhy5Qr8/f1x7733Ys+ePfD39wcAvPfee3BycsLIkSNhMpkQFxeHDz/8UDpeqVTiu+++w6RJk6DT6eDu7o6EhAQsXLhQKtO2bVskJydj+vTpWLp0KUJDQ/HJJ58gLi5OKvP4448jLy8Pc+fOhV6vR3R0NDZv3lxjcjgRERH9fimEEMLRjbgdGI1GeHl5wWAwcH4TERFRK9GQz+8WNaeJiIiIqKViaCIiIiKSgaGJiIiISAaGJiIiIiIZGJqIiIiIZGBoIiIiIpKBoYmIiIhIBoYmIiIiIhkYmoiIiIhkYGgiIiIikoGhiYiIiEgGhiYiIiIiGRiaiIiIiGRgaCIiIiKSgaGJiIiISAaGJiIiIiIZGJqIiIiIZGBoIiIiIpKBoYmIiIhIBoYmIiIiIhkYmoiIiIhkYGgiIiIikoGhiYiIiEgGhiYiIiIiGRiaiIiIiGRgaCIiIiKSgaGJiIiISAaGJiIiIiIZGJqIiIiIZGBoIiIiIpKBoYmIiIhIBoYmIiIiIhkYmoiIiIhkYGgiIiIikoGhiYiIiEgGhiYiIiIiGRiaiIiIiGRgaCIiIiKSgaGJiIiISIZGhabPPvsMycnJ0uNZs2bB29sb/fr1w9mzZ5uscUREREQtRaNC01tvvQVXV1cAQFpaGpYvX47FixfDz88P06dPb9IGEhEREbUEzo056Ny5c+jQoQMAYOPGjRg5ciQmTJiA/v37Y+DAgU3ZPiIiIqIWoVEjTR4eHrhy5QoAYOvWrbj//vsBABqNBqWlpU3XOiIiIqIWolEjTffffz/Gjx+Pnj174rfffsOwYcMAAEeOHEFkZGRTto+IiIioRWjUSNPy5cuh0+mQl5eHr7/+Gm3atAEApKenY9SoUY1qyKJFi6BQKDBt2jRp28CBA6FQKOxuEydOtDsuOzsb8fHxcHNzQ0BAAGbOnImKigq7Mjt37kSvXr2gVqvRoUMHrF69utY+RUZGQqPRICYmBnv37m1UP4iIiOj21KiRJm9vbyxbtqzG9gULFjSqEfv27cNHH32E7t2719j37LPPYuHChdJjNzc36b7FYkF8fDyCgoLw008/4dKlS3j66afh4uKCt956CwCQlZWF+Ph4TJw4EWvWrEFqairGjx+P4OBgxMXFAQDWrVuHxMRErFy5EjExMViyZAni4uKQmZmJgICARvWJiIiIbi+NGmnavHkzfvzxR+nx8uXLER0djSeffBL5+fkNqquoqAijR4/Gxx9/DB8fnxr73dzcEBQUJN20Wq20b+vWrTh69Cg+//xzREdHY+jQoXjjjTewfPlymM1mAMDKlSvRtm1bvPvuu+jcuTOmTJmCRx99FO+9955Uz9/+9jc8++yzeOaZZxAVFYWVK1fCzc0Nn376aUNfGiIiIrpNNSo0zZw5E0ajEQBw6NAhzJgxA8OGDUNWVhYSExMbVNfkyZMRHx+P2NjYWvevWbMGfn5+6Nq1K+bMmYOSkhJpX1paGrp164bAwEBpW1xcHIxGI44cOSKVub7uuLg4pKWlAQDMZjPS09Ptyjg5OSE2NlYqQ0RERNSo03NZWVmIiooCAHz99dd48MEH8dZbb+HAgQPSpHA51q5diwMHDmDfvn217n/yyScRERGBkJAQ/Prrr5g9ezYyMzPxzTffAAD0er1dYAIgPdbr9fWWMRqNKC0tRX5+PiwWS61ljh8/XmfbTSYTTCaT9LgqRBIREdHtqVGhSaVSSSM+27Ztw9NPPw0A8PX1lR0ezp07hxdffBEpKSnQaDS1lpkwYYJ0v1u3bggODsbgwYNx6tQptG/fvjFNbzJJSUmNnsNFRERErU+jTs/de++9SExMxBtvvIG9e/ciPj4eAPDbb78hNDRUVh3p6enIzc1Fr1694OzsDGdnZ+zatQvvv/8+nJ2dYbFYahwTExMDADh58iQAICgoCDk5OXZlqh4HBQXVW0ar1cLV1RV+fn5QKpW1lqmqozZz5syBwWCQbufOnZPVbyIiImqdGhWali1bBmdnZ3z11VdYsWIF7rjjDgDApk2b8MADD8iqY/DgwTh06BAyMjKkW58+fTB69GhkZGRAqVTWOCYjIwMAEBwcDADQ6XQ4dOgQcnNzpTIpKSnQarXS6UOdTofU1FS7elJSUqDT6QDYRs169+5tV8ZqtSI1NVUqUxu1Wg2tVmt3IyIiotuYaEEGDBggXnzxRSGEECdPnhQLFy4U+/fvF1lZWeLbb78V7dq1E/fdd59UvqKiQnTt2lUMGTJEZGRkiM2bNwt/f38xZ84cqczp06eFm5ubmDlzpjh27JhYvny5UCqVYvPmzVKZtWvXCrVaLVavXi2OHj0qJkyYILy9vYVer5fddoPBIAAIg8Fw8y8EERER3RIN+fxu1JwmwLZG0saNG3Hs2DEAQJcuXfDQQw/VOkLUGCqVCtu2bcOSJUtQXFyMsLAwjBw5Eq+99ppURqlU4rvvvsOkSZOg0+ng7u6OhIQEu3Wd2rZti+TkZEyfPh1Lly5FaGgoPvnkE2mNJgB4/PHHkZeXh7lz50Kv1yM6OhqbN2+uMTmciIiIfr8UQgjR0INOnjyJYcOG4cKFC+jYsSMAIDMzE2FhYUhOTnb4JG1HMBqN8PLygsFg4Kk6IiKiVqIhn9+NmtP0wgsvoH379jh37hwOHDiAAwcOIDs7G23btsULL7zQqEYTERERtWSNOj23a9cu7NmzB76+vtK2Nm3aYNGiRejfv3+TNY6IiIiopWjUSJNarUZhYWGN7UVFRVCpVDfdKCIiIqKWplGh6cEHH8SECRPw888/QwgBIQT27NmDiRMn4qGHHmrqNhIRERE5XKNC0/vvv4/27dtDp9NBo9FAo9GgX79+6NChA5YsWdLETSQiIiJyvEbNafL29sa3336LkydPSksOdO7cGR06dGjSxhERERG1FLJDU2JiYr37d+zYId3/29/+1vgWEREREbVAskPTwYMHZZVTKBSNbgwRERFRSyU7NFUfSSIiIiL6vWnURHAiIiKi3xuGJiIiIiIZGJqIiIiIZGBoIiIiIpKBoYmIiIhIBoYmIiIiIhkYmoiIiIhkYGgiIiIikoGhiYiIiEgGhiYiIiIiGRiaiIiIiGRgaCIiIiKSgaGJiIiISAaGJiIiIiIZGJqIiIiIZGBoIiIiIpKBoYmIiIhIBoYmIiIiIhkYmoiIiIhkYGgiIiIikoGhiYiIiEgGhiYiIiIiGRiaiIiIiGRgaCIiIiKSgaGJiIiISAaGJiIiIiIZGJqIiIiIZGBoIiIiIpKBoYmIiIhIBoYmIiIiIhkYmoiIiIhkYGgiIiIikoGhiYiIiEgGhiYiIiIiGRiaiIiIiGRgaCIiIiKSocWEpkWLFkGhUGDatGnStrKyMkyePBlt2rSBh4cHRo4ciZycHLvjsrOzER8fDzc3NwQEBGDmzJmoqKiwK7Nz50706tULarUaHTp0wOrVq2s8//LlyxEZGQmNRoOYmBjs3bu3ObpJRERErVSLCE379u3DRx99hO7du9ttnz59Ov773/9i/fr12LVrFy5evIg//elP0n6LxYL4+HiYzWb89NNP+Oyzz7B69WrMnTtXKpOVlYX4+HgMGjQIGRkZmDZtGsaPH48tW7ZIZdatW4fExETMmzcPBw4cQI8ePRAXF4fc3Nzm7zwRERG1DsLBCgsLxZ133ilSUlLEgAEDxIsvviiEEKKgoEC4uLiI9evXS2WPHTsmAIi0tDQhhBDff/+9cHJyEnq9XiqzYsUKodVqhclkEkIIMWvWLNGlSxe753z88cdFXFyc9Lhv375i8uTJ0mOLxSJCQkJEUlKS7H4YDAYBQBgMBvmdJyIiIodqyOe3w0eaJk+ejPj4eMTGxtptT09PR3l5ud32Tp06ITw8HGlpaQCAtLQ0dOvWDYGBgVKZuLg4GI1GHDlyRCpzfd1xcXFSHWazGenp6XZlnJycEBsbK5WpjclkgtFotLsRERHR7cvZkU++du1aHDhwAPv27auxT6/XQ6VSwdvb2257YGAg9Hq9VKZ6YKraX7WvvjJGoxGlpaXIz8+HxWKptczx48frbHtSUhIWLFggr6NERETU6jlspOncuXN48cUXsWbNGmg0Gkc1o9HmzJkDg8Eg3c6dO+foJhEREVEzclhoSk9PR25uLnr16gVnZ2c4Oztj165deP/99+Hs7IzAwECYzWYUFBTYHZeTk4OgoCAAQFBQUI2r6aoe36iMVquFq6sr/Pz8oFQqay1TVUdt1Go1tFqt3Y2IiIhuXw4LTYMHD8ahQ4eQkZEh3fr06YPRo0dL911cXJCamiodk5mZiezsbOh0OgCATqfDoUOH7K5yS0lJgVarRVRUlFSmeh1VZarqUKlU6N27t10Zq9WK1NRUqQwRERGRw+Y0eXp6omvXrnbb3N3d0aZNG2n7uHHjkJiYCF9fX2i1WkydOhU6nQ733HMPAGDIkCGIiorCU089hcWLF0Ov1+O1117D5MmToVarAQATJ07EsmXLMGvWLIwdOxbbt2/Hl19+ieTkZOl5ExMTkZCQgD59+qBv375YsmQJiouL8cwzz9yiV4OIiIhaOodOBL+R9957D05OThg5ciRMJhPi4uLw4YcfSvuVSiW+++47TJo0CTqdDu7u7khISMDChQulMm3btkVycjKmT5+OpUuXIjQ0FJ988gni4uKkMo8//jjy8vIwd+5c6PV6REdHY/PmzTUmhxMREdHvl0IIIRzdiNuB0WiEl5cXDAYD5zcRERG1Eg35/Hb4Ok1ERERErQFDExEREZEMDE1EREREMjA0EREREcnA0EREREQkA0MTERERkQwMTUREREQyMDQRERERycDQRERERCQDQxMRERGRDAxNRERERDIwNBERERHJwNBEREREJANDExEREZEMDE1EREREMjA0EREREcnA0EREREQkA0MTERERkQwMTUREREQyMDQRERERycDQRERERCQDQxMRERGRDAxNRERERDIwNBERERHJwNBEREREJANDExEREZEMDE1EREREMjA0EREREcnA0EREREQkA0MTERERkQwMTUREREQyMDQRERERycDQRERERCQDQxMRERGRDAxNRERERDIwNBERERHJwNBEREREJANDExEREZEMDE1EREREMjA0EREREcnA0EREREQkA0MTERERkQwMTUREREQyMDQRERERyeDQ0LRixQp0794dWq0WWq0WOp0OmzZtkvYPHDgQCoXC7jZx4kS7OrKzsxEfHw83NzcEBARg5syZqKiosCuzc+dO9OrVC2q1Gh06dMDq1atrtGX58uWIjIyERqNBTEwM9u7d2yx9JiIiotbJoaEpNDQUixYtQnp6Ovbv348//vGPePjhh3HkyBGpzLPPPotLly5Jt8WLF0v7LBYL4uPjYTab8dNPP+Gzzz7D6tWrMXfuXKlMVlYW4uPjMWjQIGRkZGDatGkYP348tmzZIpVZt24dEhMTMW/ePBw4cAA9evRAXFwccnNzb80LQURERC2eQgghHN2I6nx9ffHOO+9g3LhxGDhwIKKjo7FkyZJay27atAkPPvggLl68iMDAQADAypUrMXv2bOTl5UGlUmH27NlITk7G4cOHpeOeeOIJFBQUYPPmzQCAmJgY3H333Vi2bBkAwGq1IiwsDFOnTsXLL78sq91GoxFeXl4wGAzQarU38QoQERHRrdKQz+8WM6fJYrFg7dq1KC4uhk6nk7avWbMGfn5+6Nq1K+bMmYOSkhJpX1paGrp16yYFJgCIi4uD0WiURqvS0tIQGxtr91xxcXFIS0sDAJjNZqSnp9uVcXJyQmxsrFSGiIiIyNnRDTh06BB0Oh3Kysrg4eGBDRs2ICoqCgDw5JNPIiIiAiEhIfj1118xe/ZsZGZm4ptvvgEA6PV6u8AEQHqs1+vrLWM0GlFaWor8/HxYLJZayxw/frzOdptMJphMJumx0Whs5CtARERErYHDQ1PHjh2RkZEBg8GAr776CgkJCdi1axeioqIwYcIEqVy3bt0QHByMwYMH49SpU2jfvr0DWw0kJSVhwYIFDm0DERER3ToOPz2nUqnQoUMH9O7dG0lJSejRoweWLl1aa9mYmBgAwMmTJwEAQUFByMnJsStT9TgoKKjeMlqtFq6urvDz84NSqay1TFUdtZkzZw4MBoN0O3fuXAN6TURERK2Nw0PT9axWq91pr+oyMjIAAMHBwQAAnU6HQ4cO2V3llpKSAq1WK53i0+l0SE1NtasnJSVFmjelUqnQu3dvuzJWqxWpqal2c6uup1arpaUSqm5ERER0+3Lo6bk5c+Zg6NChCA8PR2FhIb744gvs3LkTW7ZswalTp/DFF19g2LBhaNOmDX799VdMnz4d9913H7p37w4AGDJkCKKiovDUU09h8eLF0Ov1eO211zB58mSo1WoAwMSJE7Fs2TLMmjULY8eOxfbt2/Hll18iOTlZakdiYiISEhLQp08f9O3bF0uWLEFxcTGeeeYZh7wuRERE1AIJBxo7dqyIiIgQKpVK+Pv7i8GDB4utW7cKIYTIzs4W9913n/D19RVqtVp06NBBzJw5UxgMBrs6zpw5I4YOHSpcXV2Fn5+fmDFjhigvL7crs2PHDhEdHS1UKpVo166dWLVqVY22fPDBByI8PFyoVCrRt29fsWfPngb1xWAwCAA12kdEREQtV0M+v1vcOk2tFddpIiIian1a5TpNRERERC0ZQxMRERGRDAxNRERERDIwNBERERHJwNBEREREJANDExEREZEMDE1EREREMjA0EREREcnA0EREREQkA0MTERERkQwMTUREREQyMDQRERERycDQRERERCQDQxMRERGRDAxNRERERDIwNBERERHJwNBEREREJANDExEREZEMDE1EREREMjA0EREREcnA0EREREQkA0MTERERkQwMTUREREQyMDQRERERycDQRERERCQDQxMRERGRDAxNRERERDIwNBERERHJwNBEREREJANDExEREZEMDE1EREREMjA0EREREcnA0EREREQkA0MTERERkQwMTUREREQyMDQRERERycDQRERERCQDQxMRERGRDAxNRERERDIwNBERERHJwNBEREREJANDExEREZEMDE1EREREMjg7ugF0A5dPAMf+A6i1gMoDUHtW3jzst6ncAYXC0a0lIiK6bTE0tXSXfgFSF964nMLJPlRJ9yvDld02z1rKVbs5axjAiIjo1hICqDABFhNQYQYs5mr3K3+6egN+dzqsiQ4NTStWrMCKFStw5swZAECXLl0wd+5cDB06FABQVlaGGTNmYO3atTCZTIiLi8OHH36IwMBAqY7s7GxMmjQJO3bsgIeHBxISEpCUlARn52td27lzJxITE3HkyBGEhYXhtddew5gxY+zasnz5crzzzjvQ6/Xo0aMHPvjgA/Tt27fZX4Mb8goDov8MmIyAuQgwFQKmqp+FgLkQEFbbzWS03W6Wk3NlmNJWC17VA5b22jZp+/XbqgKY6ubbQ0RETUsIWyipMFX7eV1AsVTtuz68mOQdW2ECLOX17LuuDmv5jdvd7TFg5MfN//rUwaGhKTQ0FIsWLcKdd94JIQQ+++wzPPzwwzh48CC6dOmC6dOnIzk5GevXr4eXlxemTJmCP/3pT9i9ezcAwGKxID4+HkFBQfjpp59w6dIlPP3003BxccFbb70FAMjKykJ8fDwmTpyINWvWIDU1FePHj0dwcDDi4uIAAOvWrUNiYiJWrlyJmJgYLFmyBHFxccjMzERAQIDDXh8AQHiM7VYXIYDykmphqnq4uu5W2/bq28xFtjqtFUBZge12s5TqaqGr+ohWLQGr1m2V21WegJIDo0TUCglRS3i4meAh99h6go/F7OhX5cacnG2fIc6qaz/dfB3aJIUQQji0Bdfx9fXFO++8g0cffRT+/v744osv8OijjwIAjh8/js6dOyMtLQ333HMPNm3ahAcffBAXL16URp9WrlyJ2bNnIy8vDyqVCrNnz0ZycjIOHz4sPccTTzyBgoICbN68GQAQExODu+++G8uWLQMAWK1WhIWFYerUqXj55ZdltdtoNMLLywsGgwFarbYpX5Jbx2q9FqLqDFjGWka7qrZX21ZR2vTtc3atOaerxinG6+Z61XY6UuUBOPEaCKLbhhCA1WIbqbBW2EKGtaKW++ZGhhZzI49t5QFFqQaUqprbnKu21/azqpyqlm0y6qh+zC36O92Qz+8W8193i8WC9evXo7i4GDqdDunp6SgvL0dsbKxUplOnTggPD5dCU1paGrp162Z3ui4uLg6TJk3CkSNH0LNnT6SlpdnVUVVm2rRpAACz2Yz09HTMmTNH2u/k5ITY2FikpaXV2V6TyQSTySQ9Nhqb4LSYozk5ARqt7XazLBW2U4fVA5a5egirCljXj4zVUrbqD05Fqe1WnHvz7at1/tcN5nrVdirSxY3zv6h1EKKOIFFeGTYs1e5X2H6H7UJIZSipM5BU3b9RXdfXW19d19Vrqaj9vrXC0a9uw9gFlOvDREPDRV2hpZY66juG/5GUxeGh6dChQ9DpdCgrK4OHhwc2bNiAqKgoZGRkQKVSwdvb2658YGAg9Ho9AECv19sFpqr9VfvqK2M0GlFaWor8/HxYLJZayxw/frzOdiclJWHBggWN6vPvgtIZcPWx3W5WhamWU4/VRr1udNqxellhsdVpLrLdCi/dXNsUTrWfdrx+TpjK3VYWlQFLobDdlwKXop5tuPExNfbLeB5pP5qhToX0sP6+NbTO6vuvf41upk45r4e4wQf+9WHBUs8Hfh0hpEH1Vg8hMp6j6t/+74mTM+DkAihdACel/IBiN7oiZ1SlgSMyTkpHvzLUSA4PTR07dkRGRgYMBgO++uorJCQkYNeuXY5u1g3NmTMHiYmJ0mOj0YiwsDAHtug25lz5x8e9zc3VIwRQUVZPwLruFGNtpx2rhzGIygn4BtuNqDVycqkc+aj8Wev9qtBx/f2qUOJcx/3KstL9Wuqv8zmcK+uS077r71fWxVFgamIOD00qlQodOnQAAPTu3Rv79u3D0qVL8fjjj8NsNqOgoMButCknJwdBQUEAgKCgIOzdu9euvpycHGlf1c+qbdXLaLVauLq6QqlUQqlU1lqmqo7aqNVqqNXqxnWaHEOhAFxcbTePm5zgb7Vem4Bf51yvaiNd5mJbwAJgC1ui2s/rtgH17Ect+290jCPqFNLDFl+n9BLc6Bhc+1Cu64O6rvBQIwDUFwYaElBkhJK6jnVyZrAgaiCHh6brWa1WmEwm9O7dGy4uLkhNTcXIkSMBAJmZmcjOzoZOpwMA6HQ6/OUvf0Fubq50lVtKSgq0Wi2ioqKkMt9//73dc6SkpEh1qFQq9O7dG6mpqRgxYoTUhtTUVEyZMuVWdJlaIyenylNxHo5uCRER3SIODU1z5szB0KFDER4ejsLCQnzxxRfYuXMntmzZAi8vL4wbNw6JiYnw9fWFVqvF1KlTodPpcM899wAAhgwZgqioKDz11FNYvHgx9Ho9XnvtNUyePFkaBZo4cSKWLVuGWbNmYezYsdi+fTu+/PJLJCcnS+1ITExEQkIC+vTpg759+2LJkiUoLi7GM88845DXhYiIiFog4UBjx44VERERQqVSCX9/fzF48GCxdetWaX9paal4/vnnhY+Pj3BzcxOPPPKIuHTpkl0dZ86cEUOHDhWurq7Cz89PzJgxQ5SXl9uV2bFjh4iOjhYqlUq0a9dOrFq1qkZbPvjgAxEeHi5UKpXo27ev2LNnT4P6YjAYBABhMBgadBwRERE5TkM+v1vcOk2t1W2xThMREdHvTEM+v7kwAxEREZEMDE1EREREMjA0EREREcnA0EREREQkA0MTERERkQwMTUREREQyMDQRERERycDQRERERCQDQxMRERGRDAxNRERERDIwNBERERHJ4OzoBtwuqr7Cz2g0OrglREREJFfV57acr+JlaGoihYWFAICwsDAHt4SIiIgaqrCwEF5eXvWWUQg50YpuyGq14uLFi/D09IRCoWjSuo1GI8LCwnDu3LkbfgNza8T+tX63ex9v9/4Bt38f2b/Wr7n6KIRAYWEhQkJC4ORU/6wljjQ1EScnJ4SGhjbrc2i12tv2lwFg/24Ht3sfb/f+Abd/H9m/1q85+nijEaYqnAhOREREJANDExEREZEMDE2tgFqtxrx586BWqx3dlGbB/rV+t3sfb/f+Abd/H9m/1q8l9JETwYmIiIhk4EgTERERkQwMTUREREQyMDQRERERycDQRERERCQDQ1MLsXz5ckRGRkKj0SAmJgZ79+6tt/z69evRqVMnaDQadOvWDd9///0tamnjNKR/q1evhkKhsLtpNJpb2NqG+eGHHzB8+HCEhIRAoVBg48aNNzxm586d6NWrF9RqNTp06IDVq1c3ezsbq6H927lzZ433T6FQQK/X35oGN1BSUhLuvvtueHp6IiAgACNGjEBmZuYNj2tNv4ON6WNr+j1csWIFunfvLi16qNPpsGnTpnqPaU3vX0P715reu9osWrQICoUC06ZNq7ecI95DhqYWYN26dUhMTMS8efNw4MAB9OjRA3FxccjNza21/E8//YRRo0Zh3LhxOHjwIEaMGIERI0bg8OHDt7jl8jS0f4BtxddLly5Jt7Nnz97CFjdMcXExevTogeXLl8sqn5WVhfj4eAwaNAgZGRmYNm0axo8fjy1btjRzSxunof2rkpmZafceBgQENFMLb86uXbswefJk7NmzBykpKSgvL8eQIUNQXFxc5zGt7XewMX0EWs/vYWhoKBYtWoT09HTs378ff/zjH/Hwww/jyJEjtZZvbe9fQ/sHtJ737nr79u3DRx99hO7du9dbzmHvoSCH69u3r5g8ebL02GKxiJCQEJGUlFRr+ccee0zEx8fbbYuJiRHPPfdcs7azsRrav1WrVgkvL69b1LqmBUBs2LCh3jKzZs0SXbp0sdv2+OOPi7i4uGZsWdOQ078dO3YIACI/P/+WtKmp5ebmCgBi165ddZZpbb+D15PTx9b8eyiEED4+PuKTTz6pdV9rf/+EqL9/rfW9KywsFHfeeadISUkRAwYMEC+++GKdZR31HnKkycHMZjPS09MRGxsrbXNyckJsbCzS0tJqPSYtLc2uPADExcXVWd6RGtM/ACgqKkJERATCwsJu+D+q1qY1vX83Izo6GsHBwbj//vuxe/duRzdHNoPBAADw9fWts0xrfw/l9BFonb+HFosFa9euRXFxMXQ6Xa1lWvP7J6d/QOt87yZPnoz4+Pga701tHPUeMjQ52OXLl2GxWBAYGGi3PTAwsM45IHq9vkHlHakx/evYsSM+/fRTfPvtt/j8889htVrRr18/nD9//lY0udnV9f4ZjUaUlpY6qFVNJzg4GCtXrsTXX3+Nr7/+GmFhYRg4cCAOHDjg6KbdkNVqxbRp09C/f3907dq1znKt6XfwenL72Np+Dw8dOgQPDw+o1WpMnDgRGzZsQFRUVK1lW+P715D+tbb3DgDWrl2LAwcOICkpSVZ5R72Hzs1aO1Ej6HQ6u/9B9evXD507d8ZHH32EN954w4EtIzk6duyIjh07So/79euHU6dO4b333sO//vUvB7bsxiZPnozDhw/jxx9/dHRTmo3cPra238OOHTsiIyMDBoMBX331FRISErBr1646g0Vr05D+tbb37ty5c3jxxReRkpLS4iesMzQ5mJ+fH5RKJXJycuy25+TkICgoqNZjgoKCGlTekRrTv+u5uLigZ8+eOHnyZHM08Zar6/3TarVwdXV1UKuaV9++fVt8EJkyZQq+++47/PDDDwgNDa23bGv6HayuIX28Xkv/PVSpVOjQoQMAoHfv3ti3bx+WLl2Kjz76qEbZ1vj+NaR/12vp7116ejpyc3PRq1cvaZvFYsEPP/yAZcuWwWQyQalU2h3jqPeQp+ccTKVSoXfv3khNTZW2Wa1WpKam1nm+WqfT2ZUHgJSUlHrPbztKY/p3PYvFgkOHDiE4OLi5mnlLtab3r6lkZGS02PdPCIEpU6Zgw4YN2L59O9q2bXvDY1rbe9iYPl6vtf0eWq1WmEymWve1tvevNvX173ot/b0bPHgwDh06hIyMDOnWp08fjB49GhkZGTUCE+DA97BZp5mTLGvXrhVqtVqsXr1aHD16VEyYMEF4e3sLvV4vhBDiqaeeEi+//LJUfvfu3cLZ2Vn89a9/FceOHRPz5s0TLi4u4tChQ47qQr0a2r8FCxaILVu2iFOnTon09HTxxBNPCI1GI44cOeKoLtSrsLBQHDx4UBw8eFAAEH/729/EwYMHxdmzZ4UQQrz88sviqaeeksqfPn1auLm5iZkzZ4pjx46J5cuXC6VSKTZv3uyoLtSrof177733xMaNG8WJEyfEoUOHxIsvviicnJzEtm3bHNWFek2aNEl4eXmJnTt3ikuXLkm3kpISqUxr/x1sTB9b0+/hyy+/LHbt2iWysrLEr7/+Kl5++WWhUCjE1q1bhRCt//1raP9a03tXl+uvnmsp7yFDUwvxwQcfiPDwcKFSqUTfvn3Fnj17pH0DBgwQCQkJduW//PJLcddddwmVSiW6dOkikpOTb3GLG6Yh/Zs2bZpUNjAwUAwbNkwcOHDAAa2Wp+oS++tvVX1KSEgQAwYMqHFMdHS0UKlUol27dmLVqlW3vN1yNbR/b7/9tmjfvr3QaDTC19dXDBw4UGzfvt0xjZehtr4BsHtPWvvvYGP62Jp+D8eOHSsiIiKESqUS/v7+YvDgwVKgEKL1v38N7V9reu/qcn1oainvoUIIIZp3LIuIiIio9eOcJiIiIiIZGJqIiIiIZGBoIiIiIpKBoYmIiIhIBoYmIiIiIhkYmoiIiIhkYGgiIiIikoGhiYiomezcuRMKhQIFBQWObgoRNQGGJiIiIiIZGJqIiIiIZGBoIqLbltVqRVJSEtq2bQtXV1f06NEDX331FYBrp86Sk5PRvXt3aDQa3HPPPTh8+LBdHV9//TW6dOkCtVqNyMhIvPvuu3b7TSYTZs+ejbCwMKjVanTo0AH/+Mc/7Mqkp6ejT58+cHNzQ79+/ZCZmdm8HSeiZsHQRES3raSkJPzzn//EypUrceTIEUyfPh1//vOfsWvXLqnMzJkz8e6772Lfvn3w9/fH8OHDUV5eDsAWdh577DE88cQTOHToEObPn4/XX38dq1evlo5/+umn8e9//xvvv/8+jh07ho8++ggeHh527Xj11Vfx7rvvYv/+/XB2dsbYsWNvSf+JqGnxC3uJ6LZkMpng6+uLbdu2QafTSdvHjx+PkpISTJgwAYMGDcLatWvx+OOPAwCuXr2K0NBQrF69Go899hhGjx6NvLw8bN26VTp+1qxZSE5OxpEjR/Dbb7+hY8eOSElJQWxsbI027Ny5E4MGDcK2bdswePBgAMD333+P+Ph4lJaWQqPRNPOrQERNiSNNRHRbOnnyJEpKSnD//ffDw8NDuv3zn//EqVOnpHLVA5Wvry86duyIY8eOAQCOHTuG/v3729Xbv39/nDhxAhaLBRkZGVAqlRgwYEC9benevbt0Pzg4GACQm5t7030kolvL2dENICJqDkVFRQCA5ORk3HHHHXb71Gq1XXBqLFdXV1nlXFxcpPsKhQKAbb4VEbUuHGkiottSVFQU1Go1srOz0aFDB7tbWFiYVG7Pnj3S/fz8fPz222/o3LkzAKBz587YvXu3Xb27d+/GXXfdBaVSiW7dusFqtdrNkSKi2xdHmojotuTp6YmXXnoJ06dPh9Vqxb333guDwYDdu3dDq9UiIiICALBw4UK0adMGgYGBePXVV+Hn54cRI0YAAGbMmIG7774bb7zxBh5//HGkpaVh2bJl+PDDDwEAkZGRSEhIwNixY/H++++jR48eOHv2LHJzc/HYY485qutE1EwYmojotvXGG2/A398fSUlJOH36NLy9vdGrVy+88sor0umxRYsW4cUXX8SJEycQHR2N//73v1CpVACAXr164csvv8TcuXPxxhtvIDg4GAsXLsSYMWOk51ixYgVeeeUVPP/887hy5QrCw8PxyiuvOKK7RNTMePUcEf0uVV3Zlp+fD29vb0c3h4haAc5pIiIiIpKBoYmIiIhIBp6eIyIiIpKBI01EREREMjA0EREREcnA0EREREQkA0MTERERkQwMTUREREQyMDQRERERycDQRERERCQDQxMRERGRDAxNRERERDL8Py41TnH/4/mkAAAAAElFTkSuQmCC\n"
1077 | },
1078 | "metadata": {}
1079 | }
1080 | ]
1081 | },
1082 | {
1083 | "cell_type": "code",
1084 | "metadata": {
1085 | "colab": {
1086 | "base_uri": "https://localhost:8080/",
1087 | "height": 472
1088 | },
1089 | "id": "Fs0nwFpAL7YI",
1090 | "outputId": "c8b3165d-f6cd-4e87-d19d-e89b30a62445"
1091 | },
1092 | "source": [
1093 | "# Plot changes in model accuracy during training\n",
1094 | "plt.plot(history.history[\"factorized_top_k/top_100_categorical_accuracy\"])\n",
1095 | "plt.plot(history.history[\"val_factorized_top_k/top_100_categorical_accuracy\"])\n",
1096 | "plt.title(\"Model accuracies during training\")\n",
1097 | "plt.xlabel(\"epoch\")\n",
1098 | "plt.ylabel(\"accuracy\")\n",
1099 | "plt.legend([\"train\", \"test\"], loc=\"upper right\")\n",
1100 | "plt.show()"
1101 | ],
1102 | "execution_count": 18,
1103 | "outputs": [
1104 | {
1105 | "output_type": "display_data",
1106 | "data": {
1107 | "text/plain": [
1108 | ""
1109 | ],
1110 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAkAAAAHHCAYAAABXx+fLAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAAB50UlEQVR4nO3dd3hUZdrH8e9kkkkhhRISSIGELh0CREAFNIKIKAqK6Api21VQMTawgFgIdlRcyxZZX9cFRcUOQmiCSEfpNbRAEkJJJW3mvH8MmTAQSkKSySS/z3WdSzjnOWfuZyZhbp9qMgzDQERERKQW8XB1ACIiIiJVTQmQiIiI1DpKgERERKTWUQIkIiIitY4SIBEREal1lACJiIhIraMESERERGodJUAiIiJS6ygBEhERkVpHCZBIJTCZTLzwwgtlvm/v3r2YTCZmzJhR4THVNIsXL8ZkMrF48WJXh8ILL7yAyWRym+e62qX+nJf390vkdEqApMaaMWMGJpMJk8nEsmXLzrpuGAaRkZGYTCZuuOEGF0QoUn19/vnnTJs2zdVhiFQaT1cHIFLZfHx8+Pzzz7niiiuczi9ZsoSDBw/i7e3tosjkUlx11VWcPHkSi8Xi6lAqzXPPPcf48eNd8tqff/45mzZtYty4cRX+7KZNm3Ly5Em8vLzKdf/Jkyfx9NTXl1watQBJjXf99dfz5ZdfUlRU5HT+888/JyYmhkaNGrkostojJyenwp/p4eGBj48PHh4175+x4vfL09MTHx8fF0dzYXl5edhstosubzKZ8PHxwWw2l+v1fHx8lADJJat5/3KInGHEiBEcPXqU+fPnO84VFBQwe/Zs7rjjjlLvycnJ4fHHHycyMhJvb29at27NG2+8gWEYTuXy8/N57LHHaNiwIQEBAdx4440cPHiw1GcmJydzzz33EBoaire3N+3atePf//53uep07NgxnnjiCTp06IC/vz+BgYEMHDiQP/7446yyeXl5vPDCC7Rq1QofHx8aN27MLbfcwu7dux1lbDYb77zzDh06dMDHx4eGDRty3XXXsWbNGuD8YzbOHI9RPG5ly5Yt3HHHHdSrV8/R+vbnn39y991306xZM3x8fGjUqBH33HMPR48eLfX9uvfeewkLC8Pb25vo6GgefPBBCgoKgHOPAVq5ciXXXXcdQUFB+Pn50adPH5YvX+5UJisri3HjxhEVFYW3tzchISFce+21rFu37oLv/bJly+jevTs+Pj40b96cjz766KwyFfV+lTYGyGQyMXbsWObMmUP79u0dP0tz584967UWL15Mt27dnGK9mHFFffv25ccff2Tfvn2ObuSoqCjHM00mEzNnzuS5554jPDwcPz8/MjMzL/rnsrT35+6778bf35/k5GSGDBmCv78/DRs25IknnsBqtV7Ue7hr1y7uvvtu6tatS1BQEKNHjyY3N9fp3pMnT/LII48QHBzs+J1NTk7WuKJaSCm01HhRUVH07NmT//3vfwwcOBCAn3/+mYyMDG6//Xbeffddp/KGYXDjjTeyaNEi7r33Xjp37sy8efN48sknSU5O5u2333aUve+++/jss8+444476NWrFwsXLmTQoEFnxZCamsrll1/u+PJq2LAhP//8M/feey+ZmZll7mbYs2cPc+bM4dZbbyU6OprU1FQ++ugj+vTpw5YtWwgLCwPAarVyww03kJiYyO23386jjz5KVlYW8+fPZ9OmTTRv3hyAe++9lxkzZjBw4EDuu+8+ioqK+PXXX/n999/p1q1bmWIrduutt9KyZUumTJniSBznz5/Pnj17GD16NI0aNWLz5s18/PHHbN68md9//93xxXzo0CF69OjBiRMneOCBB2jTpg3JycnMnj2b3Nzcc3Z7LVy4kIEDBxITE8OkSZPw8PDgk08+4eqrr+bXX3+lR48eAPztb39j9uzZjB07lrZt23L06FGWLVvG1q1b6dq16znrtHHjRvr370/Dhg154YUXKCoqYtKkSYSGhpbrPbrQ+3Uuy5Yt4+uvv+ahhx4iICCAd999l6FDh7J//34aNGgAwPr167nuuuto3LgxkydPxmq18uKLL9KwYcMLxvLss8+SkZHBwYMHHT/v/v7+TmVeeuklLBYLTzzxBPn5+VgsFrZs2XJRP5fnYrVaGTBgALGxsbzxxhssWLCAN998k+bNm/Pggw9eMO7bbruN6OhoEhISWLduHf/85z8JCQnh1VdfdZS5++67+eKLL7jrrru4/PLLWbJkSam/s1ILGCI11CeffGIAxurVq43p06cbAQEBRm5urmEYhnHrrbca/fr1MwzDMJo2bWoMGjTIcd+cOXMMwHj55Zednjds2DDDZDIZu3btMgzDMDZs2GAAxkMPPeRU7o477jAAY9KkSY5z9957r9G4cWMjPT3dqeztt99uBAUFOeJKSkoyAOOTTz45b93y8vIMq9XqdC4pKcnw9vY2XnzxRce5f//73wZgvPXWW2c9w2azGYZhGAsXLjQA45FHHjlnmfPFdWZdJ02aZADGiBEjzipbXM/T/e9//zMAY+nSpY5zI0eONDw8PIzVq1efM6ZFixYZgLFo0SLH+ZYtWxoDBgxwlCl+zejoaOPaa691nAsKCjLGjBlz1rMvZMiQIYaPj4+xb98+x7ktW7YYZrPZOP2f04p6v4qvnXm/xWJx/BwahmH88ccfBmC89957jnODBw82/Pz8jOTkZMe5nTt3Gp6enmc9szSDBg0ymjZtetb54ve9WbNmZ32eF/tzWdr7M2rUKANwKmcYhtGlSxcjJibmrPegtPfwnnvucSp38803Gw0aNHD8fe3atQZgjBs3zqnc3XfffdYzpeZTF5jUCrfddhsnT57khx9+ICsrix9++OGc3V8//fQTZrOZRx55xOn8448/jmEY/Pzzz45ywFnlzmzNMQyDr776isGDB2MYBunp6Y5jwIABZGRkXFTXy+m8vb0dY1+sVitHjx7F39+f1q1bOz3rq6++Ijg4mIcffvisZxS3tnz11VeYTCYmTZp0zjLl8be//e2sc76+vo4/5+XlkZ6ezuWXXw7giNtmszFnzhwGDx5cauvTuWLasGEDO3fu5I477uDo0aOO9zgnJ4drrrmGpUuXOsap1K1bl5UrV3Lo0KGLro/VamXevHkMGTKEJk2aOM5fdtllDBgw4KKfcy6lvV/nEhcX52i9A+jYsSOBgYHs2bPHEeuCBQsYMmSIU6tLixYtHK2gl2rUqFFOnydc/M/l+Zz5Plx55ZWOepXn3qNHj5KZmQng6CZ86KGHnMqV9vshNZ+6wKRWaNiwIXFxcXz++efk5uZitVoZNmxYqWX37dtHWFgYAQEBTucvu+wyx/Xi/3p4eDh9EQG0bt3a6e9HjhzhxIkTfPzxx3z88celvmZaWlqZ6lM8Zufvf/87SUlJTmMkirtAAHbv3k3r1q3PO2B09+7dhIWFUb9+/TLFcCHR0dFnnTt27BiTJ09m5syZZ9U5IyMDsL9fmZmZtG/fvkyvt3PnTsD+xXwuGRkZ1KtXj9dee41Ro0YRGRlJTEwM119/PSNHjqRZs2bnvPfIkSOcPHmSli1bnnWtdevWjoS4vEp7v87l9ASsWL169Th+/Dhg/3k6efIkLVq0OKtcaefKo7R4L/bn8lyKx5+d7vR6XciZ70u9evUAOH78OIGBgY7f2TNjr6j3RNyLEiCpNe644w7uv/9+UlJSGDhwIHXr1q2S1y1udfjLX/5yzi/njh07lumZU6ZM4fnnn+eee+7hpZdeon79+nh4eDBu3Lgyzca5WOdqdTlzcOrpzmwdAHtL3G+//caTTz5J586d8ff3x2azcd11111y3MX3v/7663Tu3LnUMsXjWG677TauvPJKvvnmG3755Rdef/11Xn31Vb7++usKaSGpqPfrXM41e8q4wNihilRavJf6c1neWWEXur8q3xdxH0qApNa4+eab+etf/8rvv//OrFmzzlmuadOmLFiwgKysLKdWoG3btjmuF//XZrM5WlmKbd++3el5xTPErFYrcXFxFVKX2bNn069fP/71r385nT9x4gTBwcGOvzdv3pyVK1dSWFh4zjVXmjdvzrx58zh27Ng5W4GK/0/6xIkTTueLW8MuxvHjx0lMTGTy5MlMnDjRcb645aZYw4YNCQwMZNOmTRf9bMDREhcYGHhR73Pjxo156KGHeOihh0hLS6Nr16688sor50yAGjZsiK+v71nxwtmfeUW8X5ciJCQEHx8fdu3adda10s6Vpjzdnxf7c+kqxb+zSUlJTi15F/ueSM2iMUBSa/j7+/PBBx/wwgsvMHjw4HOWu/7667FarUyfPt3p/Ntvv43JZHJ8QRb/98xZZGeunms2mxk6dChfffVVqV/qR44cKXNdzGbzWf9X++WXX5KcnOx0bujQoaSnp59VFyj5v+KhQ4diGAaTJ08+Z5nAwECCg4NZunSp0/W///3vZYr59GcWO/P98vDwYMiQIXz//feOafilxXSmmJgYmjdvzhtvvEF2dvZZ14vfZ6vV6uhuKxYSEkJYWBj5+fnnjX/AgAHMmTOH/fv3O85v3bqVefPmOZWtiPfrUpjNZuLi4pgzZ47TOKddu3Y5xrBdSJ06dc56ny7mdS/m59JVisdqnfk5vPfee64IR1xMLUBSq5xvfEixwYMH069fP5599ln27t1Lp06d+OWXX/j2228ZN26co6Whc+fOjBgxgr///e9kZGTQq1cvEhMTS/2/yalTp7Jo0SJiY2O5//77adu2LceOHWPdunUsWLCAY8eOlakeN9xwAy+++CKjR4+mV69ebNy4kf/+979njWEZOXIkn376KfHx8axatYorr7ySnJwcFixYwEMPPcRNN91Ev379uOuuu3j33XfZuXOnozvq119/pV+/fowdOxawT/mfOnUq9913H926dWPp0qXs2LHjomMODAzkqquu4rXXXqOwsJDw8HB++eUXkpKSzio7ZcoUfvnlF/r06cMDDzzAZZddxuHDh/nyyy9ZtmxZqd2XHh4e/POf/2TgwIG0a9eO0aNHEx4eTnJyMosWLSIwMJDvv/+erKwsIiIiGDZsGJ06dcLf358FCxawevVq3nzzzfPWYfLkycydO5crr7yShx56iKKiIt577z3atWvHn3/+6VT2Ut+vS/XCCy/wyy+/0Lt3bx588EFHUt++fXs2bNhwwftjYmKYNWsW8fHxdO/eHX9///P+jwNc/M+lq8TExDB06FCmTZvG0aNHHdPgiz+XmrjvmpyHayafiVS+06fBn8+Z0+ANwzCysrKMxx57zAgLCzO8vLyMli1bGq+//rrT9GrDMIyTJ08ajzzyiNGgQQOjTp06xuDBg40DBw6UOqU2NTXVGDNmjBEZGWl4eXkZjRo1Mq655hrj448/dpQpyzT4xx9/3GjcuLHh6+tr9O7d21ixYoXRp08fo0+fPk5lc3NzjWeffdaIjo52vO6wYcOM3bt3O8oUFRUZr7/+utGmTRvDYrEYDRs2NAYOHGisXbvW6Tn33nuvERQUZAQEBBi33XabkZaWds4pyUeOHDkr7oMHDxo333yzUbduXSMoKMi49dZbjUOHDpX6fu3bt88YOXKk0bBhQ8Pb29to1qyZMWbMGCM/P98wjLOnwRdbv369ccsttxgNGjQwvL29jaZNmxq33XabkZiYaBiGYeTn5xtPPvmk0alTJyMgIMCoU6eO0alTJ+Pvf//7ed/zYkuWLDFiYmIMi8ViNGvWzPjwww9Lna5eEe/XuabBlzaFv2nTpsaoUaOcziUmJhpdunQxLBaL0bx5c+Of//yn8fjjjxs+Pj4XrGd2drZxxx13GHXr1jUAx5T44vf9yy+/POuei/25PNc0+Dp16lz0e3Ax72HxvwFJSUmOczk5OcaYMWOM+vXrG/7+/saQIUOM7du3G4AxderUC74vUnOYDEOjw0REaoshQ4awefPmUscy1VYbNmygS5cufPbZZ9x5552uDkeqiMYAiYjUUCdPnnT6+86dO/npp5/o27evawKqBs58T8A+Ds3Dw4OrrrrKBRGJq2gMkIhIDdWsWTPH3mv79u3jgw8+wGKx8NRTT7k6NJd57bXXWLt2Lf369cPT05Off/6Zn3/+mQceeIDIyEhXhydVSF1gIiI11OjRo1m0aBEpKSl4e3vTs2dPpkyZct79zmq6+fPnM3nyZLZs2UJ2djZNmjThrrvu4tlnn9UO87WMEiARERGpdTQGSERERGodJUAiIiJS66jDsxQ2m41Dhw4REBCghbFERETchGEYZGVlERYWhofH+dt4lACV4tChQ5oNICIi4qYOHDhARETEecsoASpF8QaYBw4cIDAw0MXRiIiIyMXIzMwkMjLSaSPrc1ECVIribq/AwEAlQCIiIm7mYoavaBC0iIiI1DpKgERERKTWUQIkIiIitY7GAImIiFQhm81GQUGBq8NwS15eXpjN5gp5lhIgERGRKlJQUEBSUhI2m83VobitunXr0qhRo0tep08JkIiISBUwDIPDhw9jNpuJjIy84EJ94swwDHJzc0lLSwOgcePGl/Q8JUAiIiJVoKioiNzcXMLCwvDz83N1OG7J19cXgLS0NEJCQi6pO0zpp4iISBWwWq0AWCwWF0fi3oqTx8LCwkt6TrVIgN5//32ioqLw8fEhNjaWVatWnbPsP/7xD6688krq1atHvXr1iIuLcypfWFjI008/TYcOHahTpw5hYWGMHDmSQ4cOVUVVREREzkt7TF6ainr/XJ4AzZo1i/j4eCZNmsS6devo1KkTAwYMcPTxnWnx4sWMGDGCRYsWsWLFCiIjI+nfvz/JyckA5Obmsm7dOp5//nnWrVvH119/zfbt27nxxhursloiIiJSjZkMwzBcGUBsbCzdu3dn+vTpgH16YGRkJA8//DDjx4+/4P1Wq5V69eoxffp0Ro4cWWqZ1atX06NHD/bt20eTJk0u+MzMzEyCgoLIyMjQVhgiIlIh8vLySEpKIjo6Gh8fH1eH4zJRUVGMGzeOcePGlev+872PZfn+dmkLUEFBAWvXriUuLs5xzsPDg7i4OFasWHFRz8jNzaWwsJD69eufs0xGRgYmk4m6deteasgiIiK1Tt++fcudsJxp9erVPPDAAxXyrEvh0llg6enpWK1WQkNDnc6Hhoaybdu2i3rG008/TVhYmFMSdbq8vDyefvppRowYcc5sMD8/n/z8fMffMzMzL7IGIiIiYhgGVqsVT88LpxUNGzbkZKGVwiIbXp6ua4dx+RigSzF16lRmzpzJN998U2pzYmFhIbfddhuGYfDBBx+c8zkJCQkEBQU5jsjIyMoMW0RExG3cfffdLFmyhHfeeQeTyYTJZGLGjBmYTCZ+/vlnYmJi8Pb2ZtmyZezevZubbrqJ0NBQ/P396d69OwsWLADsSVLGyUIimjRl0iuvcSTb3vBgMpn45z//yc0334yfnx8tW7bku+++q/R6uTQBCg4Oxmw2k5qa6nQ+NTWVRo0anffeN954g6lTp/LLL7/QsWPHs64XJz/79u1j/vz55+0LnDBhAhkZGY7jwIED5auQiIjIRTIMg9yCIpccZRn++84779CzZ0/uv/9+Dh8+zOHDhx0NBePHj2fq1Kls3bqVjh07kp2dzfXXX09iYiLr16/nuuuuY/DgwazfspPtqVnsO5rjeG2brSSGyZMnc9ttt/Hnn39y/fXXc+edd3Ls2LGKfcPP4NIuMIvFQkxMDImJiQwZMgSwD4JOTExk7Nix57zvtdde45VXXmHevHl069btrOvFyc/OnTtZtGgRDRo0OG8c3t7eeHt7X1JdREREyuJkoZW2E+e55LW3vDgAP8vFpQBBQUFYLBb8/PwcjRPFw1RefPFFrr32WkfZ+vXr06lTJwDyC6387fFnmPnlV3zx9TeMuPsBzB4mPEwmQgJ8iKhfshjk3XffzYgRIwCYMmUK7777LqtWreK6666rkPqWxuUrQcfHxzNq1Ci6detGjx49mDZtGjk5OYwePRqAkSNHEh4eTkJCAgCvvvoqEydO5PPPPycqKoqUlBQA/P398ff3p7CwkGHDhrFu3Tp++OEHrFaro0z9+vW1AJWIiEgFObMRIisri2efn8RPP/1IWmoKRUVW8vNOcuTwIcLr+lLXz4LZw4Sn2Xktn9N7curUqUNgYOA5l8OpKC5PgIYPH86RI0eYOHEiKSkpdO7cmblz5zoGRu/fv99pv5QPPviAgoIChg0b5vScSZMm8cILL5CcnOzoO+zcubNTmUWLFtG3b99KrY+IiMjF8PUys+XFAS577YpQp04dAKw2gxO5BYwZM47lSxYS/9xLNImKpkFQAI/cfxd1PA0a+J+7p8XLy8vp7yaTqdI3jHV5AgQwduzYc3Z5LV682Onve/fuPe+zoqKiytS3KSIi4gomk+miu6FczWKxOLbyOF1BkZXDJ05yLLcAq81g7aoVDLntDobfOpQGdSwU5p9k/7591XL1a7eeBSYiIiKVLyoqipUrV7J3716OHDlCTp59H66dqdkcyc7HajOweHrQulUrli34ibS929m2ZRN33HFHpbfklJcSIBERETmvJ554ArPZTNu2bQkJCWHdlp0AGIC/tydRDerQOjSA6e9Oo169evTq1YvBgwczYMAAunbt6trgz8HlW2FUR9oKQ0REKpq7boVRaLVxNLuAYzkFFJ1qzfEwmajr50Wwvzc+FTSe6GJV1FYY7tH5KCIiIlUqJ7+Io9kFZJwsxMDeVuJl9qCBv4X6fhY8ze7diaQESERERACwGQaZJwtJzy4gt6DIcb6OxZNgfwuBvl7VckBzeSgBEhERqeUKrTaO5di7uQqt9m4uk8lEXV8vgv0t+LrJbLWyqHk1EhERkYtysqCI9OwCTpwsdCwh42n2oEEdC/XrWPBy826u81ECJCIiUosYhkFmnr2bKye/pJvLz2Kmgb83Qb5eeNSQbq7zUQIkIiJSCxRZbRzPLeBodgEFxd1cmAjy9aSBvzd+FnONGd9zMZQAiYiI1GB5hVaOZudzPLcQW3E3l4eJ+nUs1K/jjcWz5nZznY8SIBERkRrGMAyy8opIz84n+7RuLh8vM8H+Fur6WvDwqD2tPaVRAiQiIlJDWG2Go5srv6hk765AH/tsrjrenrWqm+t8lACJiIi4ufwiK0ezCzieU4D1VDeX2cNEPT8Lwf4WLJ5Vu1qzO1ACJCIi4oYMwyAn3z6NPfPU5qQA3p5mGvhbqOdnwVxB3Vx9+/alc+fOTJs2rUKed/fdd3PixAnmzJlTIc8rDyVAIiIibsRmMzh+0t7NlVdY0s0VcKqby1/dXBeldg79FhERcTMFRTYOZ5xka0omycdPkldoxcNkokEdb1qFBhAdXIcAn4rfquLuu+9myZIlvPPOO5hMJkwmE3v37mXTpk0MHDgQf39/QkNDueuuu0hPT3fcN3v2bDp06ICvry8NGjQgLi6OnJwcXnjhBf7zn//w7bffOp63ePHiCo35YqgFSERExBUMAwpzL1DEILfAytGcAjJPFjk2JfU2e1Df30I9Py88PWxg5EFBGV7byw8uMlF655132LFjB+3bt+fFF1+03+7lRY8ePbjvvvt4++23OXnyJE8//TS33XYbCxcu5PDhw4wYMYLXXnuNm2++maysLH799VcMw+CJJ55g69atZGZm8sknnwBQv379MgRfMZQAiYiIuEJhLkwJO28RE1Dn1FGhnjkElot7alBQEBaLBT8/Pxo1agTAyy+/TJcuXZgyZYqj3L///W8iIyPZsWMH2dnZFBUVccstt9C0aVMAOnTo4Cjr6+tLfn6+43muoARIREREyuSPP/5g0aJF+Pv7n3Vt9+7d9O/fn2uuuYYOHTowYMAA+vfvz7Bhw6hXr54Loi2dEiARERFX8PKzt8QAuQVFHMspdNqU1MvsQf06XtT3s+BZ0ZuSevld0u3Z2dkMHjyYV1999axrjRs3xmw2M3/+fH777Td++eUX3nvvPZ599llWrlxJdHT0Jb12RVECJCIi4gIGkFHkRXp2AbkFBuAJnp74WTwJ9rcQWI02JbVYLFitJTPOunbtyldffUVUVBSenqWnEiaTid69e9O7d28mTpxI06ZN+eabb4iPjz/rea6gWWAiIiJVqMhqIy0rj20pWew/lktuQREmk4m6fhZahPjTIsSfun6WapP8AERFRbFy5Ur27t1Leno6Y8aM4dixY4wYMYLVq1eze/du5s2bx+jRo7FaraxcuZIpU6awZs0a9u/fz9dff82RI0e47LLLHM/7888/2b59O+np6RQWFl4ggoqnBEhERKQKJB3J4XhuAXvSc0jJyKPQasPTw4OQAB/aNAqgSX0//CzVs2PmiSeewGw207ZtWxo2bEhBQQHLly/HarXSv39/OnTowLhx46hbty4eHh4EBgaydOlSrr/+elq1asVzzz3Hm2++ycCBAwG4//77ad26Nd26daNhw4YsX768yutkMoo7G8UhMzOToKAgMjIyCAwMdHU4IiLipqw2g4Xb0vhkeRL70k7wQr8QQsIi8PP1pYG/N3V9vWr9pqRllZeXR1JSEtHR0fj4+DhdK8v3d/VMNUVERNxYZl4hX6w+wKcr9rH/mH2tn4hAM75eZiLr+1EvoI5Wa3YxJUAiIiIVZM+RbP7z215mrz1IToF9kG+Qrxe394hkRNdGnDyWgp9FW1VUB0qARERELoFhGPy6M51PliexaPsRx/mWIf7c3TuKm7uE42fxtHfdHHNhoOJECZCIiEg55BYU8dW6ZGYsT2L3kRzH+avbhDC6dxRXtAhWS081pgRIRESkDA4cy+X/ft/HzFX7ycwrAsDf25NhMRGM6hVFdPD5t5jQ3KNLU1HvnxIgERGRCzAMg5VJx/hkeRLzt6RiO/Ud3LSBH6N6RnFrtwgCfLzO+wyz2QxAQUEBvr6+lR1yjZWbax9U7uV1/vf7QpQAiYiInENeoZXv/jjEJ8v3svVwpuP8FS2CGd07ir6tQzBf5DR2T09P/Pz8OHLkCF5eXnh4aCm+sjAMg9zcXNLS0qhbt64joSyvapEAvf/++7z++uukpKTQqVMn3nvvPXr06FFq2X/84x98+umnbNq0CYCYmBimTJniVN4wDCZNmsQ//vEPTpw4Qe/evfnggw9o2bJlldRHRETcW2pmHp/9vo/PV+7naE4BAD5eHtzcJYLRvaNoFRpQ5meaTCYaN25MUlIS+/btq+iQa426detWyC7yLk+AZs2aRXx8PB9++CGxsbFMmzaNAQMGsH37dkJCQs4qv3jxYkaMGEGvXr3w8fHh1VdfpX///mzevJnw8HAAXnvtNd59913+85//EB0dzfPPP8+AAQPYsmXLWYsmiYiIFFu//zifLN/LTxsPU3SqnyssyIeRvaK4vXskdf0sl/R8i8VCy5YtKSgoqIhwax0vL69Lbvkp5vKVoGNjY+nevTvTp08HwGazERkZycMPP8z48eMveL/VaqVevXpMnz6dkSNHYhgGYWFhPP744zzxxBMAZGRkEBoayowZM7j99tsv+EytBC0iUnsUFNn4edNhPlm+lw0HTjjOd4+qx+je0fRvG1rxu7FLpXCblaALCgpYu3YtEyZMcJzz8PAgLi6OFStWXNQzcnNzKSwspH79+gAkJSWRkpJCXFyco0xQUBCxsbGsWLGi1AQoPz+f/Px8x98zMzPPKiMiIjXL0ex8Pl+5n//7fR9pWfbvAIvZgxs6Neae3tG0Dw9ycYRSmVyaAKWnp2O1WgkNDXU6HxoayrZt2y7qGU8//TRhYWGOhCclJcXxjDOfWXztTAkJCUyePLms4YuIiBvaciiTT5Yn8e0fhygosgHQMMCbv8Q25Y7YJjQM8HZxhFIVXD4G6FJMnTqVmTNnsnjx4ksa2zNhwgTi4+Mdf8/MzCQyMrIiQhQRkWrAajOYvyWFT5bvZeVpyzF3jAhidO8oBnUIw+Kpbq7axKUJUHBwMGazmdTUVKfzqampFxzh/cYbbzB16lQWLFhAx44dHeeL70tNTaVx48ZOz+zcuXOpz/L29sbbWxm/iEhNk5FbyKw1+/nPb/tIPnESALOHiYHtGzG6dzRdm9TVas21lEsTIIvFQkxMDImJiQwZMgSwD4JOTExk7Nix57zvtdde45VXXmHevHl069bN6Vp0dDSNGjUiMTHRkfBkZmaycuVKHnzwwcqqioiIVCO70rKY8dtevlqbzMlC+6ak9fy8GNGjCXf1bErjIC1EWNu5vAssPj6eUaNG0a1bN3r06MG0adPIyclh9OjRAIwcOZLw8HASEhIAePXVV5k4cSKff/45UVFRjnE9/v7++Pv7YzKZGDduHC+//DItW7Z0TIMPCwtzJFkiIlLz2GwGS3Yc4d/Lk/h1Z7rjfJtGAYzuHcVNncPx8aqYKdTi/lyeAA0fPpwjR44wceJEUlJS6Ny5M3PnznUMYt6/f7/TapkffPABBQUFDBs2zOk5kyZN4oUXXgDgqaeeIicnhwceeIATJ05wxRVXMHfuXK0BJCJSA2XnFzF7zQH+s2IfSen2TUlNJoi7LJTRvaPo2ayBurnkLC5fB6g60jpAIiLV3/6jucz4bS9frjlAVr59U9IAb09u6x7JqJ5RNGng5+IIpaq5zTpAIiIiZWGzGfy2+ygzfttL4rZUiv8XvllwHe7uHcXQrhHU8dZXm1yYfkpERKTa23c0h6/WHuSrdcmO2VwAV7VqyOjeUfRp2RCPi9yUVASUAImISDWVnV/ETxsPM3vtQVadtnZPgI8nQzqHM6pXFC1C/F0YobgzJUAiIlJt2GwGvycdZfbag/y8McUxhd1kgitaBDMsJoIB7RppNpdcMiVAIiLicvuP5vLVuoN8te4gB4+XdHE1C67D0JgIbukarrV7pEIpARIREZfIOa2L6/TtKQK8PbmhUxjDYiK0UrNUGiVAIiJSZWw2g1V7j/HlmoP8vOkwuQVnd3H1b9sIX4u6uKRyKQESEZFKd+BYSRfXgWMlXVzRwXUYFhPBzV3CCaurLi6pOkqARESkUuQWFPHTxhRmrz3A73tKurj8vT25oWNjbu0WQdcm9dTFJS6hBEhERCqMYRisSjrG7LUH+WnjYXJO6+Lq3bxkFpe6uMTVlACJiMglO3Asl6/XJfPVuoPsP5brOB/VwM/exdU1gnB1cUk1ogRIRETKJbegiLmbUvhyzUFW7DnqOF/HYuaGjmEM6xZBt6bq4pLqSQmQiIhcNMMwWL33OLPXHuDHP0u6uAB6t2jg6OLys+jrRao3/YSKiMgFJZ84eWovroPsO1rSxdWkvr2L65au4UTU0+7r4j6UAImISKlOFliZu9m+UOFvu486dl6vYzEzqGNjhsVE0j1KXVzinpQAiYiIg2EYrNl3nNlrDvLjxsNk5xc5rvVqbu/iuq69urjE/eknWERESD5xkm/WHWT22oPsPa2LK7K+L8O6RnJL13Ai66uLS2oOJUAiIrXUyQIr8zanMHvtQZbvTnd0cflZzAzq0JhhMRF0j6qPh4e6uKTmUQIkIlKLGIbBuv3Hmb32ID/8cZis07q4Lm9Wn2ExkQxs34g63vp6kJpNP+EiIrXAoRMn+WZ9MrPXHiQpPcdxPqKeL8NiIhjaNUJdXFKrKAESEamh8gpLuriW7XLu4rr+VBdXD3VxSS2lBEhEpAaxd3GdONXFdcipiys2uj7DYiK4vkNjdXFJraffABGRGuBwxkn7XlxrD7LntC6u8LolXVxNGqiLS6SYEiARETeVV2jlly2p9i6unUewneri8vUyM7BDI4bFRHB5dAN1cYmUQgmQiIgbMQyD9QfsXVzf/3GIrLySLq4ep3Vx+auLS+S89BsiIuIGUjLy+Hq9faHCPUecu7iGdg1naEwETRvUcWGEIu5FCZCISDWVV2hl/qkurl9P6+Ly8fLg+vb2WVyXN1MXl0h5KAESEalGDMNgw2ldXJmnd3FF2bu4BnZoRICPlwujFHF/SoBERKqB1Mw8x0KFu9KyHefDgnwYemoWV1SwurhEKooSIBERF8krtLJgq72La+kO5y6ugae6uHqqi0ukUni4OoD333+fqKgofHx8iI2NZdWqVecsu3nzZoYOHUpUVBQmk4lp06adVcZqtfL8888THR2Nr68vzZs356WXXsIoXgJVRMSFDMPgjwMneG7ORnq8soCxn69n8XZ78tOtaT2m3tKB1c/G8fbwzvRuEazkR6SSuLQFaNasWcTHx/Phhx8SGxvLtGnTGDBgANu3byckJOSs8rm5uTRr1oxbb72Vxx57rNRnvvrqq3zwwQf85z//oV27dqxZs4bRo0cTFBTEI488UtlVEhEpVdppXVw7T+viahzkw9CuEQyNiSBaXVwiVcZkuLBpJDY2lu7duzN9+nQAbDYbkZGRPPzww4wfP/6890ZFRTFu3DjGjRvndP6GG24gNDSUf/3rX45zQ4cOxdfXl88+++yi4srMzCQoKIiMjAwCAwPLVikRkVPyi6ws2JLG7LUHWHJaF5e3pwcD2zdiWEwkPZs3wKxWHpEKUZbvb5e1ABUUFLB27VomTJjgOOfh4UFcXBwrVqwo93N79erFxx9/zI4dO2jVqhV//PEHy5Yt46233qqIsEVEzsswDDYmZzB77UG+3XCIjJOFjmsxTesxLCaCQR0bE6hZXCIu5bIEKD09HavVSmhoqNP50NBQtm3bVu7njh8/nszMTNq0aYPZbMZqtfLKK69w5513nvOe/Px88vPzHX/PzMws9+uLSO2UlpXHnFNdXDtSnbu4bukaztCuETRr6O/CCEXkdDVuFtgXX3zBf//7Xz7//HPatWvHhg0bGDduHGFhYYwaNarUexISEpg8eXIVRyoi7i6/yEri1jRmrz3Ikh1HsJ7q4/L29GBAO/teXL1bBKuLS6QaclkCFBwcjNlsJjU11el8amoqjRo1Kvdzn3zyScaPH8/tt98OQIcOHdi3bx8JCQnnTIAmTJhAfHy84++ZmZlERkaWOwYRqbkMw2BTciaz1x7g2z8OcSK3pIura5O6DIuJZFDHxgT5qotLpDpzWQJksViIiYkhMTGRIUOGAPZB0ImJiYwdO7bcz83NzcXDw3l2v9lsxmaznfMeb29vvL29y/2aIlLzpWXl8e36Q8xee5DtqVmO86GB3tzSNYJhMRE0VxeXiNtwaRdYfHw8o0aNolu3bvTo0YNp06aRk5PD6NGjARg5ciTh4eEkJCQA9oHTW7Zscfw5OTmZDRs24O/vT4sWLQAYPHgwr7zyCk2aNKFdu3asX7+et956i3vuucc1lRQRt1VQZGPhNvtChYu2l3RxWU7r4rpCXVwibsml0+ABpk+fzuuvv05KSgqdO3fm3XffJTY2FoC+ffsSFRXFjBkzANi7dy/R0dFnPaNPnz4sXrwYgKysLJ5//nm++eYb0tLSCAsLY8SIEUycOBGLxXJRMWkavEjtZRgGmw9lnprFlczx07q4ujSpy7CYCG7oGKYuLpFqqCzf3y5PgKojJUAitU96dr5jFte2lJIurpCAki6uFiHq4hKpztxiHSAREVezd3HZZ3Et3p5G0WldXP3bhjq6uDzNLt81SEQqmBIgEal1NjkWKnTu4uoUWZdbYyIY3DGMID91cYnUZEqARKTW2JScweTvN7N673HHuZAAb27uGs6wrhG0DA1wYXQiUpWUAIlIjXcit4A3ftnO5yv3YzPAYvbg2rahDOsWwZXq4hKplZQAiUiNZbUZzFp9gNfnbXN0dQ3uFMYz17ehcZCvi6MTEVdSAiQiNdL6/ceZ9N1m/jyYAUCrUH8m39iens0buDgyEakOlACJSI2Snp3Pa3O38cWagwAEeHvy2LWtuKtnU7zU1SUipygBEpEaochq47Pf9/Hm/B1k5RUBMCwmgqeva0PDAG11IyLOlACJiNtbuecok77b7FjAsH14IJNvbE9M03oujkxEqislQCLitlIz85jy01a+3XAIgLp+Xjw5oDW3d2+i/blE5LyUAImI2ykosvHJ8iTeTdxJToEVkwlG9GjCk/1bU6/Oxe35JyK1mxIgEXErv+48wqTvNrPnSA5g36D0xRvb0yEiyMWRiYg7UQIkIm7h4PFcXv5hK3M3pwAQ7G/h6evaMLRrBB7q7hKRMlICJCLVWl6hlY+X7uHvi3eRV2jD7GFiZM+mjItrRZCv9usSkfJRAiQi1ZJhGCRuTePFH7aw/1guALHR9Xnxpva0bqQ9u0Tk0igBEpFqJyk9hxe/38yi7UcAaBTow7ODLuOGjo0xmdTdJSKXTgmQiFQbuQVFvL9oF/9YmkSB1YaX2cS9VzTj4atbUMdb/1yJSMXRvygi4nKGYfDTxhRe/nELhzPyALiyZTAv3NiO5g39XRydiNRESoBExKV2pmYx6bvN/Lb7KADhdX2ZOLgt/duGqrtLRCqNEiARcYmsvELeWbCTGb/tpchmYPH04ME+zXmwb3N8vMyuDk9EajglQCJSpQzD4Jv1yST8vI0jWfkAXNs2lOcHtaVJAz8XRycitYUSIBGpMpsPZTDp282s2XccgOjgOkwc3JZ+rUNcHJmI1DZKgESk0p3ILeCt+Tv47Pd92Azw9TLz8DUtuPeKaLw91d0lIlVPCZCIVBqrzeCLNQd4fd52juUUAHBDx8Y8c/1lhNX1dXF0IlKbKQESkUqxfv9xJn23mT8PZgDQKtSfF25sR6/mwS6OTERECZCIVLD07Hxem7uNL9YcBCDA25Nx17ZiZM+meJk9XBydiIidEiARqRBFVhuf/b6PN+fvICuvCIChXSN4emBrQgJ8XBydiIgzJUAicslW7jnKpO82sy0lC4B2YYG8eFM7YprWd3FkIiKlUwIkIuWWmplHwk9bmbPhEABBvl48OaA1I3o0weyhVZxFpPpSAiQiZVZQZGPGb0m8s2AnOQVWTCYY0aMJT/RvTf06FleHJyJyQUqARKRMft15hBe+28zuIzkAdI6sy0s3tadDRJCLIxMRuXgun5Lx/vvvExUVhY+PD7GxsaxateqcZTdv3szQoUOJiorCZDIxbdq0UsslJyfzl7/8hQYNGuDr60uHDh1Ys2ZNJdVApHY4eDyXBz9by13/WsXuIzk0qGPh9WEd+frBXkp+RMTtuLQFaNasWcTHx/Phhx8SGxvLtGnTGDBgANu3byck5Oyl8XNzc2nWrBm33norjz32WKnPPH78OL1796Zfv378/PPPNGzYkJ07d1KvXr3Kro5IjZRXaOUfS/fw/uJd5BXaMHuYGNmzKePiWhHk6+Xq8EREysVkGIbhqhePjY2le/fuTJ8+HQCbzUZkZCQPP/ww48ePP++9UVFRjBs3jnHjxjmdHz9+PMuXL+fXX38td1yZmZkEBQWRkZFBYGBguZ8j4u4St6Yy+fst7D+WC0BsdH0m39SONo30eyEi1U9Zvr9d1gVWUFDA2rVriYuLKwnGw4O4uDhWrFhR7ud+9913dOvWjVtvvZWQkBC6dOnCP/7xj/Pek5+fT2ZmptMhUpvtTc/hnhmrufc/a9h/LJfQQG/eHdGFmQ9cruRHRGoElyVA6enpWK1WQkNDnc6HhoaSkpJS7ufu2bOHDz74gJYtWzJv3jwefPBBHnnkEf7zn/+c856EhASCgoIcR2RkZLlfX8Sd5RYU8ca87fR/eykLt6XhZTbxtz7NWfh4X27sFIbJpKntIlIz1LhZYDabjW7dujFlyhQAunTpwqZNm/jwww8ZNWpUqfdMmDCB+Ph4x98zMzOVBEmtYhgGP29K4eUftnAoIw+AK1sG88KN7Wje0N/F0YmIVDyXJUDBwcGYzWZSU1OdzqemptKoUaNyP7dx48a0bdvW6dxll13GV199dc57vL298fb2LvdririznalZvPD9ZpbvOgpAeF1fJg5uS/+2oWrxEZEay2VdYBaLhZiYGBITEx3nbDYbiYmJ9OzZs9zP7d27N9u3b3c6t2PHDpo2bVruZ4rURFl5hbzy4xYGvvMry3cdxeLpwaPXtGRBfB8GtGuk5EdEajSXdoHFx8czatQounXrRo8ePZg2bRo5OTmMHj0agJEjRxIeHk5CQgJgHzi9ZcsWx5+Tk5PZsGED/v7+tGjRAoDHHnuMXr16MWXKFG677TZWrVrFxx9/zMcff+yaSopUM4ZhMGdDMlN+2saRrHwA4i4LZeINbWnSwM/F0YmIVA2XToMHmD59Oq+//jopKSl07tyZd999l9jYWAD69u1LVFQUM2bMAGDv3r1ER0ef9Yw+ffqwePFix99/+OEHJkyYwM6dO4mOjiY+Pp7777//omPSNHipqTYfyuCF7zazeu9xAKIa+DFpcDv6tTl73S0REXdTlu9vlydA1ZESIKlpMnILeXP+dj77fR82A3y9zIy9ugX3XRmNt6fZ1eGJiFSIsnx/17hZYCJSwmYz+GLNAV6bt51jOQUADOrYmGevv4ywur4ujk5ExHWUAInUUBsOnGDSt5v442AGAC1D/Jl8Uzt6NQ92cWQiIq6nBEikhjmanc9rc7cza80BAAK8PRl3bStG9myKl9nl+x+LiFQLSoBEaogiq43/rtzPm79sJzOvCIChXSN4emBrQgJ8XBydiEj1ogRIpAZYlXSMid9uYltKFgBtGwfy4k3t6BZV38WRiYhUT0qARNxYamYeCT9tZc6GQwAE+XrxxIDW3NGjCWYPLWQoInIuSoBE3FBBkY0ZvyXxzoKd5BRYMZng9u5NeHJAa+rXsbg6PBGRak8JkIibWbYznUnfbWL3kRwAOkfW5cWb2tExoq5rAxMRcSNKgETcRPKJk7z8wxZ+3pQCQIM6Fp4e2IZhXSPwUHeXiEiZlCsBWrRoEf369avoWESkFHmFVv756x6mL9pFXqENs4eJuy5vymPXtiLI18vV4YmIuKVyJUDXXXcdERERjB49mlGjRhEZGVnRcYkIsHBbKpO/38K+o7kA9Iiuz+Qb23FZY23RIiJyKcq1KlpycjJjx45l9uzZNGvWjAEDBvDFF19QUFBQ0fGJ1Er7juZw74zV3DNjDfuO5hIa6M07t3dm1gOXK/kREakAl7wZ6rp16/jkk0/43//+B8Add9zBvffeS6dOnSokQFfQZqjiKicLrPx98S4+WrKHAqsNL7OJe66I5uGrW+LvrSF7IiLnU+W7wR86dIiPP/6YqVOn4unpSV5eHj179uTDDz+kXbt2l/r4KqcESKqaYRj8vCmFV37cSvKJkwBc2TKYSYPb0SLE38XRiYi4h7J8f5d7Y6DCwkJmz57N9ddfT9OmTZk3bx7Tp08nNTWVXbt20bRpU2699dbyPl6k1tiVlsVf/rWSh/67juQTJwmv68uHf4nh03t6KPkREakk5WoBevjhh/nf//6HYRjcdddd3HfffbRv396pTEpKCmFhYdhstgoLtqqoBUiqQlZeIe8m7uST5XspshlYPD34W5/mPNinOb4Ws6vDExFxO2X5/i7XoIItW7bw3nvvccstt+Dt7V1qmeDgYBYtWlSex4vUaIZh8O2GQ0z5aStpWfkAxF0WysQb2tKkgZ+LoxMRqR0qZAxQTaMWIKksWw5lMum7TazeexyAqAZ+TBrcjn5tQlwcmYiI+6v0FqCEhARCQ0O55557nM7/+9//5siRIzz99NPleaxIjZWRW8hb87fzf7/vw2aAr5eZsVe34L4ro/H2VHeXiEhVK9cg6I8++og2bdqcdb5du3Z8+OGHlxyUSE1hsxnMWr2ffm8u5j8r7MnPoI6NSXy8D2P6tVDyIyLiIuVqAUpJSaFx48ZnnW/YsCGHDx++5KBEaoI/Dpxg4reb+ONgBgAtQ/yZfGM7erUIdnFkIiJSrgQoMjKS5cuXEx0d7XR++fLlhIWFVUhgIu7qaHY+r8/bzqw1BzAM8Pf2ZFxcS0b1isLLXO6VJ0REpAKVKwG6//77GTduHIWFhVx99dUAJCYm8tRTT/H4449XaIAi7qLIauPzVft5Y952MvOKALilazjjB7YhJMDHxdGJiMjpypUAPfnkkxw9epSHHnrIsf+Xj48PTz/9NBMmTKjQAEXcweq9x5j47Wa2Hs4EoG3jQF68qR3douq7ODIRESnNJU2Dz87OZuvWrfj6+tKyZctzrgnkbjQNXi5WWmYeCT9v45v1yQAE+XrxxIDW3NGjCWYPk4ujExGpXSp9Gnwxf39/unfvfimPEHFLhVYbM5bvZdqCHeQUWDGZ4PbuTXhyQGvq17G4OjwREbmAcidAa9as4YsvvmD//v2ObrBiX3/99SUHJlJdLduZzgvfb2ZXWjYAnSPr8uJN7egYUde1gYmIyEUr15SUmTNn0qtXL7Zu3co333xDYWEhmzdvZuHChQQFBVV0jCLVxsdLd/OXf61kV1o2DepYeG1oR75+sJeSHxERN1OuFqApU6bw9ttvM2bMGAICAnjnnXeIjo7mr3/9a6nrA4nUBLvSsnlj3g4A/nJ5E57s34YgPy8XRyUiIuVRrhag3bt3M2jQIAAsFgs5OTmYTCYee+wxPv744woNUKQ6sNkMnvl6IwVWG1e3CeGlm9or+RERcWPlSoDq1atHVlYWAOHh4WzatAmAEydOkJubW+bnvf/++0RFReHj40NsbCyrVq06Z9nNmzczdOhQoqKiMJlMTJs27bzPnjp1KiaTiXHjxpU5LpFis9YcYNXeY/hZzLx4UztMJs3wEhFxZ+VKgK666irmz58PwK233sqjjz7K/fffz4gRI7jmmmvK9KxZs2YRHx/PpEmTWLduHZ06dWLAgAGkpaWVWj43N5dmzZoxdepUGjVqdN5nr169mo8++oiOHTuWKSaR06Vl5jHlp60APN6/NRH1/FwckYiIXKpyJUDTp0/n9ttvB+DZZ58lPj6e1NRUhg4dyr/+9a8yPeutt97i/vvvZ/To0bRt25YPP/wQPz8//v3vf5davnv37rz++uvcfvvt5113KDs7mzvvvJN//OMf1KtXr0wxiZxu8g9byMoromNEEHf3inJ1OCIiUgHKnAAVFRXxww8/YDbbd7H28PBg/PjxfPfdd7z55ptlSjYKCgpYu3YtcXFxJQF5eBAXF8eKFSvKGpqTMWPGMGjQIKdni5RV4tZUfvzzMGYPE1Nu7qDFDUVEaogyzwLz9PTkb3/7G1u3br3kF09PT8dqtRIaGup0PjQ0lG3btpX7uTNnzmTdunWsXr36osrn5+eTn5/v+HtmZma5X1tqjpz8Ip6fYx/fdt8V0bQP1xIPIiI1Rbm6wHr06MGGDRsqOJSKceDAAR599FH++9//4uNzcRtQJiQkEBQU5DgiIyMrJ7iCXEj6FYryL1xWXO6NX7ZzKCOPyPq+jItr5epwRESkApVrHaCHHnqI+Ph4Dhw4QExMDHXq1HG6frGDjoODgzGbzaSmpjqdT01NveAA53NZu3YtaWlpdO3a1XHOarWydOlSpk+fTn5+vqP7rtiECROIj493/D0zM7NykqB9v8F/h4KnLzTtBc36QLO+ENoBPMqVi0ol2XDgBDN+2wvAK0M64Gsxn/8GERFxK+VKgIoHQD/yyCOOcyaTCcMwMJlMWK3Wi3qOxWIhJiaGxMREhgwZAoDNZiMxMZGxY8eWJzSuueYaNm7c6HRu9OjRtGnThqeffvqs5AfA29u7ajZyPXkc6oRAThrsTrQfAL717clQ9KmEqH505cci51RotTHh640YBtzcJZyrWjV0dUgiIlLBypUAJSUlVVgA8fHxjBo1im7dutGjRw+mTZtGTk4Oo0ePBmDkyJGEh4eTkJAA2AdOb9myxfHn5ORkNmzYgL+/Py1atCAgIID27ds7vUadOnVo0KDBWeerXMdbocMwSNsKexbbj33L4eQx2PyN/QCo27SkdSi6D9QJdmHQtc+/liWx9XAmdf28eG7QZa4OR0REKkG5EqCmTZtWWADDhw/nyJEjTJw4kZSUFDp37szcuXMdA6P379+Px2ndQ4cOHaJLly6Ov7/xxhu88cYb9OnTh8WLF1dYXJXGZILQtvaj50NgLYTktSUJ0cHVcGIfrPvUfgA06nCqdagfNO0JljrnewW5BPuO5jBtgX27i+cGtaWBfxW0DIqISJUzGYZhlPWmTz/99LzXR44cWe6AqoPMzEyCgoLIyMggMDCwal88P9s+VmjPYkhaAqmbnK97eEFkD3vrULO+ENYVzOXKY+UMhmFw179WsWxXOr1bNOCze2O14rOIiBspy/d3uRKgM9f6KSwsJDc3F4vFgp+fH8eOHSvrI6sVlyZAZ8pOg6SlJS1EGQecr1sCIOqKkoSoYWt7K5OU2dfrDhL/xR94e3owb9xVRAWrpU1ExJ2U5fu7XE0Hx48fP+vczp07efDBB3nyySfL80g5F/8Q+7ihDsPAMODYnpLWoT1LIO8E7PjZfgD4N3IePxQU7sLg3cexnAJe+sE+tuzRuJZKfkREarhytQCdy5o1a/jLX/5ySYsYVgfVqgXofGxWSPnzVOvQEti/AorynMs0aFnSOhR1BfjWrfo43UD8Fxv4el0ybRoF8P3DV+Bl1rIEIiLuptJbgM75ME9PDh06VJGPlPPxMENYF/txxWNQmAcHVp5qHVoMh9bD0Z32Y/U/wORhL1ucEEX0AK+LWyyyJlu2M52v1yVjMkHCLR2U/IiI1ALlagH67rvvnP5uGAaHDx9m+vTpREZG8vPPP1dYgK7gNi1AF3LyOOxdZm8d2rPYngidztMHmvQ8lRD1gUYd7UlVLXKywMqAaUvZfyyXu3tF8cKN7VwdkoiIlFOlD4L2OGPVYpPJRMOGDbn66qt58803ady4cVkfWa3UmAToTBnJJa1DexZDtvMK3PjWg+irTluQsVmNH1A99edtfLhkN42DfJgf3wd/b82oExFxV5WeANV0NTYBOp1hwJHtJcnQ3mVQkOVcJqjJaQOqr7IPyK5BthzKZPD0ZVhtBv8c2Y24tqEXvklERKotJUCXqFYkQGeyFtrHDBUnRAdWga3QuUxo+5LZZU17gbe/CwKtGFabwS1/X84fBzO4vkMj/n5njKtDEhGRS1TpCdDQoUPp0aMHTz/9tNP51157jdWrV/Pll1+W9ZHVSq1MgM5UkAP7VsCeRfYxRKnO+6vh4WkfRF3cQhQeA2Yvl4RaHp8sT2Ly91sI8PEkMb4PIYEaDC4i4u4qPQFq2LAhCxcupEOHDk7nN27cSFxc3Fm7u7sbJUClyD4Ce09bkPHEfufrFn/7NPvi8UMhl1Xb8UPJJ05y7VtLyC2wMuXmDtwR28TVIYmISAWo9Gnw2dnZWCyWs857eXmRmZlZnkdKdeffENoPtR8Ax5JKkqGkpfYNXXfMtR9g3/X+9AUZ60a6KHBnhmHw/JxN5BZY6R5Vj9u7V4+4RESkapUrAerQoQOzZs1i4sSJTudnzpxJ27ZtKyQwqebqR9uPbqPBZrN3kTl2uF8BOWmw8Uv7AVC/ecn6Q9FX2mecucBPG1NYuC0NL7OJhFs64OFRPVupRESkcpUrAXr++ee55ZZb2L17N1dffTUAiYmJ/O9//3P78T9SDh4e0LiT/ej9KBTl2wdRF2/ZkbwWju22H2v+BZggrHNJQhR5eZUsyJiRW8ik7zYD8FDfFrQICaj01xQRkeqp3LPAfvzxR6ZMmcKGDRvw9fWlY8eOTJo0iT59+lR0jFVOY4Aq2MkTsG95yZYd6dudr5u9ocnlJQlR406VsiDjhK838r9V+2nesA4/PXol3p61a9FHEZGaTtPgL5ESoEqWeahkdeqkJZB12Pm6T117N1mzvtCsX4UsyLhyz1GGf/w7AF/8tSc9outf0vNERKT6qfRB0KtXr8ZmsxEbG+t0fuXKlZjNZrp161aex0ptERgGnUfYD8OA9B0lrUN7f7XvcL/1e/sBEBhx2vihqyCgbAsW5hdZmfCNfRr/iB5NlPyIiEj5EqAxY8bw1FNPnZUAJScn8+qrr7Jy5coKCU5qAZMJGra2H7F/BWuRfUHGpMX2hOjASsg8CBs+sx8AIW1LEqKmvcD7/GN5/r5oN3uO5NAwwJvxA9tUdo1ERMQNlKsLzN/fnz///JNmzZo5nU9KSqJjx45kZWWd4073oC6waqQgB/avKOkyS/nT+bqHJ4R3K9nQNbwbeJYs0bArLYuB7/xKodXg/Tu6Mqije+9TJyIi51bpXWDe3t6kpqaelQAdPnwYT09tJikVyFIHWsTZD4Cco84LMh7fCwd+tx9LpoJXHYjqDdF9sEX34Zk5GRRaDa5pE8L1HRq5sCIiIlKdlKsFaMSIERw+fJhvv/2WoKAgAE6cOMGQIUMICQnhiy++qPBAq5JagNzI8b3OA6pzjzpdPmIEsooO9L52KHXbXwt1teqziEhNVemzwJKTk7nqqqs4evQoXbp0AWDDhg2EhoYyf/58IiPde3VdJUBuymaD1E2QtIT8HQux7l2OH/nOZeo3K1mdOvoq8NOAaBGRmqJKpsHn5OTw3//+lz/++MOxDtCIESPw8nKfDTHPRQmQ+3vov2uZv/Egw0IP80rHdDyKF2Q0rKeVMtnXHCresqNJT/DydVXIIiJyiapsHaAtW7awf/9+CgoKnM7feOON5X1ktaAEyL3N35LK/Z+uwexh4vuxV9A27NRnmJd52oKMi+HINucbzd7QJPbUhq797KtVV8KCjCIiUjkqPQHas2cPN998Mxs3bsRkMmEYBqbTFqqzWq3nubv6UwLkvrLzi7j2rSUczsjjb32an3/ae+Zh+0auxQlR1iHn6z5BEHVlyZT7Bi2q7Q73IiJSBbPAHn30UaKjo0lMTCQ6OpqVK1dy7NgxHn/8cd54441yBS1SEd6Yt53DGXk0beDHuLiW5y8c2Bg6DbcfhgFHd522w/2vkJcB236wHwABYSXJUMtrNX5IRMSNlasFKDg4mIULF9KxY0eCgoJYtWoVrVu3ZuHChTz++OOsX7++MmKtMmoBck/r9x/nlg9+wzDgs3tjuaJlcPkfZi2Cw3/AnkX2hOjASrCe1tVrMtsXYWxzA7S5XrPLRESqgUpvAbJarQQE2FffDQ4O5tChQ7Ru3ZqmTZuyffv2C9wtUvEKrTYmfL0Rw4BbuoZfWvIDYPaEiBj7cdUTUJBrX2toz2LYuQDSNtu37dj7K8x9Ghp1PJUMDYLQduoqExGp5sqVALVv354//viD6OhoYmNjee2117BYLHz88cdnLY4oUhX+8esetqVkUc/Pi+cGta34F7D4QfOr7ce1L8KxJNj+E2z70b5Sdcqf9mPxFKjbtCQZanK5BlKLiFRD5eoCmzdvHjk5Odxyyy3s2rWLG264gR07dtCgQQNmzZrF1VdfXRmxVhl1gbmXvek5DJi2lPwiG2/d1olbukZUbQA56bBjrj0Z2r0QivJKrvk1gNYD7QlRs76aZi8iUomqbBr86Y4dO0a9evWcZoO5KyVA7sMwDP7yr5Us33WUK1sG8+k9PVz7M1iQA7sS7cnQjrn2ne2LeflBi2vsyVDL/hpELSJSwVySANUkSoDcx+y1B3niyz/w8fLgl3F9aNLAz9UhlbAWwr7f7MnQth/tu9oXM5nte5a1uQFaXw913Xv1dBGR6qAs398eVRTTeb3//vtERUXh4+NDbGwsq1atOmfZzZs3M3ToUKKiojCZTEybNu2sMgkJCXTv3p2AgABCQkIYMmSIBmfXQEez83n5xy0AjItrVb2SHwCzl32V6etfg8c2wQNL4KqnIKSdfUXqpKXw81MwrT18dBUseR1St9in5IuISKVyeQI0a9Ys4uPjmTRpEuvWraNTp04MGDCAtLS0Usvn5ubSrFkzpk6dSqNGpe/uvWTJEsaMGcPvv//O/PnzKSwspH///uTk5FRmVaSKvfzjVk7kFnJZ40DuvSLa1eGcn8lkX1n66mfhod/g4XXQ/2X79huY7FPuF70MH/SEd7vAvGdh3wqwufeioiIi1ZXLu8BiY2Pp3r0706dPB8BmsxEZGcnDDz/M+PHjz3tvVFQU48aNY9y4cectd+TIEUJCQliyZAlXXXXVBWNSF1j1t3THEUb+exUmE8x5qDedIuu6OqTyyz4CO34+NYh6EVhP28DVL/iMQdQ+LgtTRKS6q/R1gCpKQUEBa9euZcKECY5zHh4exMXFsWLFigp7nYyMDADq1y990Gl+fj75+SVfOpmZmRX22lLxThZYeXbORgDu7hXl3skPgH9D6DrSfuRnw+7TBlHnpsP6/7MfXnVKBlG36g++9VwduYiI23JpApSeno7VaiU0NNTpfGhoKNu2bTvHXWVjs9kYN24cvXv3pn379qWWSUhIYPLkyRXyelL5pi3YwYFjJwmv68sT/Vu7OpyK5e0PbW+yH9ZC++atW3+wJ0RZh2Drd/bDwxOirigZRB0U7urIRUTciksToKowZswYNm3axLJly85ZZsKECcTHxzv+npmZSWSkZuVUR5uSM/jnsiQAXhrSjjreNfhH2OxVsvfY9a/DofUlM8qObC3Zt+ynJyCsi33hxTY3QMM2WolaROQCXPrtERwcjNlsJjU11el8amrqOQc4l8XYsWP54YcfWLp0KRER514cz9vbG29v70t+PalcVpvBhK83YrUZDOrYmKvbhF74pprCZILwrvbjmufh6O6SZOjASntydGg9LHwZ6jc7lQwNhohuWolaRKQULp0FZrFYiImJITEx0XHOZrORmJhIz549y/1cwzAYO3Ys33zzDQsXLiQ6uprPEJKLMuO3vWxMziDAx5NJgythuwt30qA59H4E7p0HT+yAwe9CywFgtsCxPfDbe/Dv/vBmG/juEdjxCxTmXfi5IiK1hMv7D+Lj4xk1ahTdunWjR48eTJs2jZycHEaPHg3AyJEjCQ8PJyEhAbAPnN6yZYvjz8nJyWzYsAF/f39atGgB2Lu9Pv/8c7799lsCAgJISUkBICgoCF9fbUXgjg4ez+XNX+xrOT1z/WWEBGg2lIN/CMSMsh/5WbBrwalB1L9AThqs+4/9sPhDi7hTK1FfC751XR25iIjLuHwaPMD06dN5/fXXSUlJoXPnzrz77rvExsYC0LdvX6KiopgxYwYAe/fuLbVFp0+fPixevBjgnFshfPLJJ9x9990XjEfT4KsXwzC4Z8ZqFm0/Qo/o+sy8/3I8PDTG5YKKCmDfspKusqzDJdc8PCHqylNdZYMgMMx1cYqIVBBthXGJlABVL9//cYiH/7cei9mDnx69khYh/q4Oyf3YbHD49EHUZ8yyDI8pGUQd3EqDqEXELSkBukRKgKqPjNxCrnlrMenZBTwW14pH41q6OqSaIX0XbDs1vf7gauC0fwYatChJhsK7gYfLF4wXEbkoSoAukRKg6mP8V38yc/UBWoT48+MjV+DtqRlNFS4rBbafWok6aQlYC0qu+YeWrEQdfRV4arakiFRfSoAukRKg6uH3PUe5/ePfAfjybz3pHlX6St5SgfIySwZR7/wF8k9bFd0SAC1PG0TtE+S6OEVESuE2W2GInEteoZVnvrZvd3FnbBMlP1XFJxDa32I/igpg79JT44Z+guwU2PyN/fDwsrcItRlkX4k6sLGrIxcRKRO1AJVCLUCu99Yv23l34S5CAryZH9+HIF8vV4dUu9lscGhdybih9B3O18O7nbYSdSvXxCgitZ66wC6REiDX2pGaxaB3f6XQavDBnV0Z2EGtC9XOkR2w/cfTBlGfpkHL0wZRx2gQtYhUGSVAl0gJkOvYbAa3frSCtfuOE3dZKP8YGXPOdZ2kmsg8DDt+tm/amrQUbIUl1/wbQZvr7QlR1FXgaXFdnCJS4ykBukRKgFzns9/38dycTdSxmJkf34ewulq5263kZcDO+acGUc+HgqySa96B9sHTbQZBi2vt441ERCqQBkGLW0rNzOPVn+0L9D11XRslP+7IJwg6DLMfRfmQ9Kt93ND2nyA7FTZ9ZT88vKBZn1ODqAdBQC3a2FZEqgW1AJVCLUCu8bf/W8vczSl0jqzLVw/2wqztLmoOmw2S154aRP0DHN112kUTRHQ/bSXqFi4LU0Tcm7rALpESoKo3b3MKf/2/tXh6mPj+4Su4rLHe9xrtyPaSGWXJa52vBbcuSYbCumgQtYhcNCVAl0gJUNXKyivk2reWkpKZx0N9m/PUdW1cHZJUpcxD9i6ybT+eGkRdVHItoLF9naE2g+ybt2oQtYichxKgS6QEqGpN+nYT/1mxj6gGfswddxU+XtruotY6eeLUStQ/nBpEnV1yzTsIWvU/NYg6DrwDXBamiFRPSoAukRKgqrNu/3GGfvAbhgGf3xdLrxbBrg5JqovCPHuLUPEg6pwjJdfMFmjWt2Qlav8Ql4UpItWHEqBLpASoahQU2Rj83jK2p2YxLCaCN27t5OqQpLqyWeHgmpJB1Mf2nHbRBJE9SsYNNWjusjBFxLWUAF0iJUBV4/1Fu3h93nbq17GQGN+HenU0vkMugmE4D6I+tM75esPLTiVDg+yDqLWQpkitoQToEikBqnxJ6TkMmLaUgiIb04Z3ZkiXcFeHJO4qI/nUIOofYO+yMwZRh5UkQ1FXgFl7yonUZEqALpESoMplGAZ3/GMlK/Yc5apWDfnP6O7a7kIqxsnjp1ai/gF2LoDCnJJrPkHQcsBpg6j9XReniFQKJUCXSAlQ5fpyzQGenP0nPl4ezH+sD5H1/VwdktREhXmQtORUV9lPkJtecs3Dy774YrM+EN0HIrqpdUikBlACdImUAFWe9Ox84t5awoncQp65vg0PXKUBq1IFbFb7rvXbfrBv2no8yfm6Vx1o2qskIQptrwUYRdyQEqBLpASo8jw6cz3fbjhE28aBfDe2N55mfclIFTMM+yyypCWwZ4l9qv3JY85l/BrYF14sTojqN9NgahE3oM1QpVpavD2NbzccwsMEU4d2UPIjrmEy2afKN2gO3e6x71OWuqkkIdr3G+QehS1z7AdAUKQ9ESpOiLR5q4jbUwtQKdQCVPFyC4ro//ZSDh4/yb1XRPP8DW1dHZJI6YoK7PuTFSdEB1eDrdC5TMPLSpKhqN72AdYi4nLqArtESoAq3pSftvLx0j2E1/Xll8euoo63Gh/FTRTkwL4VkLTYnhClbARO+2fT5AFhXUsSoshY8PJxVbQitZoSoEukBKhibUrO4Mbpy7AZ8Mnd3enXRtsWiBvLPWYfN1TcQnRst/N1Tx97EtSsD0T3hbDO4KH97USqghKgS6QEqOIUWW0M+ftyNiVnMrhTGO+N6OLqkEQqVsbBU4OpTyVE2SnO172D7IswFrcQNWytAdUilUSDoKXamPHbXjYlZxLo48lEjfuRmigoArrcaT8MA9J3lCRESb9CfgZs/9F+APg3guirShKiupGujV+kllILUCnUAlQxDhzLpf/bSzlZaOXVoR0Y3r2Jq0MSqVo2KxzeUJIQ7f8divKcy9RvVjLDLOoqqNPAJaGK1ATqArtESoAunWEY3P3JapbsOEJsdH1mPnC5trsQKcyDg6tKEqLkdWBYncs06nAqIeoLTXpqyw6RMlACdImUAF26bzck8+jMDVg8PZj76JU0a6h/xEXOkpdhX3eoOCFK2+J83cPTvmVHcQtReDfwtLgmVhE3UJbv72qxEt37779PVFQUPj4+xMbGsmrVqnOW3bx5M0OHDiUqKgqTycS0adMu+ZlSsU7kFvDi9/Z/yB/u10LJj8i5+ARB64EwcCo8tAKe2AlD/wVd7oK6Tew72+9fAUumwicD4dUo+GwoLH8XDv9hX8RRRMrF5YOgZ82aRXx8PB9++CGxsbFMmzaNAQMGsH37dkJCzp4unZubS7Nmzbj11lt57LHHKuSZUrGm/LSVozkFtAzx5699tNeXyEXzD4EOw+wHwLEk5y07ctNh1wL7AeBbH6KvLOky05YdIhfN5V1gsbGxdO/enenTpwNgs9mIjIzk4YcfZvz48ee9NyoqinHjxjFu3LgKeyaoC+xS/LY7nTv+sRKArx7sSUzT+i6OSKSGsNnsXWSOLTuWQ0G2c5nAiJLZZc36QEAj18Qq4iJuMw2+oKCAtWvXMmHCBMc5Dw8P4uLiWLFiRbV5plycvEIrz36zCYC/XN5EyY9IRfLwgEbt7UfPMWAttA+iLk6IDqyEzIOw4b/2AyC49WlbdlwBvnVdWgWR6sSlCVB6ejpWq5XQUOeNBUNDQ9m2bVuVPTM/P5/8/HzH3zMzM8v12rXd9IW7SErPITTQm6eua+PqcERqNrMXNIm1H32egoJc+3ih4oTo8B+Qvt1+rPrYvmVH4872rrJmxVt2+Lq6FiIu4/IxQNVBQkICkydPdnUYbm17ShYfLrFvCTD5xvYE+ni5OCKRWsbiBy2usR9g37Jj77KShOjoTji0zn4sewvM3vbkqXj8UOPOYNZXgtQeLv1pDw4Oxmw2k5qa6nQ+NTWVRo3K13ddnmdOmDCB+Ph4x98zMzOJjNTqrBfLZjMY//WfFNkM+rcN5br2Gncg4nJ+9aHtjfYDICPZPpB6z2J7UpR1+NSeZkth4UvgHWjvJiseP9SwjQZUS43m0gTIYrEQExNDYmIiQ4YMAewDlhMTExk7dmyVPdPb2xtvb+9yvZ7Af1fuY/3+E/h7ezL5pnauDkdEShMUDp1H2A/DgPSdp1qHFsPeX+1rEm3/yX4A+Ifat+woTojqaiV3qVlc3t4ZHx/PqFGj6NatGz169GDatGnk5OQwevRoAEaOHEl4eDgJCQmAfZDzli1bHH9OTk5mw4YN+Pv706JFi4t6plSclIw8Xp27HYCnrmtN4yCNKRCp9kwmaNjKfvS4/9SWHX+UdJft/x2yU2Hjl/YDoF50yYDq6KugTrBr6yByiVyeAA0fPpwjR44wceJEUlJS6Ny5M3PnznUMYt6/fz8eHiXrNR46dIguXUp2FH/jjTd444036NOnD4sXL76oZ0rFmfjtJrLzi+japC5/iW3q6nBEpDw8zBDe1X5c8RgU5cOBVSUJUfJaOJ4Ea5Ng7Qz7PaEdShKipr20ZYe4HZevA1QdaR2gizN3Uwp/+2wtnh4mfnzkSlo3CnB1SCJSGfIy7Vt2FCdEaZudr3t42rfpKE6IIrpryw5xCe0FdomUAF1YZl4h1761hNTMfMb2a8ETA1q7OiQRqSrZR+zJUHFCdGKf83UvP/tGrsUJUaOO9nWMRCqZ2yyEKO7r9bnbSc3MJzq4DmOvbuHqcESkKvk3dN6y4/jekg1dk5ZCzhHYnWg/AHzrQdSVpxKivtCguWaYicupBagUagE6v7X7jjHswxUYBnx+fyy9mmswpIicYhj2LTuKE6K9y6Egy7lMYHjJ7LLoPhDY2DWxSo2jLrBLpATo3AqKbNzw3q/sSM3mtm4RvDask6tDEpHqzFoIh9aXJEQHVoK1wLlMcKuShCjqCnuLkUg5KAG6REqAzu29xJ28OX8HDepYSHy8D3X9NNBRRMqgIBcO/F6SEB3aAJz2NWTygMadShKiJj21ZYdcNCVAl0gJUOl2H8lm4LRfKbDaeOf2ztzUOdzVIYmIuzt53L5lR3FClL7D+brZYt+3rHj8UFgXbdkh56QE6BIpATqbYRjc/vHvrEw6Rp9WDZkxujsmDWIUkYqWeejUlh2nEqLMZOfr3oHQtDdExECDlvbus/rNwMvHNfFKtaJZYFLhvlxzkJVJx/D1MvPykPZKfkSkcgSGQafb7YdhwNHdkLTYnhDt/dXeYrTjZ/vhYLJv1RF8KiFq0ML+5wYtIaCRZpxJqZQAyQUdycrnlZ+2AvB4/1ZE1vdzcUQiUiuYTBDcwn50vw9sNkj5095ClLbF3l2WvgvyM+xrEZ3YB7sWOD/DEmC/v0HLUwnSqcSoQXONLarllADJBb34wxYyThbSPjyQu3tFuTocEamtPDwgrLP9KGYY9nWH0nfaE6Kju+x/PrrTvj5RQZZ9Ftqh9Wc8zARBkacSrDNajQLD1GpUCygBkvNatC2N7/84hIcJpt7SEU+zVnMVkWrEZAL/EPsR1dv5WlE+HEuyJ0PpO08lRzvsf847ARn77cfuhc73WfztLURntRq1AItawGsKJUByTjn5RTw3ZxMA914RTfvwIBdHJCJSBp7eENLGfpzOMCD36GmtRjvtXWlHd9oTpoJsOPyH/ThTYMRprUYtS7rXAsO13YebUQIk5/T2/B0knzhJRD1fHru2lavDERGpGCYT1Am2H017Ol8rKrB3nTlajU79N30nnDwGmQftx57Fzvd5+Z3RanSqW61BC/D2r6qaSRkoAZJSbTyYwb+XJwHw8pD2+Fn0oyIitYCnBRq2sh9nyj12jlajPVCYCykb7ceZAsKcu9KKW42CItVq5EL6VpOzFFltjP/6T2wG3NQ5jL6tQ1wdkoiI6/nVhyax9uN01kI4vu+01qLTBmPnpkPWIfuRtMT5Pk/fU61GLc6ewu8dUHX1qqWUAMlZ/r08ic2HMgny9eL5G9q6OhwRkerN7FUyXb/1QOdrJ4/bW4ocrUanBmMf3Q1FJyF1k/04k3+jM1qNTg3CrtsEPMxVU68aTgmQODlwLJe35tuXon920GUE+3u7OCIRETfmWw8iu9uP01mL7OsWFbcUnd5qlJMG2Sn2Y++vzveZvUtpNTrVreajiSploQRIHAzD4Nk5m8grtNGzWQNujYlwdUgiIjWT2fNUItMcWg1wvnbyhL2F6KxWo11gzbcvApm25exn1gmxJ0RnLvxYt6lajUqhBEgcvvvjEEt3HMHi6cGUWzpouwsREVfwrWvf6ywixvm8zQon9pfeapSdYm85ykmDfcuc7zNb7PulNTg1ff/0wdi+9aqsWtWNEiAB4HhOAS9+b/8/ikevaUl0cB0XRyQiIk48zFA/2n60vNb5Wl6m8yrYp7caFeXBkW3240x+waW0GrWytxqZa3aKULNrJxftlZ+2cjSngNahAdx/ZTNXhyMiImXhEwjhXe3H6Ww2yDhQMm3/9Cn8WYfss9T2p8P+35zv8/CyJ1pnroYd3NI+G64GUAIk/LYrndlrD2IywZRbOmDx1LoUIiI1gocH1GtqP1rEOV/Lzy6l1ehUclR08tS2ITtg+xnP9K1feqtRvSj7jDg3oQSolssrtPLMN/aFu+66vCkxTWtvf7CISK3i7X/25rJgbzXKTC691SjzoH1F7AO/24/TeXjak6BSW40aVLsNZpUA1XLvLdzJ3qO5NAr04ckBrV0djoiIuJqHB9SNtB/Nr3a+VpBzWqvRLudWo8KcknFHO352vs+3nvN6RsEtIaStfRaciygBqsW2Hs7koyV7AHjxpnYE+LhP06WIiLiApQ407mQ/TmcYkHnIee+04sQo44B9QciDq+xHsVbXwR2zqjb+0ygBqqWsNoMJX2+kyGZwXbtG9G/XyNUhiYiIuzKZICjcfjTr63ytIBeO7T671ejMJKqKKQGqpT77fR8bDpwgwNuTF25s5+pwRESkprL4QaMO9qMa0XSfWujQiZO8Nte+HsRTA9vQKMjHxRGJiIhULSVAtYxhGEz8djM5BVZimtbjzh5NXB2SiIhIlVMCVMvM3ZTCgq2peJlNJNzSAQ+P6jUtUUREpCooAapFMk4WMum7zQA82Kc5rUIDXByRiIiIa1SLBOj9998nKioKHx8fYmNjWbVq1XnLf/nll7Rp0wYfHx86dOjATz/95HQ9OzubsWPHEhERga+vL23btuXDDz+szCq4hdfmbiMtK59mwXV4qF8LV4cjIiLiMi5PgGbNmkV8fDyTJk1i3bp1dOrUiQEDBpCWllZq+d9++40RI0Zw7733sn79eoYMGcKQIUPYtGmTo0x8fDxz587ls88+Y+vWrYwbN46xY8fy3XffVVW1qp3Ve4/x35X7Aft2Fz5eZhdHJCIi4jomwzAMVwYQGxtL9+7dmT59OgA2m43IyEgefvhhxo8ff1b54cOHk5OTww8//OA4d/nll9O5c2dHK0/79u0ZPnw4zz//vKNMTEwMAwcO5OWXX75gTJmZmQQFBZGRkUFgYOClVtHl8ousDHp3GbvSshneLZJXh3V0dUgiIiIVrizf3y5tASooKGDt2rXExZVs0Obh4UFcXBwrVqwo9Z4VK1Y4lQcYMGCAU/levXrx3XffkZycjGEYLFq0iB07dtC/f/9Sn5mfn09mZqbTUZN8tGQPu9KyCfb35pnrL3N1OCIiIi7n0gQoPT0dq9VKaGio0/nQ0FBSUlJKvSclJeWC5d977z3atm1LREQEFouF6667jvfff5+rrrqq1GcmJCQQFBTkOCIjIy+xZtXHrrRspi/cBcCkwW0J8tN2FyIiIi4fA1QZ3nvvPX7//Xe+++471q5dy5tvvsmYMWNYsGBBqeUnTJhARkaG4zhw4EAVR1w5bDaDZ77eSIHVRr/WDbmhY2NXhyQiIlItuHQrjODgYMxmM6mpqU7nU1NTadSo9L2pGjVqdN7yJ0+e5JlnnuGbb75h0KBBAHTs2JENGzbwxhtvnNV9BuDt7Y23t3dFVKla+WLNAVbtPYavl5mXhrTHZNKaPyIiIuDiFiCLxUJMTAyJiYmOczabjcTERHr27FnqPT179nQqDzB//nxH+cLCQgoLC/HwcK6a2WzGZrNVcA2qr7SsPKb8tBWAx/u3IqKen4sjEhERqT5cvhlqfHw8o0aNolu3bvTo0YNp06aRk5PD6NGjARg5ciTh4eEkJCQA8Oijj9KnTx/efPNNBg0axMyZM1mzZg0ff/wxAIGBgfTp04cnn3wSX19fmjZtypIlS/j000956623XFbPqvbi91vIzCuiY0QQo3tHuzocERGRasXlCdDw4cM5cuQIEydOJCUlhc6dOzN37lzHQOf9+/c7teb06tWLzz//nOeee45nnnmGli1bMmfOHNq3b+8oM3PmTCZMmMCdd97JsWPHaNq0Ka+88gp/+9vfqrx+rrBwWyo//HkYs4eJKTd3wKztLkRERJy4fB2g6sid1wHKyS/i2reWcCgjj79e1YwJmvYuIiK1hNusAyQV781fdnAoI4/I+r48GtfS1eGIiIhUS0qAapA/Dpxgxm9JALwypAN+Fpf3cIqIiFRLSoBqiEKrjfFfb8RmwM1dwrmqVUNXhyQiIlJtKQGqIf61LImthzOp6+fFc4M07kdEROR8lADVAPuO5jBtwQ4AnhvUlgb+NW9RRxERkYqkBMjNGYbBc3M2kVdoo1fzBgztGu7qkERERKo9JUBubs6GZH7dmY63pwdTbu6g7S5EREQughIgN3Ysp4CXfrBvd/FoXEuiguu4OCIRERH3oATIjb384xaO5RTQplEA91/ZzNXhiIiIuA0lQG5q2c50vl6XjMkECbd0wMusj1JERORi6VvTDZ0ssPLMNxsBGNUzii5N6rk4IhEREfeiBMgNvbtwJ/uP5dI4yIcnBrR2dTgiIiJuRwmQm9l6OJOPl+4B4KWb2uPvre0uREREykoJkBux2gzGf/UnVpvB9R0aEdc21NUhiYiIuCUlQG7k0xV7+eNgBgE+nrwwuJ2rwxEREXFbSoDcRPKJk7w+bzsA4we2ISTQx8URiYiIuC8lQG7AMAwmztlEboGV7lH1GNG9iatDEhERcWtKgNzAz5tSSNyWhpfZRMItHfDw0HYXIiIil0IJUDWXkVvIpO82A/BQ3xa0CAlwcUQiIiLuTwlQNTd17jaOZOXTrGEdHurX3NXhiIiI1AhKgKqxVUnH+N+q/QAk3NwBb0+ziyMSERGpGZQAVVP5RVYmfP0nACN6RBLbrIGLIxIREak5lABVUx8s3s3uIzk0DPBm/MDLXB2OiIhIjaIEqBralZbF3xftBuCFwe0I8vVycUQiIiI1ixKgasZmM5jw9UYKrDauaRPC9R0auTokERGRGkcJUDUzc/UBVu89jp/FzItD2mMyac0fERGRiqYEqBpJy8wj4eetADzRvzXhdX1dHJGIiEjNpASoGpn8/Ray8oroFBHEqF5Rrg5HRESkxlICVE0s2JLKjxsPY/YwkXBLR8za7kJERKTSKAGqBrLzi3j+200A3H9lM9qGBbo4IhERkZqtWiRA77//PlFRUfj4+BAbG8uqVavOW/7LL7+kTZs2+Pj40KFDB3766aezymzdupUbb7yRoKAg6tSpQ/fu3dm/f39lVeGSvDFvO4cz8mhS349Hr2np6nBERERqPJcnQLNmzSI+Pp5Jkyaxbt06OnXqxIABA0hLSyu1/G+//caIESO49957Wb9+PUOGDGHIkCFs2rTJUWb37t1cccUVtGnThsWLF/Pnn3/y/PPP4+PjU1XVumgbDpzgPyv2AvDKze3xtWi7CxERkcpmMgzDcGUAsbGxdO/enenTpwNgs9mIjIzk4YcfZvz48WeVHz58ODk5Ofzwww+Oc5dffjmdO3fmww8/BOD222/Hy8uL//u//ytXTJmZmQQFBZGRkUFgYOV1RxVabQx+bxnbUrK4pUs4bw3vXGmvJSIiUtOV5fvbpS1ABQUFrF27lri4OMc5Dw8P4uLiWLFiRan3rFixwqk8wIABAxzlbTYbP/74I61atWLAgAGEhIQQGxvLnDlzzhlHfn4+mZmZTkdV+OevSWxLyaKenxfP3dC2Sl5TREREXJwApaenY7VaCQ0NdTofGhpKSkpKqfekpKSct3xaWhrZ2dlMnTqV6667jl9++YWbb76ZW265hSVLlpT6zISEBIKCghxHZGRkBdTu/Pam5zBtwQ4Anr+hLfXrWCr9NUVERMTO5WOAKprNZgPgpptu4rHHHqNz586MHz+eG264wdFFdqYJEyaQkZHhOA4cOFCpMRqGwbNzNpJfZOOKFsHc3CW8Ul9PREREnHm68sWDg4Mxm82kpqY6nU9NTaVRo9L3wGrUqNF5ywcHB+Pp6Unbts5dSpdddhnLli0r9Zne3t54e3uXtxpl9vW6ZJbvOoq3pwev3KztLkRERKqaS1uALBYLMTExJCYmOs7ZbDYSExPp2bNnqff07NnTqTzA/PnzHeUtFgvdu3dn+/btTmV27NhB06ZNK7gGZXc0O5+Xf9wCwLi4VjRtUMfFEYmIiNQ+Lm0BAoiPj2fUqFF069aNHj16MG3aNHJychg9ejQAI0eOJDw8nISEBAAeffRR+vTpw5tvvsmgQYOYOXMma9as4eOPP3Y888knn2T48OFcddVV9OvXj7lz5/L999+zePFiV1TRySs/buV4biGXNQ7kviujXR2OiIhIreTyBGj48OEcOXKEiRMnkpKSQufOnZk7d65joPP+/fvx8ChpqOrVqxeff/45zz33HM888wwtW7Zkzpw5tG/f3lHm5ptv5sMPPyQhIYFHHnmE1q1b89VXX3HFFVdUef1Ot3THEb5en4zJBFNv6YCXucYNwRIREXELLl8HqDqqrHWAfvjzEBO+3siwmAgmDW5XYc8VERGRsn1/u7wFqDa5oWMYPaLq4+ett11ERMSV9E1cxUICq992HCIiIrWNBqGIiIhIraMESERERGodJUAiIiJS6ygBEhERkVpHCZCIiIjUOkqAREREpNZRAiQiIiK1jhIgERERqXWUAImIiEitowRIREREah0lQCIiIlLrKAESERGRWkcJkIiIiNQ62g2+FIZhAJCZmeniSERERORiFX9vF3+Pn48SoFJkZWUBEBkZ6eJIREREpKyysrIICgo6bxmTcTFpUi1js9k4dOgQAQEBmEymCn12ZmYmkZGRHDhwgMDAwAp9dnWg+rm/ml7Hml4/qPl1VP3cX2XV0TAMsrKyCAsLw8Pj/KN81AJUCg8PDyIiIir1NQIDA2vsDzaofjVBTa9jTa8f1Pw6qn7urzLqeKGWn2IaBC0iIiK1jhIgERERqXWUAFUxb29vJk2ahLe3t6tDqRSqn/ur6XWs6fWDml9H1c/9VYc6ahC0iIiI1DpqARIREZFaRwmQiIiI1DpKgERERKTWUQIkIiIitY4SoErw/vvvExUVhY+PD7Gxsaxateq85b/88kvatGmDj48PHTp04KeffqqiSMunLPWbMWMGJpPJ6fDx8anCaMtm6dKlDB48mLCwMEwmE3PmzLngPYsXL6Zr1654e3vTokULZsyYUelxlldZ67d48eKzPj+TyURKSkrVBFxGCQkJdO/enYCAAEJCQhgyZAjbt2+/4H3u9DtYnjq60+/hBx98QMeOHR0L5PXs2ZOff/75vPe40+dX1vq502dXmqlTp2IymRg3btx5y7niM1QCVMFmzZpFfHw8kyZNYt26dXTq1IkBAwaQlpZWavnffvuNESNGcO+997J+/XqGDBnCkCFD2LRpUxVHfnHKWj+wr/R5+PBhx7Fv374qjLhscnJy6NSpE++///5FlU9KSmLQoEH069ePDRs2MG7cOO677z7mzZtXyZGWT1nrV2z79u1On2FISEglRXhplixZwpgxY/j999+ZP38+hYWF9O/fn5ycnHPe426/g+WpI7jP72FERARTp05l7dq1rFmzhquvvpqbbrqJzZs3l1re3T6/stYP3OezO9Pq1av56KOP6Nix43nLuewzNKRC9ejRwxgzZozj71ar1QgLCzMSEhJKLX/bbbcZgwYNcjoXGxtr/PWvf63UOMurrPX75JNPjKCgoCqKrmIBxjfffHPeMk899ZTRrl07p3PDhw83BgwYUImRVYyLqd+iRYsMwDh+/HiVxFTR0tLSDMBYsmTJOcu42+/gmS6mju78e2gYhlGvXj3jn//8Z6nX3P3zM4zz189dP7usrCyjZcuWxvz5840+ffoYjz766DnLuuozVAtQBSooKGDt2rXExcU5znl4eBAXF8eKFStKvWfFihVO5QEGDBhwzvKuVJ76AWRnZ9O0aVMiIyMv+H867sadPr9L0blzZxo3bsy1117L8uXLXR3ORcvIyACgfv365yzj7p/hxdQR3PP30Gq1MnPmTHJycujZs2epZdz587uY+oF7fnZjxoxh0KBBZ302pXHVZ6gEqAKlp6djtVoJDQ11Oh8aGnrOMRMpKSllKu9K5alf69at+fe//823337LZ599hs1mo1evXhw8eLAqQq505/r8MjMzOXnypIuiqjiNGzfmww8/5KuvvuKrr74iMjKSvn37sm7dOleHdkE2m41x48bRu3dv2rdvf85y7vQ7eKaLraO7/R5u3LgRf39/vL29+dvf/sY333xD27ZtSy3rjp9fWernbp8dwMyZM1m3bh0JCQkXVd5Vn6F2g5dK1bNnT6f/s+nVqxeXXXYZH330ES+99JILI5OL0bp1a1q3bu34e69evdi9ezdvv/02//d//+fCyC5szJgxbNq0iWXLlrk6lEpzsXV0t9/D1q1bs2HDBjIyMpg9ezajRo1iyZIl50wS3E1Z6udun92BAwd49NFHmT9/frUfrK0EqAIFBwdjNptJTU11Op+amkqjRo1KvadRo0ZlKu9K5anfmby8vOjSpQu7du2qjBCr3Lk+v8DAQHx9fV0UVeXq0aNHtU8qxo4dyw8//MDSpUuJiIg4b1l3+h08XVnqeKbq/ntosVho0aIFADExMaxevZp33nmHjz766Kyy7vj5laV+Z6run93atWtJS0uja9eujnNWq5WlS5cyffp08vPzMZvNTve46jNUF1gFslgsxMTEkJiY6Dhns9lITEw8Z/9uz549ncoDzJ8//7z9wa5SnvqdyWq1snHjRho3blxZYVYpd/r8KsqGDRuq7ednGAZjx47lm2++YeHChURHR1/wHnf7DMtTxzO52++hzWYjPz+/1Gvu9vmV5nz1O1N1/+yuueYaNm7cyIYNGxxHt27duPPOO9mwYcNZyQ+48DOs1CHWtdDMmTMNb29vY8aMGcaWLVuMBx54wKhbt66RkpJiGIZh3HXXXcb48eMd5ZcvX254enoab7zxhrF161Zj0qRJhpeXl7Fx40ZXVeG8ylq/yZMnG/PmzTN2795trF271rj99tsNHx8fY/Pmza6qwnllZWUZ69evN9avX28AxltvvWWsX7/e2Ldvn2EYhjF+/HjjrrvucpTfs2eP4efnZzz55JPG1q1bjffff98wm83G3LlzXVWF8ypr/d5++21jzpw5xs6dO42NGzcajz76qOHh4WEsWLDAVVU4rwcffNAICgoyFi9ebBw+fNhx5ObmOsq4++9geeroTr+H48ePN5YsWWIkJSUZf/75pzF+/HjDZDIZv/zyi2EY7v/5lbV+7vTZncuZs8Cqy2eoBKgSvPfee0aTJk0Mi8Vi9OjRw/j9998d1/r06WOMGjXKqfwXX3xhtGrVyrBYLEa7du2MH3/8sYojLpuy1G/cuHGOsqGhocb1119vrFu3zgVRX5ziad9nHsV1GjVqlNGnT5+z7uncubNhsViMZs2aGZ988kmVx32xylq/V1991WjevLnh4+Nj1K9f3+jbt6+xcOFC1wR/EUqrG+D0mbj772B56uhOv4f33HOP0bRpU8NisRgNGzY0rrnmGkdyYBju//mVtX7u9Nmdy5kJUHX5DE2GYRiV28YkIiIiUr1oDJCIiIjUOkqAREREpNZRAiQiIiK1jhIgERERqXWUAImIiEitowRIREREah0lQCIiIlLrKAESEbkIixcvxmQyceLECVeHIiIVQAmQiIiI1DpKgERERKTWUQIkIm7BZrORkJBAdHQ0vr6+dOrUidmzZwMl3VM//vgjHTt2xMfHh8svv5xNmzY5PeOrr76iXbt2eHt7ExUVxZtvvul0PT8/n6effprIyEi8vb1p0aIF//rXv5zKrF27lm7duuHn50evXr3Yvn175VZcRCqFEiARcQsJCQl8+umnfPjhh2zevJnHHnuMv/zlLyxZssRR5sknn+TNN99k9erVNGzYkMGDB1NYWAjYE5fbbruN22+/nY0bN/LCCy/w/PPPM2PGDMf9I0eO5H//+x/vvvsuW7du5aOPPsLf398pjmeffZY333yTNWvW4OnpyT333FMl9ReRiqXNUEWk2svPz6d+/fosWLCAnj17Os7fd9995Obm8sADD9CvXz9mzpzJ8OHDATh27BgRERHMmDGD2267jTvvvJMjR47wyy+/OO5/6qmn+PHHH9m8eTM7duygdevWzJ8/n7i4uLNiWLx4Mf369WPBggVcc801APz0008MGjSIkydP4uPjU8nvgohUJLUAiUi1t2vXLnJzc7n22mvx9/d3HJ9++im7d+92lDs9Oapfvz6tW7dm69atAGzdupXevXs7Pbd3797s3LkTq9XKhg0bMJvN9OnT57yxdOzY0fHnxo0bA5CWlnbJdRSRquXp6gBERC4kOzsbgB9//JHw8HCna97e3k5JUHn5+vpeVDkvLy/Hn00mE2AfnyQi7kUtQCJS7bVt2xZvb2/2799PixYtnI7IyEhHud9//93x5+PHj7Njxw4uu+wyAC677DKWL1/u9Nzly5fTqlUrzGYzHTp0wGazOY0pEpGaSy1AIlLtBQQE8MQTT/DYY49hs9m44ooryMjIYPny5QQGBtK0aVMAXnzxRRo0aEBoaCjPPvsswcHBDBkyBIDHH3+c7t2789JLLzF8+HBWrFjB9OnT+fvf/w5AVFQUo0aN4p577uHdd9+lU6dO7Nu3j7S0NG677TZXVV1EKokSIBFxCy+99BINGzYkISGBPXv2ULduXbp27cozzzzj6IKaOnUqjz76KDt37qRz5858//33WCwWALp27coXX3zBxIkTeemll2jcuDEvvvgid999t+M1PvjgA5555hkeeughjh49SpMmTXjmmWdcUV0RqWSaBSYibq94htbx48epW7euq8MRETegMUAiIiJS6ygBEhERkVpHXWAiIiJS66gFSERERGodJUAiIiJS6ygBEhERkVpHCZCIiIjUOkqAREREpNZRAiQiIiK1jhIgERERqXWUAImIiEitowRIREREap3/B8rsZo3ViDsKAAAAAElFTkSuQmCC\n"
1111 | },
1112 | "metadata": {}
1113 | }
1114 | ]
1115 | },
1116 | {
1117 | "cell_type": "markdown",
1118 | "metadata": {
1119 | "id": "X25xwNezOn3v"
1120 | },
1121 | "source": [
1122 | "As we can see, the model is `overfitted` on the training dataset. It works well for training data, simply because it memorized them. However, the performance on training data is not important. The model should be able to `generalize` to new unseen data.\n",
1123 | "\n",
1124 | "The overfitting phenomenon is especially strong when models have many parameters. It can be mediated by `model regularization` and use of user and movie features that help the model generalize better to unseen data.\n",
1125 | "\n",
1126 | "Moreover, The model is re-recommending some of users' already watched movies. These known-positive watches can crowd out test movies out of top K recommendations and reduce the model accuracy for test data. This can be tackled by excluding previously seen movies from test recommendations in a third stage. This approach is relatively common in the recommender systems literature, but we don't follow it in these tutorials. If not recommending past watches is important, we should expect appropriately specified models to learn this behaviour automatically from past user history and contextual information. Additionally, it is often appropriate to recommend the same item multiple times (say, an evergreen TV series or a regularly purchased item)."
1127 | ]
1128 | },
1129 | {
1130 | "cell_type": "markdown",
1131 | "metadata": {
1132 | "id": "vu1I9SGxXIGy"
1133 | },
1134 | "source": [
1135 | "### Making predictions\n",
1136 | "To make recommendations, at first, we have to find the representation embedding vectors for all candidates in the corpus and `index` them all for future retrieval.\n",
1137 | "\n",
1138 | "Then, we can get a `query`, pass it through the query tower and find its representation embedding vector.\n",
1139 | "\n",
1140 | "Finally, we can calculate the affinity score between the query and all the candidates, sort them and find `k-nearest` candidates to the query. It is called a `brute-force` search and TFRS exposes a layer named `tfrs.layers.factorized_top_k.BruteForce` to do this."
1141 | ]
1142 | },
1143 | {
1144 | "cell_type": "code",
1145 | "metadata": {
1146 | "id": "_OqB4kwwS1mR",
1147 | "colab": {
1148 | "base_uri": "https://localhost:8080/"
1149 | },
1150 | "outputId": "38e0ed4a-d4db-4c52-c5f4-4a1f7e6ea979"
1151 | },
1152 | "source": [
1153 | "brute_force_layer = tfrs.layers.factorized_top_k.BruteForce(\n",
1154 | " movielens_retrieval_model.query_model\n",
1155 | ")\n",
1156 | "\n",
1157 | "brute_force_layer.index_from_dataset(\n",
1158 | " tf.data.Dataset.zip(\n",
1159 | " (\n",
1160 | " candidates_corpus_dataset.batch(100),\n",
1161 | " candidates_corpus_dataset.batch(100).map(\n",
1162 | " movielens_retrieval_model.candidate_model\n",
1163 | " )\n",
1164 | " )\n",
1165 | " )\n",
1166 | ")"
1167 | ],
1168 | "execution_count": 19,
1169 | "outputs": [
1170 | {
1171 | "output_type": "execute_result",
1172 | "data": {
1173 | "text/plain": [
1174 | ""
1175 | ]
1176 | },
1177 | "metadata": {},
1178 | "execution_count": 19
1179 | }
1180 | ]
1181 | },
1182 | {
1183 | "cell_type": "code",
1184 | "metadata": {
1185 | "id": "ccdAoIjAS0bC",
1186 | "colab": {
1187 | "base_uri": "https://localhost:8080/"
1188 | },
1189 | "outputId": "f6637372-b3c6-41a2-ecf5-c95a08aa6b1a"
1190 | },
1191 | "source": [
1192 | "user_id = '42'\n",
1193 | "afinity_scores, movie_ids = brute_force_layer(\n",
1194 | " tf.constant([user_id])\n",
1195 | ")\n",
1196 | "\n",
1197 | "print(f\"Recommendations for user {user_id} using BruteForce: {movie_ids[0, :5]}\")"
1198 | ],
1199 | "execution_count": 20,
1200 | "outputs": [
1201 | {
1202 | "output_type": "stream",
1203 | "name": "stdout",
1204 | "text": [
1205 | "Recommendations for user 42 using BruteForce: [b'2771' b'3142' b'3695' b'3120' b'4207']\n"
1206 | ]
1207 | }
1208 | ]
1209 | },
1210 | {
1211 | "cell_type": "markdown",
1212 | "metadata": {
1213 | "id": "OVlilfUEYPMa"
1214 | },
1215 | "source": [
1216 | "Performing a brute-force search in a large corpus can be too slow and impractical in a production environment. In practice we can speed this up by using an `Approximate Nearest Neighbor` search instead of the brute-force search. Approximate Nearest Neighbor (`ANN`) search can greatly outperform brute-force search speed while sacrificing little in terms of accuracy.\n",
1217 | "\n",
1218 | "We can use Google [ScaNN](https://ai.googleblog.com/2020/07/announcing-scann-efficient-vector.html), Facebook [Faiss](https://engineering.fb.com/2017/03/29/data-infrastructure/faiss-a-library-for-efficient-similarity-search/), Spotify [Annoy](https://github.com/spotify/annoy), or other approximate vector similarity search libraries to build an `approximate retrieval index`. However, TFRS uses `ScaNN` library and exposes the layer `tfrs.layers.factorized_top_k.ScaNN` letting us make the model really end-to-end.\n",
1219 | "\n",
1220 | "Note: to use `tfrs.layers.factorized_top_k.ScaNN` layer, `ScaNN` library should be installed using `pip install scann` before importing the `TFRS` library.\n"
1221 | ]
1222 | },
1223 | {
1224 | "cell_type": "code",
1225 | "metadata": {
1226 | "colab": {
1227 | "base_uri": "https://localhost:8080/"
1228 | },
1229 | "id": "2_c77jsdiPp9",
1230 | "outputId": "4b775ddb-a1ba-47b0-d396-571a82896834"
1231 | },
1232 | "source": [
1233 | "scann_layer = tfrs.layers.factorized_top_k.ScaNN(\n",
1234 | " movielens_retrieval_model.query_model\n",
1235 | ")\n",
1236 | "\n",
1237 | "scann_layer.index_from_dataset(\n",
1238 | " tf.data.Dataset.zip(\n",
1239 | " (\n",
1240 | " candidates_corpus_dataset.batch(100),\n",
1241 | " candidates_corpus_dataset.batch(100).map(\n",
1242 | " movielens_retrieval_model.candidate_model\n",
1243 | " )\n",
1244 | " )\n",
1245 | " )\n",
1246 | ")\n",
1247 | "\n",
1248 | "user_id = '42'\n",
1249 | "afinity_scores, movie_ids = scann_layer(\n",
1250 | " tf.constant([user_id])\n",
1251 | ")\n",
1252 | "\n",
1253 | "print(f\"Recommendations for user {user_id} using ScaNN: {movie_ids[0, :5]}\")"
1254 | ],
1255 | "execution_count": 21,
1256 | "outputs": [
1257 | {
1258 | "output_type": "stream",
1259 | "name": "stdout",
1260 | "text": [
1261 | "Recommendations for user 42 using ScaNN: [b'3142' b'2574' b'3695' b'4565' b'4442']\n"
1262 | ]
1263 | }
1264 | ]
1265 | },
1266 | {
1267 | "cell_type": "markdown",
1268 | "metadata": {
1269 | "id": "iumhoS_lucaV"
1270 | },
1271 | "source": [
1272 | "### Save the model to use in future\n",
1273 | "To make it possible to serve the Keras model using [TensorFlow Serving](https://www.tensorflow.org/tfx/guide/serving), export it to a `SavedModel` format."
1274 | ]
1275 | },
1276 | {
1277 | "cell_type": "code",
1278 | "metadata": {
1279 | "colab": {
1280 | "base_uri": "https://localhost:8080/"
1281 | },
1282 | "id": "Fowdeb85j6OS",
1283 | "outputId": "e9426126-6287-4bba-f8b6-e07ef5d53c16"
1284 | },
1285 | "source": [
1286 | "import os\n",
1287 | "import tempfile\n",
1288 | "\n",
1289 | "with tempfile.TemporaryDirectory() as tmp_dir:\n",
1290 | " retrieval_model_path = os.path.join(tmp_dir, \"retrieval_model\")\n",
1291 | "\n",
1292 | "\n",
1293 | "scann_layer.save(\n",
1294 | " retrieval_model_path,\n",
1295 | " options=tf.saved_model.SaveOptions(namespace_whitelist=[\"Scann\"])\n",
1296 | ")"
1297 | ],
1298 | "execution_count": 22,
1299 | "outputs": [
1300 | {
1301 | "output_type": "stream",
1302 | "name": "stderr",
1303 | "text": [
1304 | "WARNING:tensorflow:Model's `__init__()` arguments contain non-serializable objects. Please implement a `get_config()` method in the subclassed Model for proper saving and loading. Defaulting to empty config.\n",
1305 | "WARNING:tensorflow:Model's `__init__()` arguments contain non-serializable objects. Please implement a `get_config()` method in the subclassed Model for proper saving and loading. Defaulting to empty config.\n",
1306 | "WARNING:tensorflow:Model's `__init__()` arguments contain non-serializable objects. Please implement a `get_config()` method in the subclassed Model for proper saving and loading. Defaulting to empty config.\n",
1307 | "WARNING:tensorflow:Model's `__init__()` arguments contain non-serializable objects. Please implement a `get_config()` method in the subclassed Model for proper saving and loading. Defaulting to empty config.\n"
1308 | ]
1309 | }
1310 | ]
1311 | },
1312 | {
1313 | "cell_type": "code",
1314 | "metadata": {
1315 | "id": "UNET0awMnJ-X",
1316 | "outputId": "ce2eaf05-8e10-44ed-c913-11d1c6e4dc1d",
1317 | "colab": {
1318 | "base_uri": "https://localhost:8080/"
1319 | }
1320 | },
1321 | "source": [
1322 | "# Reload the saved model to confirm that it works correctly\n",
1323 | "reloaded_model = tf.keras.models.load_model(retrieval_model_path)\n",
1324 | "afinity_scores, movie_ids = reloaded_model(\n",
1325 | " tf.constant([user_id])\n",
1326 | ")\n",
1327 | "\n",
1328 | "print(f\"Recommendations for user {user_id} using reloaded saved model: {movie_ids[0, :5]}\")"
1329 | ],
1330 | "execution_count": 23,
1331 | "outputs": [
1332 | {
1333 | "output_type": "stream",
1334 | "name": "stderr",
1335 | "text": [
1336 | "WARNING:tensorflow:No training configuration found in save file, so the model was *not* compiled. Compile it manually.\n"
1337 | ]
1338 | },
1339 | {
1340 | "output_type": "stream",
1341 | "name": "stdout",
1342 | "text": [
1343 | "Recommendations for user 42 using reloaded saved model: [b'3142' b'2574' b'3695' b'4565' b'4442']\n"
1344 | ]
1345 | }
1346 | ]
1347 | },
1348 | {
1349 | "cell_type": "markdown",
1350 | "metadata": {
1351 | "id": "tnRXsM-3xsGU"
1352 | },
1353 | "source": [
1354 | "If the saved model works correctly, back it up from the temporary directory to your Google Drive, download it from there, copy to your server (`retrieval_model/1/saved_model.pb`) and serve it."
1355 | ]
1356 | },
1357 | {
1358 | "cell_type": "markdown",
1359 | "metadata": {
1360 | "id": "FZznewEYzYh1"
1361 | },
1362 | "source": [
1363 | "### Serve the saved model using [TensorFlow Serving](https://www.tensorflow.org/tfx/guide/serving)\n",
1364 | "\n",
1365 | "We can use TensorFlow Serving Docker image to run a Docker container serving the saved model. Note that `tensorflow/serving` image doesn't support `ScaNN` layer, so it's important to use `google/tf-serving-scann` custom image instead.\n",
1366 | "\n",
1367 | "TF Serving uses the port 8500 for gRPC and the port 8501 for REST APIs.\n",
1368 | "\n",
1369 | "```\n",
1370 | "docker run -p 8501:8501 \\\n",
1371 | "--mount type=bind,source=PATH_TO_RETRIEVAL_DIR/retrieval,target=/models/retrieval \\\n",
1372 | "-e MODEL_NAME=retrieval -t google/tf-serving-scann &\n",
1373 | "```\n",
1374 | "\n",
1375 | "Now we can send the `query` as an HTTP request to the server and get the recommended candidates.\n",
1376 | "```\n",
1377 | "curl --location --request POST 'http://SERVER_IP_ADDRESS:8501/v1/models/retrieval:predict' \\\n",
1378 | "--header 'Content-Type: application/json' \\\n",
1379 | "--data-raw '{\n",
1380 | " \"instances\":\n",
1381 | " [\n",
1382 | " \"42\"\n",
1383 | " ]\n",
1384 | "}'\n",
1385 | "```"
1386 | ]
1387 | },
1388 | {
1389 | "cell_type": "markdown",
1390 | "metadata": {
1391 | "id": "3Ez1GmXvB8L-"
1392 | },
1393 | "source": [
1394 | "## Build the Ranking (Scoring) task\n",
1395 | "\n",
1396 | "The `ranking model` takes the outputs of the retrieval model and fine-tunes them to select the best possible handful of recommendations.\n",
1397 | "\n",
1398 | " Its task is to narrow down the set of items the user may be interested in to a shortlist of likely candidates. So the traing set should be expressesing ***how much the users liked the movies they did watch***. This is a form of `explicit feedback` - given that a user watched a movie, we can tell roughly how much they liked by looking at the rating they have given.\n",
1399 | "```\n",
1400 | "[\n",
1401 | " (('user1', 'Star Wars'), 4.0),\n",
1402 | " (('user1', 'Harry Potter'), 5.0),\n",
1403 | " ...\n",
1404 | "]\n",
1405 | "```\n",
1406 | " This time we are going to predict `user_rating` value as the objective. So like the other `regression` problems we can use `MSE (Mean Squared Error)` as loss function and `RMSE (Root Mean Squared Error)` as an accuracy metric. The state of the art (`SOTA`) RMSE value for `MovieLens/100k` is equal to `0.909`.\n",
1407 | "\n",
1408 | "`tfrs.tasks.Ranking` layer gets the predicted ratings and the `ground truth` as input, calculates the metrics and returns the loss value.\n",
1409 | "\n",
1410 | "Ranking models do not face the same efficiency constraints as retrieval models do, and so we have a little bit more freedom in our choice of architectures. A model composed of multiple stacked `Dense` layers is a relatively common architecture for ranking tasks.\n",
1411 | "\n",
1412 | "In most cases, a ranking model can be substantially improved by using more features rather than just user and candidate identifiers."
1413 | ]
1414 | },
1415 | {
1416 | "cell_type": "code",
1417 | "metadata": {
1418 | "id": "TLwaxI00k1xX"
1419 | },
1420 | "source": [
1421 | "class RankingModel(tfrs.models.Model):\n",
1422 | " \"\"\"MovieLens ranking model\"\"\"\n",
1423 | "\n",
1424 | " def __init__(self, query_model, candidate_model):\n",
1425 | " super().__init__()\n",
1426 | "\n",
1427 | " self.query_model: tf.keras.Model = query_model\n",
1428 | " self.candidate_model: tf.keras.Model = candidate_model\n",
1429 | " self.rating_model = tf.keras.Sequential(\n",
1430 | " [\n",
1431 | " tf.keras.layers.Dense(256, activation='relu'),\n",
1432 | " tf.keras.layers.Dense(64, activation='relu'),\n",
1433 | " tf.keras.layers.Dense(1)\n",
1434 | " ]\n",
1435 | " )\n",
1436 | " self.ranking_task_layer: tf.keras.layers.Layer = tfrs.tasks.Ranking(\n",
1437 | " loss=tf.keras.losses.MeanSquaredError(),\n",
1438 | " metrics=[\n",
1439 | " tf.keras.metrics.RootMeanSquaredError()\n",
1440 | " ]\n",
1441 | " )\n",
1442 | "\n",
1443 | "\n",
1444 | " def compute_loss(self, features, training=False) -> tf.Tensor:\n",
1445 | " query_embeddings = self.query_model(features['user_id'])\n",
1446 | " candidate_embeddings = self.candidate_model(features[\"movie_id\"])\n",
1447 | " rating_predictions = self.rating_model(\n",
1448 | " tf.concat(\n",
1449 | " [query_embeddings, candidate_embeddings],\n",
1450 | " axis=1\n",
1451 | " )\n",
1452 | " # We could use `tf.keras.layers.Concatenate(axis=1)([x, y])`\n",
1453 | " )\n",
1454 | "\n",
1455 | " loss = self.ranking_task_layer(\n",
1456 | " predictions=rating_predictions,\n",
1457 | " labels=features[\"user_rating\"]\n",
1458 | " )\n",
1459 | " return loss"
1460 | ],
1461 | "execution_count": 24,
1462 | "outputs": []
1463 | },
1464 | {
1465 | "cell_type": "markdown",
1466 | "metadata": {
1467 | "id": "it4JTHYF1eh-"
1468 | },
1469 | "source": [
1470 | "### Fit the ranking model"
1471 | ]
1472 | },
1473 | {
1474 | "cell_type": "code",
1475 | "metadata": {
1476 | "id": "BijEncJG1iaW"
1477 | },
1478 | "source": [
1479 | "movielens_ranking_model = RankingModel(query_model, candidate_model)\n",
1480 | "\n",
1481 | "optimizer_step_size = 0.1\n",
1482 | "movielens_ranking_model.compile(\n",
1483 | " optimizer=tf.keras.optimizers.Adagrad(\n",
1484 | " learning_rate=optimizer_step_size\n",
1485 | " )\n",
1486 | ")"
1487 | ],
1488 | "execution_count": 25,
1489 | "outputs": []
1490 | },
1491 | {
1492 | "cell_type": "code",
1493 | "metadata": {
1494 | "colab": {
1495 | "base_uri": "https://localhost:8080/"
1496 | },
1497 | "id": "JXZykvH6-DvF",
1498 | "outputId": "cf398902-4e5e-4038-d2a3-2d5d3b2370f3"
1499 | },
1500 | "source": [
1501 | "ranking_ratings_trainset = ratings_trainset.shuffle(100_000).batch(8192).cache()\n",
1502 | "ranking_ratings_testset = ratings_testset.batch(4096).cache()\n",
1503 | "\n",
1504 | "history = movielens_ranking_model.fit(\n",
1505 | " ranking_ratings_trainset,\n",
1506 | " validation_data=ranking_ratings_testset,\n",
1507 | " validation_freq=1,\n",
1508 | " epochs=5\n",
1509 | ")"
1510 | ],
1511 | "execution_count": 26,
1512 | "outputs": [
1513 | {
1514 | "output_type": "stream",
1515 | "name": "stdout",
1516 | "text": [
1517 | "Epoch 1/5\n",
1518 | "10/10 [==============================] - 19s 1s/step - root_mean_squared_error: 2.4954 - loss: 5.6800 - regularization_loss: 0.0000e+00 - total_loss: 5.6800 - val_root_mean_squared_error: 0.9894 - val_loss: 0.9845 - val_regularization_loss: 0.0000e+00 - val_total_loss: 0.9845\n",
1519 | "Epoch 2/5\n",
1520 | "10/10 [==============================] - 1s 91ms/step - root_mean_squared_error: 0.9699 - loss: 0.9353 - regularization_loss: 0.0000e+00 - total_loss: 0.9353 - val_root_mean_squared_error: 0.9536 - val_loss: 0.9175 - val_regularization_loss: 0.0000e+00 - val_total_loss: 0.9175\n",
1521 | "Epoch 3/5\n",
1522 | "10/10 [==============================] - 1s 88ms/step - root_mean_squared_error: 0.9457 - loss: 0.8907 - regularization_loss: 0.0000e+00 - total_loss: 0.8907 - val_root_mean_squared_error: 0.9412 - val_loss: 0.8968 - val_regularization_loss: 0.0000e+00 - val_total_loss: 0.8968\n",
1523 | "Epoch 4/5\n",
1524 | "10/10 [==============================] - 1s 81ms/step - root_mean_squared_error: 0.9343 - loss: 0.8695 - regularization_loss: 0.0000e+00 - total_loss: 0.8695 - val_root_mean_squared_error: 0.9334 - val_loss: 0.8839 - val_regularization_loss: 0.0000e+00 - val_total_loss: 0.8839\n",
1525 | "Epoch 5/5\n",
1526 | "10/10 [==============================] - 1s 76ms/step - root_mean_squared_error: 0.9266 - loss: 0.8554 - regularization_loss: 0.0000e+00 - total_loss: 0.8554 - val_root_mean_squared_error: 0.9277 - val_loss: 0.8744 - val_regularization_loss: 0.0000e+00 - val_total_loss: 0.8744\n"
1527 | ]
1528 | }
1529 | ]
1530 | },
1531 | {
1532 | "cell_type": "code",
1533 | "metadata": {
1534 | "colab": {
1535 | "base_uri": "https://localhost:8080/",
1536 | "height": 472
1537 | },
1538 | "id": "1-uNjmoY-gsA",
1539 | "outputId": "8fbd9476-ab44-4a1d-c1ac-b4ed06252b08"
1540 | },
1541 | "source": [
1542 | "# Plot changes in model loss during training\n",
1543 | "plt.plot(history.history[\"loss\"])\n",
1544 | "plt.plot(history.history[\"val_loss\"])\n",
1545 | "plt.title(\"Model losses during training\")\n",
1546 | "plt.xlabel(\"epoch\")\n",
1547 | "plt.ylabel(\"loss\")\n",
1548 | "plt.legend([\"train\", \"test\"], loc=\"upper right\")\n",
1549 | "plt.show()"
1550 | ],
1551 | "execution_count": 27,
1552 | "outputs": [
1553 | {
1554 | "output_type": "display_data",
1555 | "data": {
1556 | "text/plain": [
1557 | ""
1558 | ],
1559 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAkgAAAHHCAYAAABEEKc/AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAAB8NUlEQVR4nO3dd3gU1f7H8fdm00kTUmiB0HtRhAgiRQJRFBXBAirFigJXig1Fsf3k6lWKgsrVK6jIFZVyVRSB0ARpBqLUEEIJLQkBkpAAaTu/PxYWlgQIIZvNhs/refYxzJ6Z+U7WkA/nnDljMgzDQERERERs3JxdgIiIiEh5o4AkIiIicgEFJBEREZELKCCJiIiIXEABSUREROQCCkgiIiIiF1BAEhEREbmAApKIiIjIBRSQRERERC6ggCTi4kwmE6+//voV77d3715MJhMzZsy4ZLvly5djMplYvnx5iepzZREREQwaNMhljutsr7/+OiaTqUT7zpgxA5PJxN69e0u3KJESUkASKQVn/3I3mUysWrWq0PuGYRAeHo7JZOLOO+90QoUicPLkSV5//fVrMuyKXCkFJJFS5O3tzaxZswptX7FiBQcOHMDLy8sJVUl5Ex8fz2effVbm5z158iRvvPGGwwLS2LFjOXXqVIn2feSRRzh16hS1a9cu5apESkYBSaQU9ezZk++//578/Hy77bNmzaJNmzZUrVrVSZWJsxmGYQsPXl5eeHh4OLmiy8vOzr6i9u7u7nh7e5foXGazGW9v7xIP0YmUNgUkkVLUr18/jh49yuLFi23bcnNz+eGHH+jfv3+R+2RnZzN69GjCw8Px8vKiUaNGvP/++xiGYdcuJyeHkSNHEhISgr+/P3fddRcHDhwo8pgHDx7k0UcfJSwsDC8vL5o1a8YXX3xRehcKfP/997Rp0wYfHx+Cg4N5+OGHOXjwoF2b5ORkBg8eTM2aNfHy8qJatWrcfffddvNM/vzzT6KjowkODsbHx4c6derw6KOP2h3HYrEwadIkmjVrhre3N2FhYTz11FMcP37crl1xjlUUwzB4++23qVmzJr6+vnTt2pWtW7cWanexOTZFzZ+JiIjgzjvv5LfffuPGG2/Ex8eHadOm2d47fw7S2f1Xr17NqFGjCAkJoVKlSvTu3ZsjR44U+l68/vrrVK9e3Vbrtm3bLjuvae/evYSEhADwxhtv2IaEz85fGzRoEH5+fiQmJtKzZ0/8/f156KGHAPj999+57777qFWrFl5eXoSHhzNy5MhCvUVFfX9MJhPDhg1j/vz5NG/e3Pb/48KFC4v9PVy1ahXt2rXD29ubunXr8tVXXxW6vr///pvOnTvj4+NDzZo1efvtt5k+fbrmNUmJuTu7AJGKJCIigvbt2/Pf//6X22+/HYBff/2VjIwMHnzwQT788EO79oZhcNddd7Fs2TIee+wxWrduzW+//cbzzz/PwYMHmThxoq3t448/zsyZM+nfvz8dOnRg6dKl3HHHHYVqSElJ4aabbrL9YgoJCeHXX3/lscceIzMzkxEjRlz1dc6YMYPBgwfTtm1bxo8fT0pKCpMnT2b16tVs2rSJoKAgAPr06cPWrVsZPnw4ERERpKamsnjxYpKSkmx/7tGjByEhIbz00ksEBQWxd+9e5s6da3e+p556ynbOf/zjH+zZs4cpU6awadMmVq9ejYeHR7GPVZTXXnuNt99+m549e9KzZ082btxIjx49yM3NvarvU3x8PP369eOpp57iiSeeoFGjRpdsP3z4cK677jrGjRvH3r17mTRpEsOGDWP27Nm2NmPGjOG9996jV69eREdH89dffxEdHc3p06cveeyQkBA++eQTnn76aXr37s29994LQMuWLW1t8vPziY6OpmPHjrz//vv4+voC1jB88uRJnn76aapUqcL69ev56KOPOHDgAN9///1lvw+rVq1i7ty5PPPMM/j7+/Phhx/Sp08fkpKSqFKlyiX33bVrF3379uWxxx5j4MCBfPHFFwwaNIg2bdrQrFkzwPoPgq5du2IymRgzZgyVKlXi888/15C2XB1DRK7a9OnTDcDYsGGDMWXKFMPf3984efKkYRiGcd999xldu3Y1DMMwateubdxxxx22/ebPn28Axttvv213vL59+xomk8nYtWuXYRiGERcXZwDGM888Y9euf//+BmCMGzfOtu2xxx4zqlWrZqSlpdm1ffDBB43AwEBbXXv27DEAY/r06Ze8tmXLlhmAsWzZMsMwDCM3N9cIDQ01mjdvbpw6dcrW7ueffzYA47XXXjMMwzCOHz9uAMa//vWvix573rx5tu/bxfz+++8GYHzzzTd22xcuXGi3vTjHKkpqaqrh6elp3HHHHYbFYrFtf/nllw3AGDhwoG3buHHjjKL+2jz7+e/Zs8e2rXbt2gZgLFy4sFD72rVr2x337P5RUVF2NYwcOdIwm81Genq6YRiGkZycbLi7uxv33HOP3fFef/31QrUW5ciRI4X+fzlr4MCBBmC89NJLhd47+//M+caPH2+YTCZj3759tm1FfX8Aw9PT0/b/smEYxl9//WUAxkcffVToe1DU93DlypW2bampqYaXl5cxevRo27bhw4cbJpPJ2LRpk23b0aNHjcqVKxc6pkhxaYhNpJTdf//9nDp1ip9//pkTJ07w888/X3R47ZdffsFsNvOPf/zDbvvo0aMxDINff/3V1g4o1O7C3iDDMJgzZw69evXCMAzS0tJsr+joaDIyMti4ceNVXd+ff/5JamoqzzzzjN18kzvuuIPGjRuzYMECAHx8fPD09GT58uWFhsLOOtvT9PPPP5OXl1dkm++//57AwEC6d+9udz1t2rTBz8+PZcuWFftYRVmyZAm5ubkMHz7cbnioNHra6tSpQ3R0dLHbP/nkk3Y13HLLLRQUFLBv3z4AYmJiyM/P55lnnrHbb/jw4Vdd61lPP/10oW0+Pj62r7Ozs0lLS6NDhw4YhsGmTZsue8yoqCjq1atn+3PLli0JCAhg9+7dl923adOm3HLLLbY/h4SE0KhRI7t9Fy5cSPv27WndurVtW+XKlW1DhCIloYAkUspCQkKIiopi1qxZzJ07l4KCAvr27Vtk23379lG9enX8/f3ttjdp0sT2/tn/urm52f2SAQoN2Rw5coT09HT+/e9/ExISYvcaPHgwAKmpqVd1fWdrKmq4qHHjxrb3vby8ePfdd/n1118JCwujU6dOvPfeeyQnJ9vad+7cmT59+vDGG28QHBzM3XffzfTp08nJybG1SUhIICMjg9DQ0ELXlJWVZbue4hzrUtfToEEDu+0hISFcd911JfgOnVOnTp0ral+rVi27P589/9mAebbW+vXr27WrXLnyVdcK1knWNWvWLLQ9KSmJQYMGUblyZfz8/AgJCaFz584AZGRkXPa4F14XWK/tYsH5Svfdt29foe8JFP4+iVwJzUEScYD+/fvzxBNPkJyczO23327r3XA0i8UCwMMPP8zAgQOLbHP+nBNHGzFiBL169WL+/Pn89ttvvPrqq4wfP56lS5dy/fXXYzKZ+OGHH1i7di0//fQTv/32G48++igffPABa9euxc/PD4vFQmhoKN98802R5zg78bg4x7paF7vDqqCgoMjt5/e8FIfZbC5yu3HBhH1H8fLyws3N/t/NBQUFdO/enWPHjvHiiy/SuHFjKlWqxMGDBxk0aJDt/7lLuZrrcvb3RK5dCkgiDtC7d2+eeuop1q5dazfB9kK1a9dmyZIlnDhxwq4XaceOHbb3z/7XYrGQmJho13MTHx9vd7yzd7gVFBQQFRVVmpdkV/PZc996661278XHxxdax6ZevXqMHj2a0aNHk5CQQOvWrfnggw+YOXOmrc1NN93ETTfdxP/93/8xa9YsHnroIb799lsef/xx6tWrx5IlS7j55puLFTgudaxLXU9CQgJ169a1bT9y5EihHo6zvTTp6el2ofdsz46jna11165ddr1TR48eLVZvTEluod+8eTM7d+7kyy+/ZMCAAbbt59+p6Wy1a9dm165dhbYXtU2kuDTEJuIAfn5+fPLJJ7z++uv06tXrou169uxJQUEBU6ZMsds+ceJETCaT7U64s/+98C64SZMm2f3ZbDbTp08f5syZw5YtWwqd78JbxkvixhtvJDQ0lE8//dRu+OrXX39l+/bttjvrTp48WejOqnr16uHv72/b7/jx44V6As7OIznb5v7776egoIC33nqrUC35+fmkp6cX+1hFiYqKwsPDg48++shu/wu/t2frB1i5cqVtW3Z2Nl9++eVFj1+aunXrhru7O5988ond9gv//7mYs3elnf2eFcfZHpzzvzeGYTB58uRiH8PRoqOjWbNmDXFxcbZtx44du2ivo0hxqAdJxEEuNsR1vl69etG1a1deeeUV9u7dS6tWrVi0aBH/+9//GDFihO0XcuvWrenXrx8ff/wxGRkZdOjQgZiYmCL/hfzPf/6TZcuWERkZyRNPPEHTpk05duwYGzduZMmSJRw7duyqrsvDw4N3332XwYMH07lzZ/r162e7zT8iIoKRI0cCsHPnTrp168b9999P06ZNcXd3Z968eaSkpPDggw8C8OWXX/Lxxx/Tu3dv6tWrx4kTJ/jss88ICAigZ8+egHVu0VNPPcX48eOJi4ujR48eeHh4kJCQwPfff8/kyZPp27dvsY5VlJCQEJ577jnGjx/PnXfeSc+ePdm0aRO//vorwcHBdm179OhBrVq1eOyxx3j++ecxm8188cUXhISEkJSUdFXf1+IICwvj2Wef5YMPPuCuu+7itttu46+//rLVerkeIh8fH5o2bcrs2bNp2LAhlStXpnnz5jRv3vyi+zRu3Jh69erx3HPPcfDgQQICApgzZ06xeqzKygsvvMDMmTPp3r07w4cPt93mX6tWLY4dO6bFJ6VEFJBEnMjNzY0ff/yR1157jdmzZzN9+nQiIiL417/+xejRo+3anv1F/M033zB//nxuvfVWFixYQHh4uF27sLAw1q9fz5tvvsncuXP5+OOPqVKlCs2aNePdd98tlboHDRqEr68v//znP3nxxRdtixq+++67tqGn8PBw+vXrR0xMDF9//TXu7u40btyY7777jj59+gDW8LN+/Xq+/fZbUlJSCAwMpF27dnzzzTd2Q0iffvopbdq0Ydq0abz88su4u7sTERHBww8/zM0333xFxyrK22+/jbe3N59++qktXC5atKjQOlMeHh7MmzePZ555hldffZWqVasyYsQIrrvuOtskeEd799138fX15bPPPmPJkiW0b9+eRYsW0bFjx2KtYv35558zfPhwRo4cSW5uLuPGjbtkQPLw8OCnn37iH//4B+PHj8fb25vevXszbNgwWrVqVZqXVmLh4eEsW7aMf/zjH7zzzjuEhIQwdOhQKlWqxD/+8Y8Sr+4t1zaToZluIiIuLT09neuuu463336bV155xdnllBsjRoxg2rRpZGVlXXSyt8jFaA6SiIgLKephsGfnS3Xp0qVsiylHLvy+HD16lK+//pqOHTsqHEmJaIhNRMSFzJ49mxkzZtCzZ0/8/PxYtWoV//3vf+nRo4dtuPFa1L59e7p06UKTJk1ISUnhP//5D5mZmbz66qvOLk1clAKSiIgLadmyJe7u7rz33ntkZmbaJm6//fbbzi7NqXr27MkPP/zAv//9b0wmEzfccAP/+c9/6NSpk7NLExelOUgiIiIiF9AcJBEREZELKCCJiIiIXEBzkErIYrFw6NAh/P39tQiZiIiIizAMgxMnTlC9evVCzx48nwJSCR06dKjQAn0iIiLiGvbv30/NmjUv+r4CUgmdfbDo/v37CQgIcHI1IiIiUhyZmZmEh4fbPSC8KApIJXR2WC0gIEABSURExMVcbnqMJmmLiIiIXEABSUREROQCCkgiIiIiF9AcJBERkXKmoKCAvLw8Z5fhkjw8PErlAcUKSCIiIuWEYRgkJyeTnp7u7FJcWlBQEFWrVr2qdQoVkERERMqJs+EoNDQUX19fLUR8hQzD4OTJk6SmpgJQrVq1Eh9LAUlERKQcKCgosIWjKlWqOLscl+Xj4wNAamoqoaGhJR5u0yRtERGRcuDsnCNfX18nV+L6zn4Pr2YelwKSiIhIOaJhtatXGt9DBSQRERGRCyggiYiISLkRERHBpEmTnF2GJmmLiIjI1enSpQutW7culWCzYcMGKlWqdPVFXSX1IJUzFovB8vhUDMNwdikiIiKlwjAM8vPzi9U2JCSkXExUV0AqRywWg96f/MGg6RtYmZDm7HJEREQua9CgQaxYsYLJkydjMpkwmUzMmDEDk8nEr7/+Sps2bfDy8mLVqlUkJiZy9913ExYWhp+fH23btmXJkiV2x7twiM1kMvH555/Tu3dvfH19adCgAT/++KPDr0sBqRxxczNxY+3rAJiwKF69SCIi1zjDMDiZm++UV3F/B02ePJn27dvzxBNPcPjwYQ4fPkx4eDgAL730Ev/85z/Zvn07LVu2JCsri549exITE8OmTZu47bbb6NWrF0lJSZc8xxtvvMH999/P33//Tc+ePXnooYc4duzYVX9/L0VzkMqZp7vUY9a6JP46kEHM9lSimoY5uyQREXGSU3kFNH3tN6ece9ub0fh6Xj4mBAYG4unpia+vL1WrVgVgx44dALz55pt0797d1rZy5cq0atXK9ue33nqLefPm8eOPPzJs2LCLnmPQoEH069cPgHfeeYcPP/yQ9evXc9ttt5Xo2opDPUjlTLCfF4NujgBgwuKdWCzqRRIREdd044032v05KyuL5557jiZNmhAUFISfnx/bt2+/bA9Sy5YtbV9XqlSJgIAA2+NEHEU9SOXQk7fU5es1+9h2OJOFW5Pp2aLkz5IRERHX5eNhZtub0U4799W68G605557jsWLF/P+++9Tv359fHx86Nu3L7m5uZc8joeHh92fTSYTFovlquu7FAWkcui6Sp482rEOH8YkMHHxTqKbVcXsppVVRUSuNSaTqVjDXM7m6elJQUHBZdutXr2aQYMG0bt3b8Dao7R3714HV1cyGmIrpx7rWIdAHw8SUrP4+e9Dzi5HRETkoiIiIli3bh179+4lLS3tor07DRo0YO7cucTFxfHXX3/Rv39/h/cElZQCUjkV6OPBk53qAjBpSQL5BeXzfyAREZHnnnsOs9lM06ZNCQkJueicogkTJnDdddfRoUMHevXqRXR0NDfccEMZV1s8JkP3kpdIZmYmgYGBZGRkEBAQ4JBzZOfkc8t7yziWnct7fVty/43hDjmPiIg43+nTp9mzZw916tTB29vb2eW4tEt9L4v7+1s9SOVYJS93hnS29iJ9GJNAbr56kURERMqCAlI598hNEYT4e3Hg+Cm++3O/s8sRERG5Jjg1IK1cuZJevXpRvXp1TCYT8+fPv+w+y5cv54YbbsDLy4v69eszY8aMQm2mTp1KREQE3t7eREZGsn79erv3T58+zdChQ6lSpQp+fn706dOHlJSUUrqq0uXjaWZol3oATFm6i9N5l79LQERERK6OUwNSdnY2rVq1YurUqcVqv2fPHu644w66du1KXFwcI0aM4PHHH+e3386tMjp79mxGjRrFuHHj2LhxI61atSI6OtpuQamRI0fy008/8f3337NixQoOHTrEvffeW+rXV1oebFeLaoHeJGee5r/rL72YloiIiFy9cjNJ22QyMW/ePO65556LtnnxxRdZsGABW7ZssW178MEHSU9PZ+HChQBERkbStm1bpkyZAoDFYiE8PJzhw4fz0ksvkZGRQUhICLNmzaJv376AdUn0Jk2asGbNGm666aZi1VsWk7TPN2tdEi/P20ywnxe/v9AVH8+rX8BLRETKD03SLj3X3CTtNWvWEBUVZbctOjqaNWvWAJCbm0tsbKxdGzc3N6KiomxtYmNjycvLs2vTuHFjatWqZWtTlJycHDIzM+1eZem+G2sSXtmHtKwcvlqzt0zPLSIicq1xqYCUnJxMWJj9w1vDwsLIzMzk1KlTpKWlUVBQUGSb5ORk2zE8PT0JCgq6aJuijB8/nsDAQNvr7JOKy4qH2Y1/3NoAgE9XJJKVk1+m5xcREbmWuFRAcqYxY8aQkZFhe+3fX/Z3lPW+vgZ1gytx/GQeM1bvKfPzi4iIXCtcKiBVrVq10N1mKSkpBAQE4OPjQ3BwMGazucg2VatWtR0jNzeX9PT0i7YpipeXFwEBAXavsuZuduPZKGsv0r9X7ibjVF6Z1yAiInItcKmA1L59e2JiYuy2LV68mPbt2wPWh+W1adPGro3FYiEmJsbWpk2bNnh4eNi1iY+PJykpydamPOvVsjoNw/zIPJ3Pf37f7exyREREKiSnBqSsrCzi4uKIi4sDrLfxx8XF2Z7hMmbMGAYMGGBrP2TIEHbv3s0LL7zAjh07+Pjjj/nuu+8YOXKkrc2oUaP47LPP+PLLL9m+fTtPP/002dnZDB48GIDAwEAee+wxRo0axbJly4iNjWXw4MG0b9++2HewOZObm4mRUQ0B+GL1Xo5l5zq5IhERudZ16dKFESNGlNrxBg0adMm72suCuzNP/ueff9K1a1fbn0eNGgXAwIEDmTFjBocPH7Z74F2dOnVYsGABI0eOZPLkydSsWZPPP/+c6OhoW5sHHniAI0eO8Nprr5GcnEzr1q1ZuHCh3cTtiRMn4ubmRp8+fcjJySE6OpqPP/64DK64dEQ3q0qz6gFsPZTJtJWJjLm9ibNLEhERqVDKzTpIrqas10G6UMz2FB778k98PMysfKErIf5eZV6DiIiUHlddB2nQoEF8+eWXdtv27NlDVlYWzz//PL///juVKlWiR48eTJw4keDgYAB++OEH3njjDXbt2oWvry/XX389//vf//jXv/7FG2+8YXe8ZcuW0aVLl2LXdM2tgyTn3No4lNbhQZzKK+CT5YnOLkdERBzBMCA32zmvYvafTJ48mfbt2/PEE09w+PBhDh8+jL+/P7feeivXX389f/75JwsXLiQlJYX7778fgMOHD9OvXz8effRRtm/fzvLly7n33nsxDIPnnnuO+++/n9tuu812vA4dOjjyu1wkpw6xScmZTCZG92jII/9Zz8x1+3iyU12qBrrOvzhERKQY8k7CO9Wdc+6XD4Fnpcs2CwwMxNPTE19fX9vd4G+//TbXX38977zzjq3dF198QXh4ODt37iQrK4v8/HzuvfdeateuDUCLFi1sbX18fMjJybnk3eWOph4kF9axfjDtIiqTm29hyrIEZ5cjIiICwF9//cWyZcvw8/OzvRo3bgxAYmIirVq1olu3brRo0YL77ruPzz77jOPHjzu5anvqQXJhJpOJUT0a8uC/1zJ7w36GdK5Hzet8nV2WiIiUFg9fa0+Os85dQllZWfTq1Yt333230HvVqlXDbDazePFi/vjjDxYtWsRHH33EK6+8wrp166hTp87VVF1qFJBc3E11q9CxfjCrdqXxUcwu3u3b0tkliYhIaTGZijXM5Wyenp4UFBTY/nzDDTcwZ84cIiIicHcvOmqYTCZuvvlmbr75Zl577TVq167NvHnzGDVqVKHjOYOG2CqAUT2s6yL9sPEAe9OynVyNiIhcayIiIli3bh179+4lLS2NoUOHcuzYMfr168eGDRtITEzkt99+Y/DgwRQUFLBu3Treeecd/vzzT5KSkpg7dy5HjhyhSZMmtuP9/fffxMfHk5aWRl5e2T85QgGpArih1nV0bRRCgcVgcozmIomISNl67rnnMJvNNG3alJCQEHJzc1m9ejUFBQX06NGDFi1aMGLECIKCgnBzcyMgIICVK1fSs2dPGjZsyNixY/nggw+4/fbbAXjiiSdo1KgRN954IyEhIaxevbrMr0nrIJWQs9dButDmAxn0mrIKkwkWjehEgzB/Z5ckIiJXwFXXQSqPtA6S2LSoGUh0szAMAyYtUS+SiIjI1VBAqkBGdm+IyQQLNh9m26FMZ5cjIiLishSQKpDGVQO4o0U1ACYu2enkakRERFyXAlIFMyKqIW4mWLwthb/2pzu7HBEREZekgFTB1A/1457rawAwYbF6kUREXI3unbp6pfE9VECqgJ7t1gCzm4kVO48Qu++Ys8sREZFi8PDwAODkyZNOrsT1nf0env2eloRW0q6AalepxP031uS/6/fzwaKdzHriJmeXJCIil2E2mwkKCiI1NRUAX19fTCaTk6tyLYZhcPLkSVJTUwkKCsJsNpf4WApIFdSwWxswJ/YgfyQe5Y/ENDrUC3Z2SSIichlnn15/NiRJyQQFBdm+lyWlgFRB1Qjy4cF24Xy1Zh8TFu2k/ZAq+peIiEg5ZzKZqFatGqGhoU55vEZF4OHhcVU9R2cpIFVgQ7vWZ/aG/fy57zgrE9Lo3DDE2SWJiEgxmM3mUvklLyWnSdoVWFiAN4/cVBuADxbF684IERGRYlJAquCGdKmHr6eZvw9ksGS7xrRFRESKQwGpvDEMSN5SaocL9vNiYIcIwLouksWiXiQREZHLUUAqTwry4cdh8O8usHtFqR32qU518fdyZ/vhTH7dklxqxxUREamoFJDKE5MJcrLAkgffPgSH/y6Vwwb5evJoxzqA9RltBepFEhERuSQFpPLEzQy9p0HELZB7Ar7pC8f3lsqhH7ulDoE+HuxKzeKnvw6VyjFFREQqKgWk8sbDGx78BsKaQ1YKfH0vZKdd9WEDvD14slNdACYt2Ul+geWqjykiIlJRKSCVR96B8NAPEBgOxxJh1v2Qm33Vhx3UIYIqlTzZe/QkczceLIVCRUREKiYFpPIqoBo8PBd8roODsfD9ICi4ulVVK3m5M6RzPQAmxySQm69eJBERkaIoIJVnIQ2h/3fg7gMJi+CnZ63LAFyFh2+qTYi/FwfTT/Hdn/tLqVAREZGKRQGpvAtvB/fNAJMZ4r6BpW9d1eF8PM0M61ofgClLd3E6r6AUihQREalYFJBcQaPboNck69e/fwDrpl3V4R5sF071QG+SM08za13S1dcnIiJSwSgguYobBsCtY61f//oibJ1X4kN5uZsZdmsDAD5ensjJ3PzSqFBERKTCUEByJbc8B20fBwyY+yTs+b3Eh7rvxprUquxLWlYOX63ZV3o1ioiIVAAKSK7EZILb34MmvaAgF77tX+LntnmY3fhHN2sv0rQViWTlqBdJRETkLAUkV+Nmhns/h9o3Q04mzOwD6SWbR3RP6+rUDa7E8ZN5TF+1p5QLFRERcV0KSK7IwxsenAWhTSEr2bra9sljV3wYd7MbI7o3BODfv+8m4+TVrbMkIiJSUSgguSqfIOtq2wE14WjCmdW2T17xYe5sUY1GYf6cOJ3P56t2l36dIiIiLsjpAWnq1KlERETg7e1NZGQk69evv2jbvLw83nzzTerVq4e3tzetWrVi4cKFdm0iIiIwmUyFXkOHDrW16dKlS6H3hwwZ4rBrdJjAGvDwHPAOggMb4IfBUHBlc4nc3EyM7G6di/TFqj0cy851QKEiIiKuxakBafbs2YwaNYpx48axceNGWrVqRXR0NKmpqUW2Hzt2LNOmTeOjjz5i27ZtDBkyhN69e7Np0yZbmw0bNnD48GHba/HixQDcd999dsd64okn7Nq99957jrtQRwptDP1ng7s37FwIP4+44tW2o5tVpVn1ALJzC5i2MtExdYqIiLgQpwakCRMm8MQTTzB48GCaNm3Kp59+iq+vL1988UWR7b/++mtefvllevbsSd26dXn66afp2bMnH3zwga1NSEgIVatWtb1+/vln6tWrR+fOne2O5evra9cuICDAodfqULVugr7TweQGm76GZf93RbubTCZG97DORfryj72knjjtiCpFRERchtMCUm5uLrGxsURFRZ0rxs2NqKgo1qxZU+Q+OTk5eHt7223z8fFh1apVFz3HzJkzefTRRzGZTHbvffPNNwQHB9O8eXPGjBnDyZOXnr+Tk5NDZmam3atcadwT7pxo/Xrlv2D9Z1e0e9dGobQOD+J0noVPlqsXSURErm1OC0hpaWkUFBQQFhZmtz0sLIzk5OQi94mOjmbChAkkJCRgsVhYvHgxc+fO5fDhw0W2nz9/Punp6QwaNMhue//+/Zk5cybLli1jzJgxfP311zz88MOXrHf8+PEEBgbaXuHh4cW/2LLSZhB0edn69S/Pw7Yfi73r+b1I36xL4nDGKQcUKCIi4hqcPkn7SkyePJkGDRrQuHFjPD09GTZsGIMHD8bNrejL+M9//sPtt99O9erV7bY/+eSTREdH06JFCx566CG++uor5s2bR2LixXtOxowZQ0ZGhu21f//+Ur22UtP5BWgzGDBgzuOwd3Wxd+1YP5h2dSqTm29hytJdjqtRRESknHNaQAoODsZsNpOSkmK3PSUlhapVqxa5T0hICPPnzyc7O5t9+/axY8cO/Pz8qFu3bqG2+/btY8mSJTz++OOXrSUyMhKAXbsuHgq8vLwICAiwe5VLJhPc8QE0vhMKcuC//SBlazF3NTH6zLpI3/25n/3HrnzZABERkYrAaQHJ09OTNm3aEBMTY9tmsViIiYmhffv2l9zX29ubGjVqkJ+fz5w5c7j77rsLtZk+fTqhoaHccccdl60lLi4OgGrVql3ZRZRXbmbo8zmE3wQ5GTCzL6QXr8crsm4VOtYPJq/A4KOlCQ4uVEREpHxy6hDbqFGj+Oyzz/jyyy/Zvn07Tz/9NNnZ2QwePBiAAQMGMGbMGFv7devWMXfuXHbv3s3vv//ObbfdhsVi4YUXXrA7rsViYfr06QwcOBB3d3e79xITE3nrrbeIjY1l7969/PjjjwwYMIBOnTrRsmVLx190WfHwgX7/hZDGcOKQ9ZEkxVxte9SZuUhzNh5kT1q2I6sUEREpl5wakB544AHef/99XnvtNVq3bk1cXBwLFy60TdxOSkqym4B9+vRpxo4dS9OmTenduzc1atRg1apVBAUF2R13yZIlJCUl8eijjxY6p6enJ0uWLKFHjx40btyY0aNH06dPH3766SeHXqtT+Fa2LiTpXx3S4uG/D0Le5Sdf31DrOm5tHEqBxWDykp1lUKiIiEj5YjKMK1xVUADIzMwkMDCQjIyM8jsf6azU7fBFNJzOgEY94f6vwex+yV22HMzgzo9WYTLBohGdaBDmX0bFioiIOE5xf3+71F1sUkKhTaDft2D2gvhf4JfRl11tu3mNQKKbhWEYMGmJ5iKJiMi1RQHpWlG7A/T9wrraduwMWP7Py+4ysntDTCZYsPkwWw9lOL5GERGRckIB6VrS5E7o+b716xX/hD+LfqTLWY2rBnBnS+saUhMXqxdJRESuHQpI15q2j0HnF61fLxgN23++ZPMRUQ1wM8GS7Sn8tT/d8fWJiIiUAwpI16IuY+CGAWBYYM5jsK/oZ98B1Avxo/f1NQH4YLHuaBMRkWuDAtK1yGSCOyZCw9sh/zT89wHrnW4X8Wy3Bri7mVi58wh/7i3eWkoiIiKuTAHpWmV2t07artnOevv/zD6QcbDIprWq+HLfjWd6kRapF0lERCo+BaRrmacv9J8NwQ0h86A1JJ06XmTTYbc2wNPsxprdR/ljV1oZFyoiIlK2FJCudbbVtqvBke3w3/5FrrZdI8iHfu3CAetcJK0vKiIiFZkCkkBQLWtI8gqEpD9gzuNgKSjUbGjX+ni5uxG77zgrdh5xQqEiIiJlQwFJrMKaQb9Z1tW2d/wMvzxXaLXt0ABvHrmpNgAT1IskIiIVmAKSnBPREfp8Bpisi0iu/FehJkO61MPX08zfBzJYvC2l7GsUEREpAwpIYq/p3dDzTDBa9n8Q+6Xd28F+XgzqEAFYe5EsFvUiiYhIxaOAJIW1ewJuec769c8jYMcvdm8/2aku/l7u7Eg+wa9bksu+PhEREQdTQJKi3ToWrn/Yutr2D4MhaZ3trSBfTx67pQ4AE5fspEC9SCIiUsEoIEnRTCa4czI0iD632vaReNvbj3asQ6CPB7tSs/jxr6IXmBQREXFVCkhycWZ3uG861LjRuoDkzD6QeQiAAG8PnuxUF4DJSxLIL7A4s1IREZFSpYAkl+ZZCfp/B1XqQ8Z+mNkXTqUDMKhDBFUqebL36EnmblQvkoiIVBwKSHJ5larAw3PBryqkboVvH4K801TycufpLvUAmByTQG6+epFERKRiUECS4rmuNjz8A3gFwL5VMPcJsBTw8E21CfX34mD6KWb/ud/ZVYqIiJQKBSQpvqot4MFvwOwJ23+EX1/E292NoV3rAzBlaQKn8wo/okRERMTVKCDJlanTCXpPA0yw4TP4/QMebBdO9UBvUjJz+GZdkrMrFBERuWoKSHLlmt8Lt79r/XrpW3ht/i/DuzUA4JPluziZm+/E4kRERK6eApKUTORT0HGk9esf/8F9AVupVdmXtKxcvlqzz7m1iYiIXCUFJCm5buOgVT8wCnD/YTCv33AKgE9XJHLidJ6TixMRESk5BSQpOZMJ7voI6kdB/im6xg6lc5XjpJ/MY/rqvc6uTkREpMQUkOTqmD3gvi+h+g2YTh3jE+P/COU4n/2+m4yT6kUSERHXpIAkV8/LDx76HirXw/fkIf7r+z6czuTzVbudXZmIiEiJKCBJ6agUDI/MBb8w6ln28G+PCXy9aifHsnOdXZmIiMgVU0CS0nNdBDz0PYanP+3N23jb+IhpyxOcXZWIiMgVU0CS0lWtFaYHZ2Jx8+BO8zpqrnuT1MxTzq5KRETkiiggSemr2wVT708BeMRtIX99+4aTCxIREbkyCkjiEKYWfdndZiwA3Q99QvofM5xbkIiIyBVQQBKHqXPnc/yvUl8A/BeNgoTFTq5IRESkeBSQxGFMJhNV7/0ncws6YqYAy+xH4ECss8sSERG5LKcHpKlTpxIREYG3tzeRkZGsX7/+om3z8vJ48803qVevHt7e3rRq1YqFCxfatXn99dcxmUx2r8aNG9u1OX36NEOHDqVKlSr4+fnRp08fUlJSHHJ917rIeiH8WHsMKwta4JZ/CmbdB0cTnV2WiIjIJTk1IM2ePZtRo0Yxbtw4Nm7cSKtWrYiOjiY1NbXI9mPHjmXatGl89NFHbNu2jSFDhtC7d282bdpk165Zs2YcPnzY9lq1apXd+yNHjuSnn37i+++/Z8WKFRw6dIh7773XYdd5rXu2RzOezhvBZksdOHkUvu4NJxRIRUSk/DIZhmE46+SRkZG0bduWKVOmAGCxWAgPD2f48OG89NJLhdpXr16dV155haFDh9q29enTBx8fH2bOnAlYe5Dmz59PXFxckefMyMggJCSEWbNm0bevdX7Mjh07aNKkCWvWrOGmm24qVu2ZmZkEBgaSkZFBQEDAlVz2NemxGRuI25HAQv+3CMk7BFVbwqAF4K3vnYiIlJ3i/v52Wg9Sbm4usbGxREVFnSvGzY2oqCjWrFlT5D45OTl4e3vbbfPx8SnUQ5SQkED16tWpW7cuDz30EElJSbb3YmNjycvLsztv48aNqVWr1kXPK1dvZPeGHCWQvlnPk+8TDMl/w+yHIT/H2aWJiIgU4rSAlJaWRkFBAWFhYXbbw8LCSE5OLnKf6OhoJkyYQEJCAhaLhcWLFzN37lwOHz5saxMZGcmMGTNYuHAhn3zyCXv27OGWW27hxIkTACQnJ+Pp6UlQUFCxzwvWcJaZmWn3kuJrXiOQ25pVZZ8Rxr+C3wZPP9izAuY/DRaLs8sTERGx4/RJ2ldi8uTJNGjQgMaNG+Pp6cmwYcMYPHgwbm7nLuP222/nvvvuo2XLlkRHR/PLL7+Qnp7Od999d1XnHj9+PIGBgbZXeHj41V7ONWdk94aYTDAtIYC9UZ+CmztsmQOLXgHnjfSKiIgU4rSAFBwcjNlsLnT3WEpKClWrVi1yn5CQEObPn092djb79u1jx44d+Pn5Ubdu3YueJygoiIYNG7Jr1y4AqlatSm5uLunp6cU+L8CYMWPIyMiwvfbv31/MK5WzGlX1p1fL6gC8vb0q3POJ9Y21H8MfHzmxMhEREXtOC0ienp60adOGmJgY2zaLxUJMTAzt27e/5L7e3t7UqFGD/Px85syZw913333RtllZWSQmJlKtWjUA2rRpg4eHh9154+PjSUpKuuR5vby8CAgIsHvJlXs2qgFuJliyPZW463pAj7etbyx+Ff6a7dziREREznDqENuoUaP47LPP+PLLL9m+fTtPP/002dnZDB48GIABAwYwZswYW/t169Yxd+5cdu/eze+//85tt92GxWLhhRdesLV57rnnWLFiBXv37uWPP/6gd+/emM1m+vXrB0BgYCCPPfYYo0aNYtmyZcTGxjJ48GDat29f7DvYpOTqhfjR+/qaAExYvBM6DIf2w6xv/u8Z2BVzib1FRETKhrszT/7AAw9w5MgRXnvtNZKTk2ndujULFy60TdxOSkqym190+vRpxo4dy+7du/Hz86Nnz558/fXXdhOuDxw4QL9+/Th69CghISF07NiRtWvXEhISYmszceJE3Nzc6NOnDzk5OURHR/Pxxx+X2XVf657t1oD/xR1k5c4jbNh7jLbd34ITybDlB5j9CAz6GWrc4OwyRUTkGubUdZBcmdZBujpj5m7mv+uTuKluZb59sj3k51pX2d69HHyD4bFFUKWes8sUEZEKptyvgyTXtuG31sfT7Mba3cf4Y1cauHvC/V9bF5A8mQYz+0BW0Suqi4iIOJoCkjhF9SAf+kfWAuCDxTsxDMO6qvbDc+C6CDi+B77pCzknnFuoiIhckxSQxGme6VIPL3c3YvcdZ/nOI9aNfqHw8FzrMNvhv6xzkvJznVuoiIhccxSQxGlCA7wZ0L42ABPP9iKBde7RQ9+BRyXYvQz+N1SrbYuISJlSQBKnGtK5Hr6eZv4+kMHibectGlqjDdz/lXW17c3fwZLXnFekiIhccxSQxKmq+Hkx+OYIwLouksVy3k2VDaLg7qnWr//4CP6YUvYFiojINUkBSZzuiVvq4u/lzo7kE/yy5bD9m60ehKg3rF8vegU2/1D2BYqIyDVHAUmcLsjXk8duqQNY5yIVWC5YmuvmZyHyaevX84ZA4rIyrlBERK41CkhSLjzasQ5Bvh4kHsnmf3EH7d80mSD6HWjWGyx5MPth6x1uIiIiDqKAJOVCgLcHT3aqC8DkmATyCi64a83NDXpPgzqdIDcLZvaFY3ucUKmIiFwLFJCk3BjYPoIqlTzZd/QkczceKNzA3Qse+AbCWkB2Ksy8F7KOlH2hIiJS4SkgSblRycudp7tYn7/2YcwucvILCjfyDoCHf4CgWnBst/X5bTlZZVypiIhUdApIUq48fFNtwgK8OJh+iu827C+6kX9VeHge+FaBQ5vguwFQkFe2hYqISIWmgCTlireHmaFd6wMwZdkuTucV0YsEEFwf+n8HHr6QGAP/GwaGUXRbERGRK6SAJOXOA23DqRHkQ0pmDt+sS7p4w5o3wn1fgskMf38LS14vsxpFRKRiU0CScsfL3czwW629SJ8s38XJ3PyLN27YA+76yPr16kmw9hPHFygiIhWeApKUS33a1KRWZV/SsnL58o99l258/UPQ7cyz2haOgS1zHF+giIhUaApIUi55mN14tlsDAKatTOTE6ctMwu44Cto9CRjW1bZ3r3B8kSIiUmEpIEm5dc/1NagXUon0k3l8sWrvpRubTHDbP6Hp3VCQC98+BIf/LpM6RUSk4lFAknLL7GZiRFRDAD5ftZuMk5fpRXIzQ+9/Q8QtkHsCvukLx/c6vlAREalwFJCkXLujRTUaV/XnxOl8Pvt99+V38PCGB7+BsOaQlQJf3wvZaY4vVEREKhQFJCnX3NxMjOxu7UWavnoPR7NyLr+TdyA89AMEhsOxRJh1P+RmO7hSERGpSBSQpNzr0TSMFjUCyc4tYNrKYvQiAQRUg4fngs91cDAWvh+k1bZFRKTYFJCk3DOZTIw604v01Zq9pJ44XbwdQxpaV9t294GERfDTs1ptW0REikUBSVxCl0Yh3FAriNN5Fj5ellj8HcPbwX0zrKttx30DS99yWI0iIlJxKCCJSzCZTIzu0QiAWeuSOJR+qvg7N7oNek2yfv37B7Du36VfoIiIVCgKSOIyOtSrQmSdyuQWWJiybNeV7XzDAOg61vr1ry/A1vmlXp+IiFQcCkjiMs7vRfpuw372Hzt5ZQfo9By0fRwwYO4TsOf30i9SREQqBAUkcSnt6lTmlgbB5FsMJsckXNnOJhPc/h406XVmte3+kLzFMYWKiIhLU0ASl3O2F2nuxgPsPpJ1ZTu7meHez6H2zZCTaV1tOz3JAVWKiIgrU0ASl9M6PIhujUOxGFx5LxKcWW17FoQ2hROHrattnzxW+oWKiIjLUkASl3R2de0f/zrEzpQTV34AnyDratsBNeFowpnVtq9wTpOIiFRYCkjikprXCOT25lUxDJi4eGfJDhJYAx6eA95BcGAD/DAYCvJLtU4REXFNCkjiskZ2b4jJBL9uSWbroYySHSS0MfSfDe7esHMh/DxCq22LiIgCkriuhmH+3NWqOnAVvUgAtW6Cvl+AyQ02fQ3L3imlCkVExFUpIIlLe7ZbA9xMsGR7KpuSjpf8QI3vgDsmWL9e+R5s+Lx0ChQREZfk9IA0depUIiIi8Pb2JjIykvXr11+0bV5eHm+++Sb16tXD29ubVq1asXDhQrs248ePp23btvj7+xMaGso999xDfHy8XZsuXbpgMpnsXkOGDHHI9Ylj1Q3x494bagIw4Wp6kQBuHAxdxli/XvAcbPvxKqsTERFX5dSANHv2bEaNGsW4cePYuHEjrVq1Ijo6mtTU1CLbjx07lmnTpvHRRx+xbds2hgwZQu/evdm0aZOtzYoVKxg6dChr165l8eLF5OXl0aNHD7Kzs+2O9cQTT3D48GHb67333nPotYrjPNutAe5uJn5PSGP9nqu8Xb/zi9BmMGDAnMdh3x+lUqOIiLgWk2E4b0ZqZGQkbdu2ZcqUKQBYLBbCw8MZPnw4L730UqH21atX55VXXmHo0KG2bX369MHHx4eZM2cWeY4jR44QGhrKihUr6NSpE2DtQWrdujWTJk0qce2ZmZkEBgaSkZFBQEBAiY8jpePleZuZtS6JyDqV+fbJmzCZTCU/mKUAvhsAO34G70AYvBDCmpZesSIi4jTF/f3ttB6k3NxcYmNjiYqKOleMmxtRUVGsWbOmyH1ycnLw9va22+bj48OqVasuep6MDOvdTZUrV7bb/s033xAcHEzz5s0ZM2YMJ09eeg2cnJwcMjMz7V5SfgzrWh9Psxvr9hzjj8SjV3cwNzP0+RzCb4LTGTCzD6TvL51CRUTEJTgtIKWlpVFQUEBYWJjd9rCwMJKTk4vcJzo6mgkTJpCQkIDFYmHx4sXMnTuXw4cPF9neYrEwYsQIbr75Zpo3b27b3r9/f2bOnMmyZcsYM2YMX3/9NQ8//PAl6x0/fjyBgYG2V3h4+BVesThS9SAf+kfWAuCDRfFcdceohw/0+y+ENIYTh6whSatti4hcM5w+SftKTJ48mQYNGtC4cWM8PT0ZNmwYgwcPxs2t6MsYOnQoW7Zs4dtvv7Xb/uSTTxIdHU2LFi146KGH+Oqrr5g3bx6JiYkXPfeYMWPIyMiwvfbvV49CefNMl3p4e7ixMSmd5TuPXP0BfStbF5L0rw5p8fDfByHv1NUfV0REyj2nBaTg4GDMZjMpKSl221NSUqhatWqR+4SEhDB//nyys7PZt28fO3bswM/Pj7p16xZqO2zYMH7++WeWLVtGzZo1L1lLZGQkALt27bpoGy8vLwICAuxeUr6EBngzoH0EABMW7bz6XiSAwJrwyFzrXKT96+CHR7XatojINcBpAcnT05M2bdoQExNj22axWIiJiaF9+/aX3Nfb25saNWqQn5/PnDlzuPvuu23vGYbBsGHDmDdvHkuXLqVOnTqXrSUuLg6AatWqlexipNx4qlNdfD3NbD6YwaJtKZffoThCm0C/b8HsBfG/wC+jtdq2iEgFV6KA9OWXX7JgwQLbn1944QWCgoLo0KED+/btK/ZxRo0axWeffcaXX37J9u3befrpp8nOzmbw4MEADBgwgDFjxtjar1u3jrlz57J7925+//13brvtNiwWCy+88IKtzdChQ5k5cyazZs3C39+f5ORkkpOTOXXKOjSSmJjIW2+9RWxsLHv37uXHH39kwIABdOrUiZYtW5bk2yHlSBU/Lx692RqKJy7eicVSSkGmdgfo+x/ratuxM2DFu6VzXBERKZdKFJDeeecdfHx8AFizZg1Tp07lvffeIzg4mJEjRxb7OA888ADvv/8+r732Gq1btyYuLo6FCxfaJm4nJSXZTcA+ffo0Y8eOpWnTpvTu3ZsaNWqwatUqgoKCbG0++eQTMjIy6NKlC9WqVbO9Zs+eDVh7rpYsWUKPHj1o3Lgxo0ePpk+fPvz0008l+VZIOfTELXXx93ZnR/IJFmwuegJ/iTTpBT3ft369fDz8Ob30ji0iIuVKidZB8vX1ZceOHdSqVYsXX3yRw4cP89VXX7F161a6dOnCkSOlMEG2nNM6SOXb5CUJTFyyk3ohlVg0sjNmt6tYF+lCy96x9iCZ3OCBmdbHlIiIiEtw6DpIfn5+HD1qXWtm0aJFdO/eHbDODTo7lCXiTI92jCDI14PEI9n8L+5g6R68yxi4YQAYFuuk7aS1pXt8ERFxuhIFpO7du/P444/z+OOPs3PnTnr27AnA1q1biYiIKM36RErE39uDpzrVA2DSkgTyCiyld3CTCe6YCA1vh/zTMOt+SN1eescXERGnK1FAmjp1Ku3bt+fIkSPMmTOHKlWqABAbG0u/fv1KtUCRkhrYoTbBfp4kHTvJnNgDpXtwszv0/QJqtju32nZGKfdUiYiI0zj1WWyuTHOQXMPnv+/m7QXbqRHkw9LnOuPlbi7dE5w8Bl9EQ9pOCGkCj/4KPteV7jlERKTUOHQO0sKFC+2efzZ16lRat25N//79OX78eEkOKeIQD99Um7AALw6mn+K7DQ5Y/dy22nY1OLId/ttfq22LiFQAJQpIzz//vO1hrZs3b2b06NH07NmTPXv2MGrUqFItUORqeHuYGda1PgAfLd3F6byC0j9JUC1rSPIKhKQ/YM7jYHHAeUREpMyUKCDt2bOHpk2bAjBnzhzuvPNO3nnnHaZOncqvv/5aqgWKXK3724ZTI8iH1BM5zFxb/IVMr0hYM+g3C8yesONnmP80pF380TUiIlK+lSggeXp6cvLkSQDboosAlStXtvUsiZQXXu5m/tHN2ov06YpEsnMc9Cy1iI5w72eACf6eDVPawLROsHoypOvhxiIirqREAaljx46MGjWKt956i/Xr13PHHdaF8nbu3HnZB8OKOMO9N9SkdhVf0rJy+XLNXsedqNk98ND3UD8KTGY4/Bcsfg0mNYf/9IB10+BEKT0jTkREHKZEAWnKlCm4u7vzww8/8Mknn1CjRg0Afv31V2677bZSLVCkNHiY3Xi2WwMA/r1yNydO5znuZA26W+ckPbcT7pwItTsCJti/Dn59ASY0hi/vgtgvrXfBiYhIuaPb/EtIt/m7ngKLQY+JK0g8ks3IqIY8G9Wg7E6eeQi2zoctc+Dgn+e2u7lDvW7QvA807gle/mVXk4jINai4v79LHJAKCgqYP38+27dbVxBu1qwZd911F2ZzKa8zU04pILmmn/8+xLBZm/D3cuf3F7sS5OtZ9kUc3wtb5lpfKZvPbXf3hgY9oEVf6389fMq+NhGRCs6hAWnXrl307NmTgwcP0qhRIwDi4+MJDw9nwYIF1KtXr+SVuwgFJNdksRj0/PB3diSfYGjXejwf3di5BR2JPxOWfoCj59315ulnfQhu8z5Qtyu4OyHIiYhUQA4NSD179sQwDL755hsqV64MwNGjR3n44Ydxc3NjwYIFJa/cRSggua7ftibz1Nex+Hqa+f2FrlTx83J2SWAYkLzZOgS3ZS5kJJ17zzsImt5lDUsRt4DbtdFLKyLiCA4NSJUqVWLt2rW0aNHCbvtff/3FzTffTFZW1pVX7GIUkFyXYRjcNWU1mw9m8GSnurzcs4mzS7JnGHBggzUsbZ0HWefd9VYp1HqnXPM+1ufAuZXoPgsRkWuWQx814uXlxYkTJwptz8rKwtNTQwFSvplMJkb1aAjAl3/sJTXztJMruoDJBOHt4PZ3YdR2GPgTtBlkfcZbdiqs/7f1+W+TWsCisXAozhqqRESk1JQoIN155508+eSTrFu3DsMwMAyDtWvXMmTIEO66667SrlGk1HVpGMINtYLIybfw8fJEZ5dzcW5mqNMJek2G5xKg//fQ8kHw9IfMA/DHR/DvzvBRG1j6f5C6w9kVi4hUCCUaYktPT2fgwIH89NNPeHh4AJCXl8fdd9/N9OnTCQoKKu06yx0Nsbm+P3al0f/zdXia3Vj+fBeqB7nQXWN5pyBhsXUYbudCyD+vFyy0GTS/1/qqXNd5NYqIlEMOv80frHeznb3Nv0mTJtSvX7+kh3I5CkgVw4P/XsPa3cfo164W4+9tcfkdyqOcExC/0BqWdi0By3mLYNZoY52v1Kw3BFR3Xo0iIuVEqQekUaNGFfvkEyZMKHZbV6WAVDFs2HuM+z5dg7ubiaWju1Criq+zS7o6p47D9p+tYWnPCjAsZ94wQe0O1l6lpvdApWBnViki4jSlHpC6du1arBObTCaWLl1avCpdmAJSxfHIf9bxe0IafdvU5P37Wjm7nNKTlQrb/mcNS0lrzm03maFu5zOrd98JPkFOK1FEpKyVyRDbtUwBqeKI25/OPVNX42aCxaM6Uy/Ez9kllb6MA9YlA7bMgUObzm03e1ofrNu8DzS8Dbwq4LWLiJxHAcnBFJAqlse/3MCS7anc1ao6H/a73tnlONbRRNh65lEnqdvObffwtYak5n2socnD23k1iog4iAKSgykgVSxbD2Vwx4erMJlg4bOdaFT1GnlobMo2a1ja/AMc33Nuu1eAdfiteR/rcJzZw3k1ioiUIgUkB1NAqnie+SaWXzYnc3vzqnzycBtnl1O2DMM69HZ29e7Mg+fe86kMTe+2hqXaHfSoExFxaQpIDqaAVPHsTDlB9KSVGAb8PLwjzWsEOrsk57BYYP86a1jaNh+yj5x7z7+adcmA5n2sSwiYTE4rU0SkJBSQHEwBqWJ69ttN/C/uEN0ah/KfQW2dXY7zFeTD3t+tYWn7j3A649x7QbWsQal5HwhrrrAkIi5BAcnBFJAqpt1Hsug+cSUFFoN5z3Tg+lrXObuk8iM/BxKXWsPSjl8gL/vce8ENz4Wl4AbOq1FE5DIUkBxMAaniev77v/g+9gC3NAjm68cinV1O+ZR7EhJ+O/Ook0VQkHPuvaotzqzefS9cV9t5NYqIFEEBycEUkCqu/cdO0vX95eRbDL57qj3t6lR2dknl2+lMiP/FGpYSl4Il/9x7NdudCUv3gH9Vp5UoInKWApKDKSBVbC/P28ysdUm0q1OZ2U/ehEnza4rn5DHrXKUtc2DP78DZv15MENHRGpaa3g2+Cp0i4hwKSA6mgFSxHc44Red/LSc338LMxyLp2EDPLrtiJ5Jh63xrWDqw/tx2N3eo2/XMo07uAG/9/IhI2VFAcjAFpIrv9R+3MuOPvVxfK4i5T3dQL9LVOL7v3KNOkv8+t93sBQ17WMNSg2jwdPGHBYtIuaeA5GAKSBVf6onTdHpvGafzLEwf1JaujUOdXVLFkJZgfczJlh8gbee57R6VoHFPa1iqdyu4ezmvRhGpsBSQHEwB6dow/pftTFu5m+Y1AvhpWEf1IpUmw4CUrdagtGUOpCede887EJr0soaliE5gdndenSJSoSggOZgC0rXhWHYut7y7lOzcAj59uA23NdedWA5hGHAw1hqUtsyFrORz71UKgab3WMNSeCS4uTmtTBFxfcX9/e30v2mmTp1KREQE3t7eREZGsn79+ou2zcvL480336RevXp4e3vTqlUrFi5ceMXHPH36NEOHDqVKlSr4+fnRp08fUlJSSv3axPVVruTJ4JvrADBx8U4sFv17wiFMJqh5I9w2HkZtg0EL4MZHrc+Byz4CGz6D6bfBpObw2ytwcKM1VImIOIhTA9Ls2bMZNWoU48aNY+PGjbRq1Yro6GhSU1OLbD927FimTZvGRx99xLZt2xgyZAi9e/dm06ZNV3TMkSNH8tNPP/H999+zYsUKDh06xL333uvw6xXX9MQtdfH3dic+5QQLNh92djkVn5vZuiTAnRPhuZ3w8Bxo1R+8AqwP0V0zBT7rCh9eDzFvQco2Z1csIhWQU4fYIiMjadu2LVOmTAHAYrEQHh7O8OHDeemllwq1r169Oq+88gpDhw61bevTpw8+Pj7MnDmzWMfMyMggJCSEWbNm0bdvXwB27NhBkyZNWLNmDTfddFOxatcQ27Xlw5gEJizeSd2QSiwa0Ql3s9M7X689eadh1xLrMFz8r5B/6tx7IU3OPOrkXqhSz3k1iki5V+6H2HJzc4mNjSUqKupcMW5uREVFsWbNmiL3ycnJwdvb226bj48Pq1atKvYxY2NjycvLs2vTuHFjatWqddHznj13Zmam3UuuHYNvjiDI14PdR7L5X9whZ5dzbfLwhiZ3wn3T4fld0Oc/0OgOMHvCke2w7G346Ab4dxf44yPIOODsikXEhTktIKWlpVFQUEBYWJjd9rCwMJKTk4vcJzo6mgkTJpCQkIDFYmHx4sXMnTuXw4cPF/uYycnJeHp6EhQUVOzzAowfP57AwEDbKzw8/EovWVyYv7cHT3Wy9kxMjkkgr8Di5IqucV5+0KIv9JsFzyXA3R9DvW5gMsOhTbBoLExsBl/cBus/g6yih+1FRC7GpcYJJk+eTIMGDWjcuDGenp4MGzaMwYMH41YGd7WMGTOGjIwM22v//v0OP6eULwM71CbYz5OkYyf5IVa9E+WGTxBc/xA8Mtc6Z+mOCVD7ZsAESWvgl+fgg0bw1d2w8Ss4ddzZFYuIC3BaQAoODsZsNhe6eywlJYWqVYu+lTokJIT58+eTnZ3Nvn372LFjB35+ftStW7fYx6xatSq5ubmkp6cX+7wAXl5eBAQE2L3k2uLr6c7TXeoD8FFMAjn5BU6uSAqpFAxtH4PBv8DIrRD9DtRoA4YFdi+HH4fDvxrArAfh7+8hJ8vZFYtIOeW0gOTp6UmbNm2IiYmxbbNYLMTExNC+fftL7uvt7U2NGjXIz89nzpw53H333cU+Zps2bfDw8LBrEx8fT1JS0mXPK/JQZC3CArw4lHGa2RvUi1iuBdaA9kPhiaXwjzjo9hqENQdLHuz8FeY+Dv+qD98NhG0/Qt6pyx5SRK4dTr2Lbfbs2QwcOJBp06bRrl07Jk2axHfffceOHTsICwtjwIAB1KhRg/HjxwOwbt06Dh48SOvWrTl48CCvv/46e/bsYePGjbY5RZc7JsDTTz/NL7/8wowZMwgICGD48OEA/PHHH8WuXXexXbu+XruPV+dvIdTfi5UvdMXbw+zskuRKpO6ArXNh8w9wLPHcdk9/68Nzm/eBel3B7OG8GkXEYYr7+9up6/c/8MADHDlyhNdee43k5GRat27NwoULbUEmKSnJbn7R6dOnGTt2LLt378bPz4+ePXvy9ddf2024vtwxASZOnIibmxt9+vQhJyeH6OhoPv744zK7bnFtD9wYzqfLEzmYfoqZa/fx+C11nV2SXInQxhD6MnQZA4f/Ord6d+YB+Ptb68vnOqjbBWrcaF3Aslor8PBxduUiUob0qJESUg/StW32hiRenLOZKpU8WflCVyp56VlhLs1igQMbrGFp6zzIvuCuNzd3CGtmDUw12lhDU5UGeuyJiAvSs9gcTAHp2pZXYKH7hBXsPXqSF25rxDNnJm9LBWApsN79lrTW+kiTg39CVhGPIvIKhBrXn+tlqtEG/ELLvl4RuSIKSA6mgCTzNh1g5Oy/CPTx4PcXuxLgrTkrFZJhWBedPPgnHPjT+lDdQ3H2K3mfFVgLarbR0JxIOaaA5GAKSFJgMYietJJdqVmMiGrAiKiGzi5JykpBPqRuOxOaYq3/PRIPXPDXqZs7hDY908N0o4bmRMoBBSQHU0ASgJ//PsSwWZvw93Ln9xe7EuTr6eySxFlOZ8KhjdYeprOh6XJDc2fnM2loTqTMKCA5mAKSAFgsBj0//J0dySd4pks9XritsbNLkvKi0NDcRutjUIozNFe1JXj6ln3NItcABSQHU0CSsxZtTebJr2Px9TTz+wtdqeLn5eySpLwq7tCcyWy9a05DcyKlTgHJwRSQ5CzDMLh76mr+PpDBE7fU4ZU7mjq7JHElpzOtPUvnh6Yih+YCoPr19qFJQ3MiV0wBycEUkOR8y+NTGTR9A17ubqx8oSthAd7OLklc1flDc2fnMxVnaK5GG+tdcxqaE7kkBSQHU0CS8xmGQd9P1xC77zgD29fmjbubO7skqUg0NCdSahSQHEwBSS70x640+n++Dk+zG8ue70KNIK1/Iw6koTmRElFAcjAFJClKv3+vZc3uo/RrF874e1s6uxy5llzp0FyNG86FJg3NyTVEAcnBFJCkKH/uPUbfT9cA0LlhCKN7NKRlzSDnFiXXrkJDc7FwZAeXHZqr0QaCG2poTiokBSQHU0CSi3n/t3g+XZFIvsX6o9W9aRijujekSTX9fyLlgIbm5BqngORgCkhyKfuOZjM5JoH5mw5yJidxZ8tqjIhqSP1QP+cWJ3I+w4DMg2cWszwTmg7HQd7Jwm0Dw8+t/q2hOXFRCkgOpoAkxbErNYtJS3by89+HAXAzwT3X12BEt4bUqqJfLFJOnT80d3Y+06WG5s4PTRqak3JOAcnBFJDkSmw/nMmExTtZvM06lOHuZuK+G8MZfmt9qutuN3EFJR2aq9EG/MPKvl6Ri1BAcjAFJCmJv/anM2HxTlbsPAKAp9mN/pG1eKZrPUL9tbikuBANzYmLUkByMAUkuRob9h7j/d/iWbfnGADeHm4MbB/BU53rUbmSp5OrEymhgnw4st0+NF10aK7pucnfGpqTMqSA5GAKSHK1DMPgj8SjvL8onk1J6QBU8jTzWMc6PHZLXQJ9PJxboEhpOH9o7uBGa3jKSi7cztMfalxvH5o0NCcOoIDkYApIUloMw2B5/BHeXxTP1kOZAAR4u/Nkp7oMurkOfl7uTq5QpBSVZGju7PBctdYampOrpoDkYApIUtoMw+C3rclMWLyTnSlZAFSu5MnTnevxSPvaeHuYnVyhiINoaE7KkAKSgykgiaMUWAx+/vsQk5YksCctG4BQfy+Gdq3Pg+3C8XJXUJJrwOlMa8/SgbNLDWhoTkqHApKDKSCJo+UXWJi76SCTlyRwMN36PK3qgd78o1sD+rSpiYdZ/2qWa8iFQ3MHN1rnNl10aO4GCG0KwQ0guBFUqQceWlJDFJAcTgFJykpuvoXZf+5nytIEUjJzAKhdxZcRUQ24q1UNzG4mJ1co4iTFHZoDwATX1bYOyZ3/CmkEvpXLunJxIgUkB1NAkrJ2Oq+Ab9Yl8cnyXaRl5QJQP9SPkVENub15VdwUlEQg54S1Z+nQJjiyE9J2Qlo8nM64+D6+Vay9TMENrIHpbHgKDNf8pgpIAcnBFJDEWbJz8vlyzV6mrdhNxqk8AJpUC2B094Z0axKKyaSgJGLHMCD7CByJPxOYEqyhKS0BMvZffD93HwiufyYwnRegKtcDDy3s6qoUkBxMAUmcLfN0Hl+s2sPnv+8hKycfgFbhQYzu3pBbGgQrKIkUR04WHN11JjjtPBOiEqzbLHkX2enscF0RvU4ariv3FJAcTAFJyovj2bn8+/fdzFi9l1N5BQC0i6jM6B4NiaxbxcnVibiognxI32cfmtLircN2OZcargs+E5jOTA4Pbmj9WsN15YYCkoMpIEl5c+REDp8sT2Tmun3k5lsAuKVBMKO6N+T6Wtc5uTqRCsIwICv13NymtIRzASrzwMX3sw3XnQlNIWd6nDRcV+YUkBxMAUnKq+SM00xZlsDsDfvJK7D+eHdrHMrI7g1pXiPQydWJVGA5WXA04bzQdGbY7mjixYfrTG4QVPu8Xqfz5jtpuM4hFJAcTAFJyrv9x07yYUwCczYewHLmp7xni6qMjGpIgzB/5xYnci0pyIfje+17ndJ2Xn64rlLIBUsSnPlvQE0N110FBSQHU0ASV7H7SBaTYxL48a9DGAaYTHB3q+o8G9WQOsGVnF2eyLXLNlwXfy4wne11yjx48f08fKFK/XPrOJ2/GKa7V9nV76IUkBxMAUlcTXzyCSYu3snCrdbHNZjdTPS9oSbDu9Wn5nV6AKhIuZJz4kxPU4J9z9Plhuuuiyi8EGZwA/DRPMSzFJAcTAFJXNWWgxlMWLyTpTtSAfAwm+jXrhZDu9YnLECTRUXKtYI8OL6v6F6nnMyL71cp5IJlCc70OgXUuOaG6xSQHEwBSVxd7L7jTFgcz+pdRwHwcnfjkZtqM6RLPYL91E0v4lIMA7JSCi9LkJZw+eG6CyeHhzSCynUr7HCdApKDKSBJRfFHYhoTFu3kz33HAfD1NDP45gieuKUuQb6eTq5ORK6abbjugsUwjyWCJb/ofUzm84brLlgM0yeoLKsvdS4TkKZOncq//vUvkpOTadWqFR999BHt2rW7aPtJkybxySefkJSURHBwMH379mX8+PF4e1uHBiIiIti3b1+h/Z555hmmTp0KQJcuXVixYoXd+0899RSffvppsetWQJKKxDAMViak8cGieP4+YL2rxt/LncdvqcujHSPw9/ZwcoUiUuoK8s7dXXdhr9Mlh+tCL1gM80yACqhhvQuknHOJgDR79mwGDBjAp59+SmRkJJMmTeL7778nPj6e0NDQQu1nzZrFo48+yhdffEGHDh3YuXMngwYN4sEHH2TChAkAHDlyhIKCAts+W7ZsoXv37ixbtowuXboA1oDUsGFD3nzzTVs7X1/fKwo6CkhSERmGweJtKUxYvJMdyScACPL1YEjnegxoXxtfT3cnVygiDmcYcCL5XI+TredpJ5w4dPH9PCpdfDFM9/LTG+0SASkyMpK2bdsyZcoUACwWC+Hh4QwfPpyXXnqpUPthw4axfft2YmJibNtGjx7NunXrWLVqVZHnGDFiBD///DMJCQm2Z1N16dKF1q1bM2nSpBLXroAkFZnFYrBg82EmLtnJ7iPZAAT7efFMl3r0j6yFt4fZyRWKiFPknDj3wN/zF8M8tvvyw3VFPYLFCcN1xf397bR/Dubm5hIbG8uYMWNs29zc3IiKimLNmjVF7tOhQwdmzpzJ+vXradeuHbt37+aXX37hkUceueg5Zs6cyahRowo9uPObb75h5syZVK1alV69evHqq6/i63vxW51zcnLIycmx/Tkz8xLdjyIuzs3NRK9W1bm9eVX+F3eISTE72X/sFG/+vI1/r9zN8G71ua9NOJ7u19bdLyLXPC9/qNHG+jpfQR4c21N0r1PuCet8p2OJEH/B8fzCLrIYpvOH65wWkNLS0igoKCAsLMxue1hYGDt27Chyn/79+5OWlkbHjh0xDIP8/HyGDBnCyy+/XGT7+fPnk56ezqBBgwodp3bt2lSvXp2///6bF198kfj4eObOnXvResePH88bb7xxZRcp4uLczW70aVOTu1pX5/s/D/DR0gQOZ5zmlXlb+HRFIs92a8g9ravjblZQErmmmT2s4Sakof1223BdfOFepxOHrXfeZaXA3t/t9/OoZO1h6v4m1O1cdtdxHqcNsR06dIgaNWrwxx9/0L59e9v2F154gRUrVrBu3bpC+yxfvpwHH3yQt99+m8jISHbt2sWzzz7LE088wauvvlqofXR0NJ6envz000+XrGXp0qV069aNXbt2Ua9evSLbFNWDFB4eriE2uaaczivg2/VJTFmWSFqW9eehbkglRkQ15M4W1XBzK/8TNEWknDidaX123ZELep3OH64b/CvU7lCqpy33Q2zBwcGYzWZSUlLstqekpFC1atUi93n11Vd55JFHePzxxwFo0aIF2dnZPPnkk7zyyiu4nbfY1b59+1iyZMkle4XOioyMBLhkQPLy8sLLq2KuCSFSXN4eZgbdXIcH2tbiqzV7+XRFIruPZPOP/27i42W7GNm9IT2ahhUa0hYRKcQ74DLDdfEQ1tw5tQFO6xf39PSkTZs2dhOuLRYLMTExdj1K5zt58qRdCAIwm62TRS/sCJs+fTqhoaHccccdl60lLi4OgGrVql3JJYhcs3w8zTzVuR6/v3gro7s3xN/bnR3JJ3jq61jumrKaZfGphX4mRUSK5exwXZNe1hDlJE69Z3fUqFEMHDiQG2+8kXbt2jFp0iSys7MZPHgwAAMGDKBGjRqMHz8egF69ejFhwgSuv/562xDbq6++Sq9evWxBCaxBa/r06QwcOBB3d/tLTExMZNasWfTs2ZMqVarw999/M3LkSDp16kTLli3L7uJFKgA/L3eGd2vAgPYRfPb7br5YvYfNBzMYPH0DbWpfx+geDelQL9jZZYqIXDGnBqQHHniAI0eO8Nprr5GcnEzr1q1ZuHChbeJ2UlKSXY/R2LFjMZlMjB07loMHDxISEkKvXr34v//7P7vjLlmyhKSkJB599NFC5/T09GTJkiW2MBYeHk6fPn0YO3asYy9WpAIL9PXguehGDL45gk9XJPLVmn3E7jtO/8/W0aFeFUb3aEib2pWdXaaISLE5fSVtV6V1kEQuLjXzNFOX7WLW+iTyCqx/xXRpFMLo7o1oUTPQydWJyLXMJRaKdGUKSCKXd+D4SaYs3cX3sQcosFj/qoluFsbI7g1pXFU/NyJS9hSQHEwBSaT49qZl82FMAvPiDmIY1vXferWszoioBtQN8XN2eSJyDVFAcjAFJJErl5BygklLEliw+TAAbia494aaPNutAeGVL76SvYhIaVFAcjAFJJGS23oog4mLE1iy3boOmrubiQfahjPs1vpUC/RxcnUiUpEpIDmYApLI1Yvbn84Hi+L5PSENAE93Nx6KrMUzXeoT4q+FWUWk9CkgOZgCkkjpWb/nGO8vimf9nmMA+HiYGdghgqc61eW6Sp5Ork5EKhIFJAdTQBIpXYZhsHrXUd5fFE/c/nTAuhDlox3r8PgtdQjw9nBugSJSISggOZgCkohjGIbB0h2pfLBoJ9sOZwIQ6OPBk53qMqhDBJW8nLq+rYi4OAUkB1NAEnEsi8Vg4dZkJi7eSUJqFgBVKnnydJd6PHxTbbw9zJc5gohIYQpIDqaAJFI2CiwGP/11iElLdrL36EkAwgK8GNa1Pve3DcfLXUFJRIpPAcnBFJBEylZ+gYW5Gw8yOSaBg+mnAKgR5MOz3Rpw7w01cDe7XeYIIiIKSA6ngCTiHDn5BXy3YT8fLd1F6okcACKq+DIiqiG9WlXH7GZycoUiUp4pIDmYApKIc53OK2Dm2n18vDyRY9m5ADQI9WNU94ZEN6uKm4KSiBRBAcnBFJBEyofsnHxm/LGXaSsSyTydD0DTagGM7tGQWxuHYjIpKInIOQpIDqaAJFK+ZJzK4z+r9vDFqj1k5ViDUuvwIJ7r0Yib61dRUBIRQAHJ4RSQRMqnY9m5TFuZyJd/7OV0ngWAyDqVGd2jEe3qVHZydSLibApIDqaAJFK+pZ44zSfLE/lmbRK5BdagdEuDYEb3aETr8CDnFiciTqOA5GAKSCKu4VD6KaYs28V3G/aTb7H+dRfVJIxR3RvStLp+dkWuNQpIDqaAJOJako6e5MOlCczdeIAzOYk7WlRjZPcG1A/1d25xIlJmFJAcTAFJxDUlHsli0pIEfvrrEABuJrindQ2ejWpA7SqVnFydiDiaApKDKSCJuLYdyZlMXLyT37amAGB2M3Ffm5oM79aAGkE+Tq5ORBxFAcnBFJBEKoa/D6QzYfFOlscfAcDT7Ea/duE81rEutar4Ork6ESltCkgOpoAkUrH8ufcYHyzayZrdR23bGob5EdUkjKimYbSuGaTVuUUqAAUkB1NAEqmY/tiVxtTlu1i7+xgFlnN/PQb7eXJr41C6NQnjlgbB+Hq6O7FKESkpBSQHU0ASqdjST+ayPP4IS7ansCL+CCfOrM4N4Onuxs31qhDVNIxujcOoGujtxEpF5EooIDmYApLItSM338KGvcdYvC2FJdtTOHD8lN37LWoE0q1JKFFNwmhWPUCPNREpxxSQHEwBSeTaZBgGO1OyWLLdGpbi9qdz/t+i1QK96dbEOhTXvm4VvD3MzitWRApRQHIwBSQRAThyIodlO1JZvD2FVQlpnMorsL3n62mmU4MQujUJ5dbGoVTx83JipSICCkgOp4AkIhc6nVfAH4lpLNmeSsz2FFIyc2zvmUxwQ63rrHfFNQmlfqifhuJEnEABycEUkETkUiwWgy2HMmxhaeuhTLv3a1fxpVvjMKKahtI2ojIeZjcnVSpybVFAcjAFJBG5EofSTxGzI5Ul21JYk3iU3AKL7b0Ab3e6NAolqmkYnRuGEOjj4cRKRSo2BSQHU0ASkZLKyslnVcIRFm9LZVl8Kseyc23vubuZaFenMt2ahNG9SZhW8xYpZQpIDqaAJCKlocBisCnpOEu2p7Jkewq7UrPs3m8Q6kdUU+u8pdbh12HWat4iV0UBycEUkETEEfamZduWENiw97jdat5VKnnStbF1vaVbGgRTyUureYtcKQUkB1NAEhFHyziZx/KdqSzZnsry+FROnLZfzbtDvSpENQmjW5NQqgX6OLFSEdehgORgCkgiUpbyCixs2HOMxWd6l/Yfs1/Nu3mNALo1DqN7U63mLXIpxf397fT7SqdOnUpERATe3t5ERkayfv36S7afNGkSjRo1wsfHh/DwcEaOHMnp06dt77/++uuYTCa7V+PGje2Ocfr0aYYOHUqVKlXw8/OjT58+pKSkOOT6RERKg4fZjQ71gxnXqxkrn+/KopGdeOG2RtxQKwiTCbYczGRyTAJ3frSK9uOX8sq8zSyLT+X0eQtXikjxOXUAe/bs2YwaNYpPP/2UyMhIJk2aRHR0NPHx8YSGhhZqP2vWLF566SW++OILOnTowM6dOxk0aBAmk4kJEybY2jVr1owlS5bY/uzubn+ZI0eOZMGCBXz//fcEBgYybNgw7r33XlavXu24ixURKSUmk4mGYf40DPPnmS71ScvKYemZJQR+T0gjOfM036xL4pt1Sfh6mrmlQTDdmoRxa+NQgrWat0ixOHWILTIykrZt2zJlyhQALBYL4eHhDB8+nJdeeqlQ+2HDhrF9+3ZiYmJs20aPHs26detYtWoVYO1Bmj9/PnFxcUWeMyMjg5CQEGbNmkXfvn0B2LFjB02aNGHNmjXcdNNNxapdQ2wiUh6dzitgTeJR20TvC1fzvj486MxdcWE00Grecg0q90Nsubm5xMbGEhUVda4YNzeioqJYs2ZNkft06NCB2NhY2zDc7t27+eWXX+jZs6ddu4SEBKpXr07dunV56KGHSEpKsr0XGxtLXl6e3XkbN25MrVq1LnpegJycHDIzM+1eIiLljbeHma6NQ/m/3i1YO6YbPw/vyLPdGtC8RgCGARuT0nlvYTw9Jq6k87+W88ZPW/ljVxp55y1cKSJOHGJLS0ujoKCAsLAwu+1hYWHs2LGjyH369+9PWloaHTt2xDAM8vPzGTJkCC+//LKtTWRkJDNmzKBRo0YcPnyYN954g1tuuYUtW7bg7+9PcnIynp6eBAUFFTpvcnLyResdP348b7zxRskvWESkjJlMJprXCKR5jUBGdm/I4YxTxJxZb+mPxKMkHTvJ9NV7mb56L/5nV/NuEkqXhqEE+mo1b7m2udQiGsuXL+edd97h448/JjIykl27dvHss8/y1ltv8eqrrwJw++2329q3bNmSyMhIateuzXfffcdjjz1W4nOPGTOGUaNG2f6cmZlJeHh4yS9GRKSMVQv04eGbavPwTbXJzsnn94Q0YransHRHKkezc/npr0P89NchzG4m2kVUpluTULo3DaN2lUrOLl2kzDktIAUHB2M2mwvdPZaSkkLVqlWL3OfVV1/lkUce4fHHHwegRYsWZGdn8+STT/LKK6/g5lZ4xDAoKIiGDRuya9cuAKpWrUpubi7p6el2vUiXOi+Al5cXXl6a3CgiFUMlL3dua16V25pXpcBiELf/zGre21JISM1ize6jrNl9lLcXbKd+qB9RTayreV9fS6t5y7XBaXOQPD09adOmjd2Ea4vFQkxMDO3bty9yn5MnTxYKQWazGYCLzTXPysoiMTGRatWqAdCmTRs8PDzszhsfH09SUtJFzysiUpGZ3Uy0qV2ZF29rzOJRnVnxfBdevbMpHepVwexmYldqFp+uSKTvp2to+39LeO77v1i45TDZOfmXP7iIi3LqENuoUaMYOHAgN954I+3atWPSpElkZ2czePBgAAYMGECNGjUYP348AL169WLChAlcf/31tiG2V199lV69etmC0nPPPUevXr2oXbs2hw4dYty4cZjNZvr16wdAYGAgjz32GKNGjaJy5coEBAQwfPhw2rdvX+w72EREKrLaVSrxWMc6PNaxDhmn8lix8whLtqWw/MyDdX+IPcAPsQfwNLvRvl4V27PitJq3VCRODUgPPPAAR44c4bXXXiM5OZnWrVuzcOFC28TtpKQkux6jsWPHYjKZGDt2LAcPHiQkJIRevXrxf//3f7Y2Bw4coF+/fhw9epSQkBA6duzI2rVrCQkJsbWZOHEibm5u9OnTh5ycHKKjo/n444/L7sJFRFxEoI8Hd7Wqzl2tqltX8957zDbRe9/Rk6zYeYQVO4/w6nxoVj2Abk3C6N4kjOY1tJq3uDY9aqSEtA6SiFzLDMNgV2qWdd7S9hQ2Jh3n/N8mYQFedDszb6lDvWC8PczOK1bkPHoWm4MpIImInJOWlcOyHanEbE9lZcIRTuaee8SJj4eZjg2C6d4kjK6NQwnx1w0v4jwKSA6mgCQiUrTTeQWs3W1dzTtmeyqHM849L9NkgtbhQWfuigujYZhW85aypYDkYApIIiKXZxgGWw9l2uYtbT6YYfd+eGUfujUOo3vTMNrVqYyH2enPUJcKTgHJwRSQRESuXHLGaWJ2pLBkWwqrE4+Sm3/uESf+Xu50bhRC96ZhWs1bHEYBycEUkERErs7JXPvVvNOycm3vmd1MtI24zjYUFxGs1byldCggOZgCkohI6bFYDOIOpLNkm3XeUnzKCbv364VUIqqpdQkBreYtV0MBycEUkEREHCfp6EnrJO8dKazbfYx8y7lfVZUredKlUQjdm4RxS8MQ/Lxc6rGi4mQKSA6mgCQiUjYyT+exIv4IS7ansGxHKpmnzz3ixNPsxk31qtC9SSjdmoRRPUireculKSA5mAKSiEjZyyuw8Ofe48RsT2HJ9hT2Hj1p937TagFEnQlLzaoH4K674uQCCkgOpoAkIuJchmGQeCT7zHpLKcTuO855I3H4epppXiOQ68ODaBUeROvwIKoFemvdpWucApKDKSCJiJQvx7JzWbbDut7SqoQ0TuTkF2oT4u9F6zNhqXV4EC1rBuLvreUEriUKSA6mgCQiUn4VWAx2H8li0/504van89f+dHYkn6DAYv8rz2SCeiF+tD7Ty3R9eBCNqvprwcoKTAHJwRSQRERcy6ncArYeyiDuTGiK25/OgeOnCrXzcnejeY1Au9BU8zofDc1VEApIDqaAJCLi+o6cyOHvA+cC01/70+3ukjurSiVP2zymVuFBtK4ZpJW+XZQCkoMpIImIVDwWi8Heo9l2gWnb4UzyCgr/qqwTXMk2l6lVeBBNqvnj5W52QtVyJRSQHEwBSUTk2nA6r4DthzNtgSluf3qh5QXAuiZTk+oBZ+6aC6R1+HVEVPHV0Fw5o4DkYApIIiLXruPZufx1IN0uNB0/mVeoXaCPh21ornV4IK1qBlHFz8sJFctZCkgOpoAkIiJnGYbB/mOn2LT/uC00bTmUSW6+pVDb8Mo+tA6/zhaamlUPxNtDQ3NlRQHJwRSQRETkUnLzLcQnnyBu/3E2nQlNiUeyC7VzdzPRuJr/mcB0Ha3DA6kb7IebHsjrEApIDqaAJCIiVyrjVB6bD2QQt/84cfutSw6kZeUUaufv5U7L8DNLDdQMonWtIEL9vZ1QccWjgORgCkgiInK1DMPgUMZp4pLSidt/nL/2Z7D5YAan8goKta0R5HNm8rc1NLWoGYivp7sTqnZtCkgOpoAkIiKOkF9gYWdKlt0E8J2pJ7jwt7WbCRqG+XN9rXNLDTQI9cesoblLUkByMAUkEREpK1k5+WeG5s6FpuTM04Xa+XqaaVEjkNa1gmwP6a0W6OOEissvBSQHU0ASERFnSs44bbeg5d8H0snOLTw0FxbgZZvH1PrM0Ny1/IBeBSQHU0ASEZHypMBikHgki7ikdNtdc/EpRT+gt0Gony00taoZROOq/rhfIw/oVUByMAUkEREp707m5rP1UKZ1EviBdOKS0jmYXvgBvd4ebrSoEWgXmirqA3oVkBxMAUlERFzRkRM5tnlMZ1cDP1HEA3qD/TytgSncGppa1gwi0Mf1h+YUkBxMAUlERCoCi8Vgz9HsM0sNWEPT9os8oLduSCVan9fL1KRaAJ7urjU0p4DkYApIIiJSUZ3OK2DbYevQ3Nlepn1FPaDX3Y1m1QNoVTOI68+Eptrl/AG9CkgOpoAkIiLXkmNnH9B7XmhKL+IBvdf5Wh/Qe/58psqVPJ1QcdEUkBxMAUlERK5lhmGQdOwkcfvT2XQmNG29yAN6a1fxtZvP1LRagNMe0KuA5GAKSCIiIvZy8y3sSM60rs905s653UU8oNfDbKJJtQC70FSnSqUyeUCvApKDKSCJiIhcXsbJPP4+aD80l5aVW6hdgLf7uaG5M6uAh/h7lXo9CkgOpoAkIiJy5QzD4MDxU3bzmTYfzOB0XuGhuUkPtOae62uU6vmL+/tbjwEWERGRMmMymQiv7Et4ZV/ubFkdgLwCCztTTtiG5v46kE5CahaNq/k7rU6nL14wdepUIiIi8Pb2JjIykvXr11+y/aRJk2jUqBE+Pj6Eh4czcuRITp8+98C+8ePH07ZtW/z9/QkNDeWee+4hPj7e7hhdunTBZDLZvYYMGeKQ6xMREZFL8zC70ax6IA9F1uZf97Vi0cjO/D2uBw1Dr9GANHv2bEaNGsW4cePYuHEjrVq1Ijo6mtTU1CLbz5o1i5deeolx48axfft2/vOf/zB79mxefvllW5sVK1YwdOhQ1q5dy+LFi8nLy6NHjx5kZ9tPEnviiSc4fPiw7fXee+859FpFRESk+Py9Pcpk0vbFOHWIbcKECTzxxBMMHjwYgE8//ZQFCxbwxRdf8NJLLxVq/8cff3DzzTfTv39/ACIiIujXrx/r1q2ztVm4cKHdPjNmzCA0NJTY2Fg6depk2+7r60vVqlUdcVkiIiLi4pzWg5Sbm0tsbCxRUVHninFzIyoqijVr1hS5T4cOHYiNjbUNw+3evZtffvmFnj17XvQ8GRkZAFSuXNlu+zfffENwcDDNmzdnzJgxnDxZeIVQERERuTY5rQcpLS2NgoICwsLC7LaHhYWxY8eOIvfp378/aWlpdOzYEcMwyM/PZ8iQIXZDbOezWCyMGDGCm2++mebNm9sdp3bt2lSvXp2///6bF198kfj4eObOnXvRenNycsjJybH9OTMz80ouV0RERFyIS93Ftnz5ct555x0+/vhjIiMj2bVrF88++yxvvfUWr776aqH2Q4cOZcuWLaxatcpu+5NPPmn7ukWLFlSrVo1u3bqRmJhIvXr1ijz3+PHjeeONN0r3gkRERKRcctoQW3BwMGazmZSUFLvtKSkpF50b9Oqrr/LII4/w+OOP06JFC3r37s0777zD+PHjsVjs108YNmwYP//8M8uWLaNmzZqXrCUyMhKAXbt2XbTNmDFjyMjIsL32799fnMsUERERF+S0gOTp6UmbNm2IiYmxbbNYLMTExNC+ffsi9zl58iRubvYlm83WZ7mcXe/SMAyGDRvGvHnzWLp0KXXq1LlsLXFxcQBUq1btom28vLwICAiwe4mIiEjF5NQhtlGjRjFw4EBuvPFG2rVrx6RJk8jOzrbd1TZgwABq1KjB+PHjAejVqxcTJkzg+uuvtw2xvfrqq/Tq1csWlIYOHcqsWbP43//+h7+/P8nJyQAEBgbi4+NDYmIis2bNomfPnlSpUoW///6bkSNH0qlTJ1q2bOmcb4SIiIiUK04NSA888ABHjhzhtddeIzk5mdatW7Nw4ULbxO2kpCS7HqOxY8diMpkYO3YsBw8eJCQkhF69evF///d/tjaffPIJYF0M8nzTp09n0KBBeHp6smTJElsYCw8Pp0+fPowdO9bxFywiIiIuQc9iKyE9i01ERMT1FPf3t9MfNSIiIiJS3iggiYiIiFxAAUlERETkAgpIIiIiIhdwqZW0y5Ozc9v1yBERERHXcfb39uXuUVNAKqETJ04AEB4e7uRKRERE5EqdOHGCwMDAi76v2/xLyGKxcOjQIfz9/TGZTKV23MzMTMLDw9m/f3+FXT6gol+jrs/1VfRrrOjXBxX/GnV9JWcYBidOnKB69eqFns5xPvUglZCbm9tln/F2Na6Fx5lU9GvU9bm+in6NFf36oOJfo66vZC7Vc3SWJmmLiIiIXEABSUREROQCCkjljJeXF+PGjcPLy8vZpThMRb9GXZ/rq+jXWNGvDyr+Ner6HE+TtEVEREQuoB4kERERkQsoIImIiIhcQAFJRERE5AIKSCIiIiIXUEBygqlTpxIREYG3tzeRkZGsX7/+ku2///57GjdujLe3Ny1atOCXX34po0pL7kquccaMGZhMJruXt7d3GVZ7ZVauXEmvXr2oXr06JpOJ+fPnX3af5cuXc8MNN+Dl5UX9+vWZMWOGw+ssqSu9vuXLlxf6/EwmE8nJyWVT8BUaP348bdu2xd/fn9DQUO655x7i4+Mvu5+r/ByW5Ppc7Wfwk08+oWXLlrZFBNu3b8+vv/56yX1c5fODK78+V/v8LvTPf/4Tk8nEiBEjLtmurD9DBaQyNnv2bEaNGsW4cePYuHEjrVq1Ijo6mtTU1CLb//HHH/Tr14/HHnuMTZs2cc8993DPPfewZcuWMq68+K70GsG6Wurhw4dtr3379pVhxVcmOzubVq1aMXXq1GK137NnD3fccQddu3YlLi6OESNG8Pjjj/Pbb785uNKSudLrOys+Pt7uMwwNDXVQhVdnxYoVDB06lLVr17J48WLy8vLo0aMH2dnZF93HlX4OS3J94Fo/gzVr1uSf//wnsbGx/Pnnn9x6663cfffdbN26tcj2rvT5wZVfH7jW53e+DRs2MG3aNFq2bHnJdk75DA0pU+3atTOGDh1q+3NBQYFRvXp1Y/z48UW2v//++4077rjDbltkZKTx1FNPObTOq3Gl1zh9+nQjMDCwjKorXYAxb968S7Z54YUXjGbNmtlte+CBB4zo6GgHVlY6inN9y5YtMwDj+PHjZVJTaUtNTTUAY8WKFRdt44o/h2cV5/pc+WfwrOuuu874/PPPi3zPlT+/sy51fa76+Z04ccJo0KCBsXjxYqNz587Gs88+e9G2zvgM1YNUhnJzc4mNjSUqKsq2zc3NjaioKNasWVPkPmvWrLFrDxAdHX3R9s5WkmsEyMrKonbt2oSHh1/2X0quxtU+w5Jq3bo11apVo3v37qxevdrZ5RRbRkYGAJUrV75oG1f+DItzfeC6P4MFBQV8++23ZGdn0759+yLbuPLnV5zrA9f8/IYOHcodd9xR6LMpijM+QwWkMpSWlkZBQQFhYWF228PCwi46XyM5OfmK2jtbSa6xUaNGfPHFF/zvf/9j5syZWCwWOnTowIEDB8qiZIe72GeYmZnJqVOnnFRV6alWrRqffvopc+bMYc6cOYSHh9OlSxc2btzo7NIuy2KxMGLECG6++WaaN29+0Xau9nN4VnGvzxV/Bjdv3oyfnx9eXl4MGTKEefPm0bRp0yLbuuLndyXX54qf37fffsvGjRsZP358sdo74zN0d9iRRYqpffv2dv8y6tChA02aNGHatGm89dZbTqxMiqNRo0Y0atTI9ucOHTqQmJjIxIkT+frrr51Y2eUNHTqULVu2sGrVKmeX4hDFvT5X/Bls1KgRcXFxZGRk8MMPPzBw4EBWrFhx0RDhaq7k+lzt89u/fz/PPvssixcvLteTyRWQylBwcDBms5mUlBS77SkpKVStWrXIfapWrXpF7Z2tJNd4IQ8PD66//np27drliBLL3MU+w4CAAHx8fJxUlWO1a9eu3IeOYcOG8fPPP7Ny5Upq1qx5ybau9nMIV3Z9F3KFn0FPT0/q168PQJs2bdiwYQOTJ09m2rRphdq64ud3Jdd3ofL++cXGxpKamsoNN9xg21ZQUMDKlSuZMmUKOTk5mM1mu32c8RlqiK0MeXp60qZNG2JiYmzbLBYLMTExFx1bbt++vV17gMWLF19yLNqZSnKNFyooKGDz5s1Uq1bNUWWWKVf7DEtDXFxcuf38DMNg2LBhzJs3j6VLl1KnTp3L7uNKn2FJru9CrvgzaLFYyMnJKfI9V/r8LuZS13eh8v75devWjc2bNxMXF2d73XjjjTz00EPExcUVCkfgpM/QYdO/pUjffvut4eXlZcyYMcPYtm2b8eSTTxpBQUFGcnKyYRiG8cgjjxgvvfSSrf3q1asNd3d34/333ze2b99ujBs3zvDw8DA2b97srEu4rCu9xjfeeMP47bffjMTERCM2NtZ48MEHDW9vb2Pr1q3OuoRLOnHihLFp0yZj06ZNBmBMmDDB2LRpk7Fv3z7DMAzjpZdeMh555BFb+927dxu+vr7G888/b2zfvt2YOnWqYTabjYULFzrrEi7pSq9v4sSJxvz5842EhARj8+bNxrPPPmu4ubkZS5YscdYlXNLTTz9tBAYGGsuXLzcOHz5se508edLWxpV/Dktyfa72M/jSSy8ZK1asMPbs2WP8/fffxksvvWSYTCZj0aJFhmG49udnGFd+fa72+RXlwrvYysNnqIDkBB999JFRq1Ytw9PT02jXrp2xdu1a23udO3c2Bg4caNf+u+++Mxo2bGh4enoazZo1MxYsWFDGFV+5K7nGESNG2NqGhYUZPXv2NDZu3OiEqovn7G3tF77OXtPAgQONzp07F9qndevWhqenp1G3bl1j+vTpZV53cV3p9b377rtGvXr1DG9vb6Ny5cpGly5djKVLlzqn+GIo6toAu8/ElX8OS3J9rvYz+Oijjxq1a9c2PD09jZCQEKNbt2628GAYrv35GcaVX5+rfX5FuTAglYfP0GQYhuG4/ikRERER16M5SCIiIiIXUEASERERuYACkoiIiMgFFJBERERELqCAJCIiInIBBSQRERGRCyggiYiIiFxAAUlEpBQsX74ck8lEenq6s0sRkVKggCQiIiJyAQUkERERkQsoIIlIhWCxWBg/fjx16tTBx8eHVq1a8cMPPwDnhr8WLFhAy5Yt8fb25qabbmLLli12x5gzZw7NmjXDy8uLiIgIPvjgA7v3c3JyePHFFwkPD8fLy4v69evzn//8x65NbGwsN954I76+vnTo0IH4+HjHXriIOIQCkohUCOPHj+err77i008/ZevWrYwcOZKHH36YFStW2No8//zzfPDBB2zYsIGQkBB69epFXl4eYA02999/Pw8++CCbN2/m9ddf59VXX2XGjBm2/QcMGMB///tfPvzwQ7Zv3860adPw8/Ozq+OVV17hgw8+4M8//8Td3Z1HH320TK5fREqXHlYrIi4vJyeHypUrs2TJEtq3b2/b/vjjj3Py5EmefPJJunbtyrfffssDDzwAwLFjx6hZsyYzZszg/vvv56GHHuLIkSMsWrTItv8LL7zAggUL2Lp1Kzt37qRRo0YsXryYqKioQjUsX76crl27smTJErp16wbAL7/8wh133MGpU6fw9vZ28HdBREqTepBExOXt2rWLkydP0r17d/z8/Gyvr776isTERFu788NT5cqVadSoEdu3bwdg+/bt3HzzzXbHvfnmm0lISKCgoIC4uDjMZjOdO3e+ZC0tW7a0fV2tWjUAUlNTr/oaRaRsuTu7ABGRq5WVlQXAggULqFGjht17Xl5ediGppHx8fIrVzsPDw/a1yWQCrPOjRMS1qAdJRFxe06ZN8fLyIikpifr169u9wsPDbe3Wrl1r+/r48ePs3LmTJk2aANCkSRNWr15td9zVq1fTsGFDzGYzLVq0wGKx2M1pEpGKSz1IIuLy/P39ee655xg5ciQWi4WOHTuSkZHB6tWrCQgIoHbt2gC8+eabVKlShbCwMF555RWCg4O55557ABg9ejRt27blrbfe4oEHHmDNmjVMmTKFjz/+GICIiAgGDhzIo48+yocffkirVq3Yt28fqamp3H///c66dBFxEAUkEakQ3nrrLUJCQhg/fjy7d+8mKCiIG264gZdfftk2xPXPf/6TZ599loSEBFq3bs1PP/2Ep6cnADfccAPfffcdr732Gm+99RbVqlXjzTffZNCgQbZzfPLJJ7z88ss888wzHD16lFq1avHyyy8743JFxMF0F5uIVHhn7zA7fvw4QUFBzi5HRFyA5iCJiIiIXEABSUREROQCGmITERERuYB6kEREREQuoIAkIiIicgEFJBEREZELKCCJiIiIXEABSUREROQCCkgiIiIiF1BAEhEREbmAApKIiIjIBRSQRERERC7w/3eztEn285aWAAAAAElFTkSuQmCC\n"
1560 | },
1561 | "metadata": {}
1562 | }
1563 | ]
1564 | },
1565 | {
1566 | "cell_type": "markdown",
1567 | "metadata": {
1568 | "id": "Haid763z4PL2"
1569 | },
1570 | "source": [
1571 | "## Next Step\n",
1572 | "Of course, making a powerful recommender system requires much more effort. You can read these materials to dive in deeper.\n",
1573 | "\n",
1574 | "
9 | 
26 | 
34 | 
45 | 
49 | 
56 | 
78 | 
86 | 
92 | 
View the code on GitHub
115 | 
Run the code in Google Colab
119 | ![](\"https://www.tensorflow.org/images/colab_logo_32px.png\"){kind=logo}
Run in Google Colab\n",
27 | " ![](\"https://www.tensorflow.org/images/GitHub-Mark-32px.png\")
View source on GitHub\n",
30 | " ![](\"https://raw.githubusercontent.com/xei/recommender-system-tutorial/main/assets/netflix_prize.jpeg\")
\n",
58 | "![](\"https://raw.githubusercontent.com/xei/recommender-system-tutorial/main/assets/recsys_io.png\")
\n",
66 | "![](\"https://raw.githubusercontent.com/xei/recommender-system-tutorial/main/assets/retrieval_ranking.png\")
\n",
77 | "![](\"https://raw.githubusercontent.com/xei/recommender-system-tutorial/main/assets/deepmind_forgoogle_recsys.png\")
\n",
81 | "![](\"https://raw.githubusercontent.com/xei/recommender-system-tutorial/main/assets/cbf_cf.png\")
\n",
88 | "![](\"https://raw.githubusercontent.com/xei/recommender-system-tutorial/main/assets/mf.png\")
\n",
110 | "![](\"https://raw.githubusercontent.com/xei/recommender-system-tutorial/main/assets/tensorflow_two_tower.gif\")
\n",
118 | "![](\"https://raw.githubusercontent.com/xei/recommender-system-tutorial/main/assets/youtube_retrieval.png\")
\n",
124 | "