├── HW ├── hw_1 │ ├── cybersequrity.csv │ └── itmo_dl_course_hw_1.md ├── hw_2 │ └── itmo_dl_course_hw_2.md ├── itmo_dl_course_extra_task_1.md ├── itmo_dl_course_extra_task_2.md ├── itmo_dl_course_extra_task_3.md └── itmo_dl_course_extra_task_4.md ├── Lecture 1 ├── Electrical_Grid_Stability.csv ├── itmo_dl_course_nn_base_01.ipynb └── Занятие 1.pptx ├── Lecture 10 ├── itmo_dl_course_huggingface.ipynb └── itmo_dl_course_nlp_clf_finetune.ipynb ├── Lecture 11-12 ├── itmo_dl_course_explainability.ipynb ├── itmo_dl_course_nlp_augmentation.ipynb ├── itmo_dl_course_nlp_knowledge_distillation.ipynb └── itmo_dl_cv_augmentation.ipynb ├── Lecture 2 ├── Electrical_Grid_Stability.csv ├── itmo_dl_course_nn_base_01.ipynb └── itmo_dl_course_nn_base_backprop.ipynb ├── Lecture 3 └── itmo_dl_course_nn_base_autograd.ipynb ├── Lecture 4-5 ├── Electrical_Grid_Stability.csv ├── itmo_dl_course_nn_activations_and_init.ipynb └── itmo_dl_course_nn_base_optimizers.ipynb ├── Lecture 6 └── itmo_dl_course_nn_dropout_and_bn.ipynb ├── Lecture 7 ├── itmo_dl_course_cnn.ipynb └── itmo_dl_course_pytorch_losses.ipynb ├── Lecture 8 └── itmo_dl_lightning_base.ipynb ├── Lecture 9 ├── itmo_dl_course__cnn_bn_and_dropout.ipynb └── itmo_dl_course__nlp_basics.ipynb └── readme.md /HW/hw_1/itmo_dl_course_hw_1.md: -------------------------------------------------------------------------------- 1 | ### Домашнее задание 1 - 10 баллов 2 | 3 | 1. Загрузите датасет артефактов вредоносного ПО, хранящихся в памяти - [cybersecurity.csv](cybersequrity.csv) 4 | 2. Подготовьте данные к обучению - **1 балл** 5 | - Разделите датасет на обучающую, валидационную и тестовую выборки со стратификацией в пропорции 60/20/20. В качестве целевой переменной используйте столбец `Class` - бинарный столбец (0 - нейтральное ПО, 1 - вредоносное ПО). 6 | - **hint**: для ускорения сходимости и стабилизации обучения можно стандартизировать входные признаки, например, с помощью `StandardScaler` 7 | - Создайте объекты для работы с данными в PyTorch - `Dataset` и `DataLoader` для обучающей, валидационной и тестовой выборок. Выберите оптимальный, на ваш взгляд, `batch_size`. 8 | 3. Реализуйте класс бейзлайновой нейросетевой модели (MLP) для решения задачи - **2 балла** 9 | - Определите оптимальное количество `Linear` слоев в структуре и их размерности. 10 | - Подберите оптимальные для задачи функции активации - `ReLU`, `Sigmoid`, `Tanh`, `LeakyReLU`... 11 | - Реализуйте логику прохождения данных по сети в методе `forward` 12 | - Cоздайте объект модели, реализуйте перевод модели на gpu 13 | 4. Напишите код цикла обучения - train-loop и валидации - eval-loop. В процессе обучения сохраняйте и визуализируйте на графике динамику функции потерь на тренировочной и валидационной выборках - **1 балл** 14 | 5. Обучите модель и проверьте ее качество - **1 балл** 15 | - Выберите оптимизатор, в качестве функции потерь используйте `nn.BCELoss` 16 | - Запустите обучение, постарайтесь подобрать оптимальные скорость обучения и количество эпох, ориентируясь на динамику функции потерь на train/val 17 | - Измерьте качество лучшей модели на тестовой выборке, постройте отчет о классификации - `classification_report` 18 | 6. Улучшите архитектуру модели из пункта 3 и добейтесь увеличения качества - **3 балла** 19 | - Попробуйте добавить слои `BatchNorm1d` и `Dropout` - поэкспериментируйте с их "расположением" в сети, напишите свои выводы о целесообразности их добавления в модель, оптимальном расположении, влиянии на качество/сходимость обучения. При использовании слоя `Dropout` - определите экспериментально или обоснуйте теоретически оптимальное значение параметра `p`. 20 | - Обучите улучшенную модель и проверьте ее качество (при необходимости измените гиперпараметры обучения - `batch_size`, скорость обучения и количество эпох, ориентируясь на динамику функции потерь на train/val и архитектуру модели) 21 | 22 | **Общее** 23 | 24 | - Принимаемые решения обоснованы (почему выбрана определенная архитектура/гиперпараметр/оптимизатор/преобразование и т.п.) - **1 балл** 25 | - Обеспечена воспроизводимость решения: зафиксированы random_state, ноутбук воспроизводится от начала до конца без ошибок - **1 балл** 26 | 27 | **Формат сдачи ДЗ** 28 | 29 | - Создать отдельный репозиторий (если еще не создавали) 30 | - Выдать доступ в репозиторий своему ментору и pacifikus (распределение по менторам и их ники на гитхабе есть в [таблице](https://docs.google.com/spreadsheets/d/1qneC-kHlNzgkCzRoGA9dgUbDthyQd-FA3GhvLCMQY_M/edit?usp=sharing) 31 | - Каждая домашняя работа – PR в отдельную ветку **hw_n**, где **n** - номер домашней работы 32 | - Добавить ментора и pacifikus в reviewers 33 | - Дождаться ревью, если все ок – мержим в main 34 | - Если не ок – вносим исправления и снова отправляем на ревью 35 | -------------------------------------------------------------------------------- /HW/hw_2/itmo_dl_course_hw_2.md: -------------------------------------------------------------------------------- 1 | ### Домашнее задание 2 - 10 баллов + 2 бонусных 2 | 3 | В этом задании вам предстоит воспроизвести и обучить одну из самых знаменитых архитектур среди CV-моделей - AlexNet. 4 | 5 | ![image](https://github.com/user-attachments/assets/7073f6d5-397e-4537-8b98-07a649673c0c) 6 | 7 | 8 | Данная модель была представлена в статье [ImageNet Classification with Deep Convolutional 9 | Neural Networks](https://proceedings.neurips.cc/paper_files/paper/2012/file/c399862d3b9d6b76c8436e924a68c45b-Paper.pdf). 10 | 11 | Подробное описание архитектуры приведено в секции статьи `3.5 Overall Architecture`. 12 | 13 | 1. Загрузите и подготовьте данные к обучению - **2 балла** 14 | - В отличие от оригинальной статьи, в данном задании модель будет применятся для классификации изображений из датасета [CIFAR100](https://pytorch.org/vision/main/generated/torchvision.datasets.CIFAR100.html#torchvision.datasets.CIFAR100) - загрузите его из torchvision, разделите датасет на обучающую, валидационную и тестовую выборки. 15 | 2. Реализуйте класс модели AlexNet для решения задачи и обучите ее, продемонстрировав уменьшение функции потерь на train/validation данных, а также измерьте релевантную, на ваш взгляд метрику классификации на test-части данных - **7 баллов** 16 | 17 | 18 | **Общее** 19 | 20 | Обеспечена воспроизводимость решения: зафиксированы random_state, ноутбук воспроизводится от начала до конца без ошибок - **1 балл** 21 | 22 | 23 | **Дополнительные баллы** 24 | 25 | Все, описанное выше, реализовано на pytorch lightning - **2 балла** 26 | 27 | -------------------------------------------------------------------------------- /HW/itmo_dl_course_extra_task_1.md: -------------------------------------------------------------------------------- 1 | ### Дополнительное задание 1 - 5 баллов 2 | 3 | 1. Загрузите датасет оценки качества воздуха в различных регионах - [air_quality.csv](https://disk.yandex.ru/d/29AVoENYZR4NyA) 4 | 2. Подготовьте данные к обучению - **1 балл** 5 | - Разделите датасет на обучающую, валидационную и тестовую выборки со стратификацией. В качестве целевой переменной используйте столбец `air_quality` - бинарный столбец (0 - плохое качество воздуха, 1 - хорошее качество воздуха). 6 | - Создайте объекты для работы с данными в PyTorch - `Dataset` и `DataLoader` для обучающей, валидационной и тестовой выборок. Выберите оптимальный, на ваш взгляд, `batch_size`. 7 | 3. Реализуйте класс нейросетевой модели для решения задачи - **1 балл** 8 | - Минимальное количество `Linear` слоев в структуре - 3 штуки: входной слой, скрытый слой, выходной классификационный слой. 9 | - Подберите оптимальные для задачи функции активации - `ReLU`, `Sigmoid`, `Tanh`, `LeakyReLU`... 10 | - Реализуйте логику прохождения данных по сети в методе `forward` 11 | - Cоздайте объект модели, реализуйте перевод модели на gpu 12 | 4. Напишите код цикла обучения - train-loop и валидации - eval-loop. В процессе обучения сохраняйте значения функции потерь на тренировочной и валидационной выборках - **1 балл** 13 | 5. Обучите модель и проверьте ее качество - **1 балл** 14 | - Выберите оптимизатор, в качестве функции потерь используйте `nn.BCELoss` 15 | - Запустите обучение, постарайтесь подобрать оптимальные скорость обучения и количество эпох, ориентируясь на динамику функции потерь на train/val 16 | - Измерьте качество лучшей модели на тестовой выборке, постройте отчет о классификации - `classification_report` 17 | 18 | Обеспечена воспроизводимость решения: зафиксированы random_state, ноутбук воспроизводится от начала до конца без ошибок - **1 балл** 19 | -------------------------------------------------------------------------------- /HW/itmo_dl_course_extra_task_2.md: -------------------------------------------------------------------------------- 1 | ### Дополнительное задание 2 - 5 баллов 2 | 3 | Реализуйте модель для классификации изображений датасета [FashionMNIST](https://pytorch.org/vision/0.20/generated/torchvision.datasets.FashionMNIST.html#torchvision.datasets.FashionMNIST) на PyTorch Lightning 4 | 1. **1 балл** Создайте класс `FashionMNISTDataModule`, реализуйте в нем: 5 | - загрузку данных, 6 | - предобработку (перевод в тензоры, нормализация, etc 7 | - разбиение на train/val/test части 8 | - создание dataloader'ов- **1 балл** 9 | 2. **2 балла** Создайте класс модели `FashionMNIST` (наследник `LightningModule`), реализуйте в нем: 10 | - training_step, validation_step, test_step 11 | - расчет метрик на валидации и тестировании из TorchMetrics: F1, ROC AUC 12 | - логирование метрик и функций потерь на каждой эпохе валидации/теста 13 | - подберите подходящие, на ваш взгляд, optimizer и lr-scheduler, а также их гиперпараметры 14 | 3. **1 балл** Обучите модель с помощью trainer'а: 15 | - добавьте `EarlyStopping` 16 | - реализуйте визуализацию логов через tensorboard 17 | - проверьте качество на тестовой части данных 18 | 19 | 20 | Обеспечена воспроизводимость решения: зафиксированы random_state, ноутбук воспроизводится от начала до конца без ошибок - **1 балл** 21 | -------------------------------------------------------------------------------- /HW/itmo_dl_course_extra_task_3.md: -------------------------------------------------------------------------------- 1 | ### Дополнительное задание 3 - 5 баллов 2 | 3 | В этом задании вам предстоит обучить две модели для предсказания рейтинга отзывов об отелях. 4 | 5 | 1. Загрузите датасет отзывов - [Trip Advisor Hotel Reviews](https://www.kaggle.com/datasets/andrewmvd/trip-advisor-hotel-reviews) 6 | 2. Подготовьте данные к обучению - **1 балл** 7 | - Разделите датасет на обучающую, валидационную и тестовую выборки со стратификацией в пропорции 60/20/20. 8 | - Создайте `Dataset` и `DataLoader` для обучающей, валидационной и тестовой выборок. Выберите оптимальный, на ваш взгляд, `batch_size`. 9 | 3. Реализуйте и обучите сверточную сеть (сеть с использованием слоев Conv1d) для решения задачи - **1 балл** 10 | 4. Реализуйте и обучите рекуррентную сеть (сеть с использованием LSTM-слоя) для решения задачи - **1 балл** 11 | 5. Сравните между собой метрики и динамику обучения обеих моделей - **1 балл** 12 | 13 | **Общее** 14 | 15 | Обеспечена воспроизводимость решения: зафиксированы random_state, ноутбук воспроизводится от начала до конца без ошибок - **1 балл** 16 | -------------------------------------------------------------------------------- /HW/itmo_dl_course_extra_task_4.md: -------------------------------------------------------------------------------- 1 | ### Дополнительное задание 4 - 5 баллов 2 | 3 | В этом задании вам предстоит обучить трансформерную модель для классификации текстов. 4 | 5 | 1. Выберите набор данных для классификации текстов из HuggingFace Datasets - **0.5 балла** 6 | 2. Выберите модель архитектуры, подходящей для задачи классификации, в HuggingFace Hub (убедитесь, что она поддерживает язык вашего датасета) - **0.5 балла** 7 | 3. Измерьте качество классификации моделью as-is перед дообучением, с учетом метрик, специфичных для задачи классификации - **0.5 балла** 8 | 4. Дообучите выбранную модель - **2 балла** 9 | 5. Сравните качество до и после дообучения - **0.5 балла** 10 | 11 | **Общее** 12 | 13 | Обеспечена воспроизводимость решения: зафиксированы random_state, ноутбук воспроизводится от начала до конца без ошибок - **1 балл** 14 | -------------------------------------------------------------------------------- /Lecture 1/Занятие 1.pptx: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/pacifikus/itmo_dl_course/2d943148b39a576288ef48dadd8becdd9e6819ba/Lecture 1/Занятие 1.pptx -------------------------------------------------------------------------------- /Lecture 2/itmo_dl_course_nn_base_backprop.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "nbformat": 4, 3 | "nbformat_minor": 0, 4 | "metadata": { 5 | "colab": { 6 | "provenance": [], 7 | "toc_visible": true 8 | }, 9 | "kernelspec": { 10 | "name": "python3", 11 | "display_name": "Python 3" 12 | }, 13 | "language_info": { 14 | "name": "python" 15 | } 16 | }, 17 | "cells": [ 18 | { 19 | "cell_type": "markdown", 20 | "metadata": { 21 | "id": "hlHeqIYvV1xj" 22 | }, 23 | "source": [ 24 | "### Обучение нейронной сети" 25 | ] 26 | }, 27 | { 28 | "cell_type": "markdown", 29 | "metadata": { 30 | "id": "mDPK9XdUV1xj" 31 | }, 32 | "source": [ 33 | "На практике обучение нейронных сетей (подбор значений весов) производится при помощи **метода градиентного спуска**.\n", 34 | "\n", 35 | "Обучение заключается в **минимизации функции потерь по обучаемым параметрам нейронной сети** — весам и смещениям.\n", 36 | "\n", 37 | "Для минимизации функции потерь методом градиентного спуска **необходимо уметь вычислять градиент функции потерь по всем обучаемым параметрам модели**." 38 | ] 39 | }, 40 | { 41 | "cell_type": "markdown", 42 | "metadata": { 43 | "id": "Xwql9YztV1xj" 44 | }, 45 | "source": [ 46 | "#### Прямое и обратное распространение" 47 | ] 48 | }, 49 | { 50 | "cell_type": "markdown", 51 | "metadata": { 52 | "id": "fpqIKuuKV1xj" 53 | }, 54 | "source": [ 55 | "Процесс расчета градиента функции потерь по обучаемым параметрам состоит из двух этапов: **прямого и обратного распространения**." 56 | ] 57 | }, 58 | { 59 | "cell_type": "markdown", 60 | "metadata": { 61 | "id": "UMPriZaLV1xk" 62 | }, 63 | "source": [ 64 | "
" 65 | ] 66 | }, 67 | { 68 | "cell_type": "markdown", 69 | "metadata": { 70 | "id": "GthGfQhXV1xk" 71 | }, 72 | "source": [ 73 | "Во время **прямого распространения** (forward pass) производится расчет значений на выходе модели $y_{pred}$, которые передаются в функцию потерь $\\text{Loss}$ для сравнения с целевыми значениями $y_{true}$.\n", 74 | "\n", 75 | "$$\\large y_{pred}=\\text{model}\\left(x, 𝐖\\right)$$\n", 76 | "\n", 77 | "$$\\large L=\\text{Loss}\\left(y_{true}, y_{pred}\\right)$$" 78 | ] 79 | }, 80 | { 81 | "cell_type": "markdown", 82 | "metadata": { 83 | "id": "KNpjlopVV1xk" 84 | }, 85 | "source": [ 86 | "Значение функции потерь зависит от целевых значений $y_{true}$, входных данных $x$ и параметров модели $𝐖$.\n", 87 | "\n", 88 | "$$\\large L=\\text{Loss}\\left(y_{true}, \\text{model}\\left(x, 𝐖\\right)\\right) = f\\left(y_{true}, x, 𝐖\\right)$$" 89 | ] 90 | }, 91 | { 92 | "cell_type": "markdown", 93 | "metadata": { 94 | "id": "LJa-CmZrV1xk" 95 | }, 96 | "source": [ 97 | "А значит, если модель и функция потерь дифференцируемы, мы можем посчитать $\\nabla_𝐖L$ — градиент функции потерь по обучаемым параметрам. Для этого нужен этап **обратного распространения** (backward pass)." 98 | ] 99 | }, 100 | { 101 | "cell_type": "markdown", 102 | "metadata": { 103 | "id": "C4BUDZ3-V1xl" 104 | }, 105 | "source": [ 106 | "
" 107 | ] 108 | }, 109 | { 110 | "cell_type": "markdown", 111 | "metadata": { 112 | "id": "S5oa_irAV1xl" 113 | }, 114 | "source": [ 115 | "#### Метод обратного распространения ошибки" 116 | ] 117 | }, 118 | { 119 | "cell_type": "markdown", 120 | "source": [ 121 | "[Learning Internal Representations by Error Propagation](https://stanford.edu/~jlmcc/papers/PDP/Volume%201/Chap8_PDP86.pdf)" 122 | ], 123 | "metadata": { 124 | "id": "0W6hJCLenEGs" 125 | } 126 | }, 127 | { 128 | "cell_type": "markdown", 129 | "metadata": { 130 | "id": "3RBKnpKCV1xl" 131 | }, 132 | "source": [ 133 | "Для эффективного численного расчета градиента функции потерь по обучаемым параметрам модели применяется **метод обратного распространения ошибки (backpropagation)**. Благодаря данному методу становится практически возможным использование метода градиентного спуска для проведения процедуры обучения." 134 | ] 135 | }, 136 | { 137 | "cell_type": "markdown", 138 | "metadata": { 139 | "id": "cicyQDR-V1xm" 140 | }, 141 | "source": [ 142 | "Метод обратного распространения ошибки использует структуру многослойной нейронной сети как сложной функции, применяя **правило дифференцирования сложной функции** для вычисления градиента от функции потерь по весам сети. Градиент от функции потерь вычисляется при движении по нейронной сети от её выходов в направлении входов. Именно такой порядок обхода вычислительного графа и обуславливает название метода." 143 | ] 144 | }, 145 | { 146 | "cell_type": "markdown", 147 | "metadata": { 148 | "id": "EFuMXMeVV1xm" 149 | }, 150 | "source": [ 151 | "#### Вычислительный граф" 152 | ] 153 | }, 154 | { 155 | "cell_type": "markdown", 156 | "metadata": { 157 | "id": "XO57ukyWV1xm" 158 | }, 159 | "source": [ 160 | "По существу, нейронная сеть является сложной функцией, работу которой можно представить как последовательное выполнение математических операций. Такое представление функций называется **вычислительным графом**." 161 | ] 162 | }, 163 | { 164 | "cell_type": "markdown", 165 | "metadata": { 166 | "id": "m7mNdT2JV1xn" 167 | }, 168 | "source": [ 169 | "
" 170 | ] 171 | }, 172 | { 173 | "cell_type": "markdown", 174 | "metadata": { 175 | "id": "86Q-6mpnV1xn" 176 | }, 177 | "source": [ 178 | "Алгоритм обратного распространения ошибки позволяет находить градиенты для любого графа вычислений, если описываемая им функция дифференцируема.\n", 179 | "\n", 180 | "В его основе лежит правило взятия производной сложной функции (chain rule)." 181 | ] 182 | }, 183 | { 184 | "cell_type": "markdown", 185 | "metadata": { 186 | "id": "N1Iu7dVTV1xq" 187 | }, 188 | "source": [ 189 | "#### Пошаговый разбор метода обратного распространения" 190 | ] 191 | }, 192 | { 193 | "cell_type": "markdown", 194 | "metadata": { 195 | "id": "bVFH1gNmV1xq" 196 | }, 197 | "source": [ 198 | "Пусть в каком-то узле графа производится вычисление\n", 199 | "\n", 200 | "$$\\large z = f(x, y),$$\n", 201 | "\n", 202 | "и далее результат вычисления $\\large z$ используется для вычисления функции $\\large L(z)=L(f(x, y))$.\n", 203 | "\n", 204 | "Тогда правило вычисления производных $\\dfrac{\\partial L}{\\partial x}$ и $\\dfrac{\\partial L}{\\partial y}$ можно представить следующим образом:" 205 | ] 206 | }, 207 | { 208 | "cell_type": "markdown", 209 | "metadata": { 210 | "id": "hUzYkiG2V1xq" 211 | }, 212 | "source": [ 213 | "
" 214 | ] 215 | }, 216 | { 217 | "cell_type": "markdown", 218 | "metadata": { 219 | "id": "uei3RxpDV1xr" 220 | }, 221 | "source": [ 222 | "Рассмотрим следующую функцию:\n", 223 | "\n", 224 | "$$\\Large f(w,x)=\\frac{1}{1+e^{-(w_0x_0+w_1x_1+w_2)}}$$\n", 225 | "\n", 226 | "Представим ее в виде вычислительного графа, состоящего из элементарных операций, от которых просто берутся производные:" 227 | ] 228 | }, 229 | { 230 | "cell_type": "markdown", 231 | "metadata": { 232 | "id": "yT4ij4EWV1xs" 233 | }, 234 | "source": [ 235 | "
" 236 | ] 237 | }, 238 | { 239 | "cell_type": "markdown", 240 | "metadata": { 241 | "id": "-ecWG8s8V1xs" 242 | }, 243 | "source": [ 244 | "На примере данной функции рассмотрим алгоритм обратного распространения ошибки и найдём величину её градиента по параметрам $\\large w$.\n", 245 | "Нам потребуется вычислить частные производные $\\dfrac{\\partial f}{\\partial w_0}, \\dfrac{\\partial f}{\\partial w_1}, \\dfrac{\\partial f}{dw_2}, \\dfrac{\\partial f}{\\partial x_0}$ и $\\dfrac{\\partial f}{\\partial x_1}$.\n", 246 | "\n", 247 | "Пусть \"веса\" $w$ инициализированы значениями $w_0=2,\\;w_1=-3,\\;w_2=-3$, а \"входные признаки\" $x$ принимают значения $x_0=-1,\\;x_1=-2$.\n", 248 | "\n", 249 | "Делая прямой проход через граф вычислений для данной функции, получаем её значение для заданных $w$ и $x$ равным $f=0.73$:" 250 | ] 251 | }, 252 | { 253 | "cell_type": "markdown", 254 | "metadata": { 255 | "id": "ZoGOHSmIV1xt" 256 | }, 257 | "source": [ 258 | "
" 259 | ] 260 | }, 261 | { 262 | "cell_type": "markdown", 263 | "metadata": { 264 | "id": "c55zvjEQV1xv" 265 | }, 266 | "source": [ 267 | "Далее, в соответствии с алгоритмом обратного распространения ошибки, рассчитаем частные производные, пройдясь последовательно по графу вычислений, постепенно накапливая искомое значение для градиента функции.\n", 268 | "\n", 269 | "Для начала зададим $\\dfrac{df}{df}=1$.\n", 270 | "\n", 271 | "Начинаем обратный проход по графу вычислений. Первая вершина содержит функцию $f(x)=\\dfrac{1}{x}$, производная которой равна $\\dfrac{df}{dx}=-\\dfrac{1}{x^2}$" 272 | ] 273 | }, 274 | { 275 | "cell_type": "markdown", 276 | "metadata": { 277 | "id": "Keqd3OIYV1xw" 278 | }, 279 | "source": [ 280 | "
\n", 281 | "\n", 282 | "$$\\large f(x)=\\frac1x \\quad \\longrightarrow \\quad \\frac{df}{dx} = -\\frac{1}{x^2}$$" 283 | ] 284 | }, 285 | { 286 | "cell_type": "markdown", 287 | "metadata": { 288 | "id": "IZBqSFS0V1xx" 289 | }, 290 | "source": [ 291 | "В следующем узле находится функция $f(x)=1+x$. Производная от выражения в данном узле равняется $\\dfrac{df}{dx}=1$:" 292 | ] 293 | }, 294 | { 295 | "cell_type": "markdown", 296 | "metadata": { 297 | "id": "SO5Q6KXRV1xy" 298 | }, 299 | "source": [ 300 | "
\n", 301 | "\n", 302 | "$$\\large f(x)=c+x \\quad \\longrightarrow \\quad \\frac{df}{dx} = 1$$" 303 | ] 304 | }, 305 | { 306 | "cell_type": "markdown", 307 | "metadata": { 308 | "id": "ahPPNDQbV1xz" 309 | }, 310 | "source": [ 311 | "Третья вершина содержит экспоненту $f(x)=e^x$. Её производная также является экспонентой $\\dfrac{df}{dx}=e^x$:" 312 | ] 313 | }, 314 | { 315 | "cell_type": "markdown", 316 | "metadata": { 317 | "id": "yhxEF0dyV1x0" 318 | }, 319 | "source": [ 320 | "
\n", 321 | "\n", 322 | "$$\\large f(x)=e^x \\quad \\longrightarrow \\quad \\frac{df}{dx} = e^x$$" 323 | ] 324 | }, 325 | { 326 | "cell_type": "markdown", 327 | "metadata": { 328 | "id": "60CPzx3UV1x2" 329 | }, 330 | "source": [ 331 | "Следующая вершина, четвертая, содержит умножение на константу $f(x)=ax$. Производная равна $\\dfrac{df}{dx}=a$ (в данном случае $a=-1$):" 332 | ] 333 | }, 334 | { 335 | "cell_type": "markdown", 336 | "metadata": { 337 | "id": "xx9Ec7-lV1x2" 338 | }, 339 | "source": [ 340 | "
\n", 341 | "\n", 342 | "$$\\large f(x)=ax \\quad \\longrightarrow \\quad \\frac{df}{dx} = a$$" 343 | ] 344 | }, 345 | { 346 | "cell_type": "markdown", 347 | "metadata": { 348 | "id": "8ZlariWFV1x3" 349 | }, 350 | "source": [ 351 | "Двигаясь по графу вычислений, мы дошли до узла суммирования, который имеет два входа. Относительно каждого из входов локальный градиент в вершине суммирования будет равен $1$:\n", 352 | "$$\\large f(x,y)=x+y \\quad \\Rightarrow \\quad \\frac{\\partial f}{\\partial x}=1 \\quad \\quad \\frac{\\partial f}{\\partial y}=1$$\n", 353 | "Так как умножение на единицу не изменит значения входного градиента, всем входам узла суммирования мы можем приписать точно такое же значение входного градиента ($0.2$), что мы имели и для самого узла суммирования. Будем действовать аналогично и со всеми остальными узлами суммирования, которые встретятся нам в вычислительном графе." 354 | ] 355 | }, 356 | { 357 | "cell_type": "markdown", 358 | "metadata": { 359 | "id": "D3yCVlcTV1x3" 360 | }, 361 | "source": [ 362 | "
" 363 | ] 364 | }, 365 | { 366 | "cell_type": "markdown", 367 | "metadata": { 368 | "id": "bPWB4baZV1x3" 369 | }, 370 | "source": [ 371 | "Двигаясь далее к началу графа вычислений, мы подходим к вершинам умножения. Для такой вершины локальный градиент по отношению к какому-либо из входов будет равен значению оставшегося входа. Остается умножить локальный градиент на входящий.\n", 372 | "\n", 373 | "$$\\large f(w,x)=wx \\quad \\Rightarrow \\quad \\frac{\\partial f}{\\partial w}=x \\quad \\quad \\frac{\\partial f}{\\partial x}=w$$\n", 374 | "\n", 375 | "Точно так же мы можем поступить и с оставшейся второй вершиной умножения, которая привязана к $w_1$ и $x_1$:" 376 | ] 377 | }, 378 | { 379 | "cell_type": "markdown", 380 | "metadata": { 381 | "id": "bpB4pfBsV1x4" 382 | }, 383 | "source": [ 384 | "
" 385 | ] 386 | }, 387 | { 388 | "cell_type": "markdown", 389 | "metadata": { 390 | "id": "vb9pNHczV1x5" 391 | }, 392 | "source": [ 393 | "Так, двигаясь по графу вычислений в обратном направлении от выхода функции к входным аргументам, мы последовательно для каждого узла умножаем локальный градиент на входящий градиент, используя цепное правило дифференцирования сложной функции. В описанном примере мы полностью разбили граф вычислений на отдельные элементарные узлы. Разбиение вычислительного графа на элементарные узлы вовсе не обязательно — мы можем сгруппировать несколько вершин вместе, если они образуют дифференцируемую функцию, от которой \"удобно\" брать производную, и рассматривать их совместно." 394 | ] 395 | }, 396 | { 397 | "cell_type": "markdown", 398 | "metadata": { 399 | "id": "BD6embMcV1x6" 400 | }, 401 | "source": [ 402 | "В нашем примере мы можем заметить, что вычислительный граф можно свести к двум операциям: получению выражения $w_0x_0+w_1x_1+w_2$ и последующему вычислению от него сигмоидальной функции.\n", 403 | "\n", 404 | "Функция сигмоиды:\n", 405 | "\n", 406 | "$$\\large \\displaystyle \\sigma(x) = \\frac{1}{1+e^{-x}}.$$" 407 | ] 408 | }, 409 | { 410 | "cell_type": "markdown", 411 | "metadata": { 412 | "id": "qeRQj6F1V1x6" 413 | }, 414 | "source": [ 415 | "Важно отметить, что сигмоида обладает важным свойством: её производная может быть выражена через саму сигмоидальную функцию:\n", 416 | "\n", 417 | "$$\\large \\frac{d}{dx}\\sigma(x) = \\frac{d}{dx}(1+e^{-x})^{-1} = \\frac{e^{-x}}{(1+e^{-x})^{2}} = \\frac{1}{1+e^{-x}} \\cdot \\frac{1+e^{-x}-1}{1+e^{-x}} = \\sigma(x)\\cdot(1-\\sigma(x))$$" 418 | ] 419 | }, 420 | { 421 | "cell_type": "markdown", 422 | "metadata": { 423 | "id": "i4sesWufV1x7" 424 | }, 425 | "source": [ 426 | "
" 427 | ] 428 | }, 429 | { 430 | "cell_type": "code", 431 | "source": [], 432 | "metadata": { 433 | "id": "r-mf0yXMK4ag" 434 | }, 435 | "execution_count": null, 436 | "outputs": [] 437 | } 438 | ] 439 | } -------------------------------------------------------------------------------- /Lecture 3/itmo_dl_course_nn_base_autograd.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "nbformat": 4, 3 | "nbformat_minor": 0, 4 | "metadata": { 5 | "colab": { 6 | "provenance": [], 7 | "toc_visible": true 8 | }, 9 | "kernelspec": { 10 | "name": "python3", 11 | "display_name": "Python 3" 12 | }, 13 | "language_info": { 14 | "name": "python" 15 | } 16 | }, 17 | "cells": [ 18 | { 19 | "cell_type": "markdown", 20 | "metadata": { 21 | "id": "9NSTCyMhHbSq" 22 | }, 23 | "source": [ 24 | "### PyTorch Autograd - automatic differentiation engine\n", 25 | "\n", 26 | "[PyTorch 101, Part 1: Understanding Graphs, Automatic Differentiation and Autograd](https://blog.paperspace.com/pytorch-101-understanding-graphs-and-automatic-differentiation/)\n", 27 | "\n", 28 | "В центре большинства современных приемов машинного обучения лежит расчет градиентов. Это в особенности касается нейронных сетей, где для обновления весовых коэффициентов используется алгоритм обратного распространения\n", 29 | "\n", 30 | "Autograd предоставляет классы и функции, реализующие автоматическое дифференцирование произвольных скалярных функций. Это требует минимальных изменений в существующем коде - нужно только объявить Tensor, для которого должны вычисляться градиенты, с атрибутом `requires_grad=True`" 31 | ] 32 | }, 33 | { 34 | "cell_type": "code", 35 | "source": [ 36 | "import torch" 37 | ], 38 | "metadata": { 39 | "id": "q1TDxt6qamOm" 40 | }, 41 | "execution_count": 1, 42 | "outputs": [] 43 | }, 44 | { 45 | "cell_type": "code", 46 | "execution_count": 2, 47 | "metadata": { 48 | "colab": { 49 | "base_uri": "https://localhost:8080/" 50 | }, 51 | "id": "JcdJCnP5HbSr", 52 | "outputId": "49c2401a-98ab-4a60-d534-1105a6aed240" 53 | }, 54 | "outputs": [ 55 | { 56 | "output_type": "stream", 57 | "name": "stdout", 58 | "text": [ 59 | "tensor([[1., 1.],\n", 60 | " [1., 1.]], requires_grad=True)\n" 61 | ] 62 | } 63 | ], 64 | "source": [ 65 | "x = torch.ones(2, 2, requires_grad=True)\n", 66 | "print(x)" 67 | ] 68 | }, 69 | { 70 | "cell_type": "code", 71 | "source": [ 72 | "x.grad == None" 73 | ], 74 | "metadata": { 75 | "colab": { 76 | "base_uri": "https://localhost:8080/" 77 | }, 78 | "id": "JbgJ9x8CcFJ6", 79 | "outputId": "3c751d58-c5fc-4553-c892-2c74a131a8f7" 80 | }, 81 | "execution_count": 3, 82 | "outputs": [ 83 | { 84 | "output_type": "execute_result", 85 | "data": { 86 | "text/plain": [ 87 | "True" 88 | ] 89 | }, 90 | "metadata": {}, 91 | "execution_count": 3 92 | } 93 | ] 94 | }, 95 | { 96 | "cell_type": "code", 97 | "source": [ 98 | "x.grad_fn == None" 99 | ], 100 | "metadata": { 101 | "colab": { 102 | "base_uri": "https://localhost:8080/" 103 | }, 104 | "id": "b_i-OcPLb-Rv", 105 | "outputId": "8c5957ce-8deb-47fe-eb97-9325816a422c" 106 | }, 107 | "execution_count": 4, 108 | "outputs": [ 109 | { 110 | "output_type": "execute_result", 111 | "data": { 112 | "text/plain": [ 113 | "True" 114 | ] 115 | }, 116 | "metadata": {}, 117 | "execution_count": 4 118 | } 119 | ] 120 | }, 121 | { 122 | "cell_type": "markdown", 123 | "metadata": { 124 | "id": "U0VM8aIlHbSr" 125 | }, 126 | "source": [ 127 | "После применения какой-либо операции к тензору атрибуту `grad_fn` присваивается объект `Function`, который добавляется в граф вычислений для обратного распространения градиента.\n", 128 | "\n" 129 | ] 130 | }, 131 | { 132 | "cell_type": "code", 133 | "execution_count": 5, 134 | "metadata": { 135 | "colab": { 136 | "base_uri": "https://localhost:8080/" 137 | }, 138 | "id": "ZIcFl4AWHbSs", 139 | "outputId": "08016a23-8ade-478b-a0b9-9531c5eb5c9c" 140 | }, 141 | "outputs": [ 142 | { 143 | "output_type": "stream", 144 | "name": "stdout", 145 | "text": [ 146 | "tensor([[3., 3.],\n", 147 | " [3., 3.]], grad_fn=)\n" 148 | ] 149 | } 150 | ], 151 | "source": [ 152 | "y = x + 2\n", 153 | "print(y)" 154 | ] 155 | }, 156 | { 157 | "cell_type": "code", 158 | "execution_count": 6, 159 | "metadata": { 160 | "colab": { 161 | "base_uri": "https://localhost:8080/" 162 | }, 163 | "id": "of4Eh18aHbSs", 164 | "outputId": "86c88fc3-5e36-4ac0-fe4a-e6e9422a4c70" 165 | }, 166 | "outputs": [ 167 | { 168 | "output_type": "stream", 169 | "name": "stdout", 170 | "text": [ 171 | "tensor([[27., 27.],\n", 172 | " [27., 27.]], grad_fn=) tensor(27., grad_fn=)\n" 173 | ] 174 | } 175 | ], 176 | "source": [ 177 | "z = y * y * 3\n", 178 | "out = z.mean()\n", 179 | "\n", 180 | "print(z, out)" 181 | ] 182 | }, 183 | { 184 | "cell_type": "markdown", 185 | "metadata": { 186 | "id": "39gLEvmAHbSt" 187 | }, 188 | "source": [ 189 | "`.grad_fn` может менять \"на лету\"" 190 | ] 191 | }, 192 | { 193 | "cell_type": "code", 194 | "execution_count": 7, 195 | "metadata": { 196 | "colab": { 197 | "base_uri": "https://localhost:8080/" 198 | }, 199 | "id": "va5zd2xMHbSt", 200 | "outputId": "6d40b575-71e2-444a-c356-18abddaa383d" 201 | }, 202 | "outputs": [ 203 | { 204 | "output_type": "stream", 205 | "name": "stdout", 206 | "text": [ 207 | "False\n", 208 | "True\n", 209 | "\n" 210 | ] 211 | } 212 | ], 213 | "source": [ 214 | "a = torch.randn(2, 2)\n", 215 | "a = ((a * 3) / (a - 1))\n", 216 | "print(a.requires_grad)\n", 217 | "a.requires_grad_(True)\n", 218 | "print(a.requires_grad)\n", 219 | "b = (a * a).sum()\n", 220 | "print(b.grad_fn)" 221 | ] 222 | }, 223 | { 224 | "cell_type": "markdown", 225 | "metadata": { 226 | "id": "lZcXFmziHbSt" 227 | }, 228 | "source": [ 229 | "Метод `backward` корневого узла графа вычислений запускает процедуру вычисления градиентов в листовых (is_leaf) узлах, имеющих атрибут requires_grad. Граф дифференцируется по цепочке (chain rule)" 230 | ] 231 | }, 232 | { 233 | "cell_type": "code", 234 | "execution_count": 8, 235 | "metadata": { 236 | "id": "yyNpdWrzHbSu" 237 | }, 238 | "outputs": [], 239 | "source": [ 240 | "out.backward()" 241 | ] 242 | }, 243 | { 244 | "cell_type": "code", 245 | "execution_count": 9, 246 | "metadata": { 247 | "colab": { 248 | "base_uri": "https://localhost:8080/" 249 | }, 250 | "id": "bSpM1ECkHbSu", 251 | "outputId": "67a757d9-b23e-4b93-e1f1-a28fe03cfdd6" 252 | }, 253 | "outputs": [ 254 | { 255 | "output_type": "stream", 256 | "name": "stdout", 257 | "text": [ 258 | "tensor([[4.5000, 4.5000],\n", 259 | " [4.5000, 4.5000]])\n" 260 | ] 261 | } 262 | ], 263 | "source": [ 264 | "print(x.grad)" 265 | ] 266 | }, 267 | { 268 | "cell_type": "markdown", 269 | "source": [ 270 | "По умолчанию промежуточные (не листовые) узлы графа не хранят прошедшие через них градиентов." 271 | ], 272 | "metadata": { 273 | "id": "LpYFZQuze1u-" 274 | } 275 | }, 276 | { 277 | "cell_type": "code", 278 | "source": [ 279 | "print(y.grad)" 280 | ], 281 | "metadata": { 282 | "colab": { 283 | "base_uri": "https://localhost:8080/" 284 | }, 285 | "id": "AEcfJluqe1eQ", 286 | "outputId": "d9192cb7-aa09-43ed-a8d1-c97458858916" 287 | }, 288 | "execution_count": 10, 289 | "outputs": [ 290 | { 291 | "output_type": "stream", 292 | "name": "stdout", 293 | "text": [ 294 | "None\n" 295 | ] 296 | }, 297 | { 298 | "output_type": "stream", 299 | "name": "stderr", 300 | "text": [ 301 | ":1: UserWarning: The .grad attribute of a Tensor that is not a leaf Tensor is being accessed. Its .grad attribute won't be populated during autograd.backward(). If you indeed want the .grad field to be populated for a non-leaf Tensor, use .retain_grad() on the non-leaf Tensor. If you access the non-leaf Tensor by mistake, make sure you access the leaf Tensor instead. See github.com/pytorch/pytorch/pull/30531 for more informations. (Triggered internally at aten/src/ATen/core/TensorBody.h:489.)\n", 302 | " print(y.grad)\n" 303 | ] 304 | } 305 | ] 306 | }, 307 | { 308 | "cell_type": "markdown", 309 | "source": [ 310 | "Эту ситуацию можно изменить, вызвав для для конкретного узла метод retain_grad" 311 | ], 312 | "metadata": { 313 | "id": "6KEkSd3oe7g_" 314 | } 315 | }, 316 | { 317 | "cell_type": "code", 318 | "source": [ 319 | "x = torch.ones(2, 2, requires_grad=True)\n", 320 | "y = x + 2\n", 321 | "y.retain_grad()\n", 322 | "z = y * y * 3\n", 323 | "out = z.mean()\n", 324 | "out.backward()" 325 | ], 326 | "metadata": { 327 | "id": "19q2HQX8e7JW" 328 | }, 329 | "execution_count": 11, 330 | "outputs": [] 331 | }, 332 | { 333 | "cell_type": "code", 334 | "source": [ 335 | "print(x.grad)" 336 | ], 337 | "metadata": { 338 | "colab": { 339 | "base_uri": "https://localhost:8080/" 340 | }, 341 | "id": "LXSMmRATfHVO", 342 | "outputId": "51d5acb3-68a4-43f0-af81-b5ce3c745ea3" 343 | }, 344 | "execution_count": 12, 345 | "outputs": [ 346 | { 347 | "output_type": "stream", 348 | "name": "stdout", 349 | "text": [ 350 | "tensor([[4.5000, 4.5000],\n", 351 | " [4.5000, 4.5000]])\n" 352 | ] 353 | } 354 | ] 355 | }, 356 | { 357 | "cell_type": "code", 358 | "source": [ 359 | "print(y.grad)" 360 | ], 361 | "metadata": { 362 | "colab": { 363 | "base_uri": "https://localhost:8080/" 364 | }, 365 | "id": "S9KDgTYLfEib", 366 | "outputId": "07a9c87e-f288-4bcb-f71c-5c5029c7b755" 367 | }, 368 | "execution_count": 13, 369 | "outputs": [ 370 | { 371 | "output_type": "stream", 372 | "name": "stdout", 373 | "text": [ 374 | "tensor([[4.5000, 4.5000],\n", 375 | " [4.5000, 4.5000]])\n" 376 | ] 377 | } 378 | ] 379 | }, 380 | { 381 | "cell_type": "markdown", 382 | "metadata": { 383 | "id": "CD3WUBM3HbSw" 384 | }, 385 | "source": [ 386 | "Иногда с листовыми узлами необходимо проделать действия, не меняя при этом графа. Такие действия проводят, используя контекстный менедежр `no_grad`, которое блокирует создание новых узлов графа" 387 | ] 388 | }, 389 | { 390 | "cell_type": "code", 391 | "execution_count": 14, 392 | "metadata": { 393 | "colab": { 394 | "base_uri": "https://localhost:8080/" 395 | }, 396 | "id": "ueu6nMZQHbSw", 397 | "outputId": "a0f903ae-aa97-4d2c-d78d-582b9036b5dc" 398 | }, 399 | "outputs": [ 400 | { 401 | "output_type": "stream", 402 | "name": "stdout", 403 | "text": [ 404 | "True\n", 405 | "True\n", 406 | "False\n" 407 | ] 408 | } 409 | ], 410 | "source": [ 411 | "print(x.requires_grad)\n", 412 | "print((x ** 2).requires_grad)\n", 413 | "\n", 414 | "with torch.no_grad(): # потом можно включить вручную torch.enable_grad()\n", 415 | " print((x ** 2).requires_grad)\n" 416 | ] 417 | }, 418 | { 419 | "cell_type": "markdown", 420 | "source": [ 421 | "### Micrograd" 422 | ], 423 | "metadata": { 424 | "id": "-wyL8Sn9apWv" 425 | } 426 | }, 427 | { 428 | "cell_type": "markdown", 429 | "source": [ 430 | "[micrograd](https://github.com/karpathy/micrograd)" 431 | ], 432 | "metadata": { 433 | "id": "Mqba-3uHj2xE" 434 | } 435 | }, 436 | { 437 | "cell_type": "code", 438 | "source": [ 439 | "!git clone https://github.com/karpathy/micrograd.git" 440 | ], 441 | "metadata": { 442 | "id": "YdrSm6g6M4DE", 443 | "colab": { 444 | "base_uri": "https://localhost:8080/" 445 | }, 446 | "outputId": "843768c8-fc3f-42b4-fe09-e19e3fd66899" 447 | }, 448 | "execution_count": 15, 449 | "outputs": [ 450 | { 451 | "output_type": "stream", 452 | "name": "stdout", 453 | "text": [ 454 | "Cloning into 'micrograd'...\n", 455 | "remote: Enumerating objects: 98, done.\u001b[K\n", 456 | "remote: Counting objects: 100% (60/60), done.\u001b[K\n", 457 | "remote: Compressing objects: 100% (22/22), done.\u001b[K\n", 458 | "remote: Total 98 (delta 39), reused 38 (delta 38), pack-reused 38 (from 1)\u001b[K\n", 459 | "Receiving objects: 100% (98/98), 258.88 KiB | 1.18 MiB/s, done.\n", 460 | "Resolving deltas: 100% (44/44), done.\n" 461 | ] 462 | } 463 | ] 464 | }, 465 | { 466 | "cell_type": "code", 467 | "source": [ 468 | "%cd micrograd" 469 | ], 470 | "metadata": { 471 | "colab": { 472 | "base_uri": "https://localhost:8080/" 473 | }, 474 | "id": "3QXCWWD3iuRu", 475 | "outputId": "af4f614d-e9b2-4558-84b9-2e9e7fe760d3" 476 | }, 477 | "execution_count": 16, 478 | "outputs": [ 479 | { 480 | "output_type": "stream", 481 | "name": "stdout", 482 | "text": [ 483 | "/content/micrograd\n" 484 | ] 485 | } 486 | ] 487 | }, 488 | { 489 | "cell_type": "code", 490 | "source": [ 491 | "import random\n", 492 | "import numpy as np\n", 493 | "\n", 494 | "import matplotlib.pyplot as plt\n", 495 | "%matplotlib inline\n", 496 | "\n", 497 | "from graphviz import Digraph\n", 498 | "\n", 499 | "from micrograd.engine import Value\n", 500 | "from micrograd.nn import Neuron, Layer, MLP, Module\n", 501 | "\n", 502 | "from sklearn.datasets import make_moons, make_blobs" 503 | ], 504 | "metadata": { 505 | "id": "OAdhBb7V7G1T" 506 | }, 507 | "execution_count": 18, 508 | "outputs": [] 509 | }, 510 | { 511 | "cell_type": "code", 512 | "source": [ 513 | "np.random.seed(42)\n", 514 | "random.seed(42)" 515 | ], 516 | "metadata": { 517 | "id": "LoaIQx1y7G4g" 518 | }, 519 | "execution_count": 19, 520 | "outputs": [] 521 | }, 522 | { 523 | "cell_type": "markdown", 524 | "source": [ 525 | "#### Пример построения графа вычислений" 526 | ], 527 | "metadata": { 528 | "id": "0Mct2DfemyYb" 529 | } 530 | }, 531 | { 532 | "cell_type": "code", 533 | "source": [ 534 | "def trace(root):\n", 535 | " nodes, edges = set(), set()\n", 536 | " def build(v):\n", 537 | " if v not in nodes:\n", 538 | " nodes.add(v)\n", 539 | " for child in v._prev:\n", 540 | " edges.add((child, v))\n", 541 | " build(child)\n", 542 | " build(root)\n", 543 | " return nodes, edges\n", 544 | "\n", 545 | "def draw_dot(root, format='svg', rankdir='LR'):\n", 546 | " \"\"\"\n", 547 | " format: png | svg | ...\n", 548 | " rankdir: TB (top to bottom graph) | LR (left to right)\n", 549 | " \"\"\"\n", 550 | " assert rankdir in ['LR', 'TB']\n", 551 | " nodes, edges = trace(root)\n", 552 | " dot = Digraph(format=format, graph_attr={'rankdir': rankdir}) #, node_attr={'rankdir': 'TB'})\n", 553 | "\n", 554 | " for n in nodes:\n", 555 | " dot.node(name=str(id(n)), label = \"{ data %.4f | grad %.4f }\" % (n.data, n.grad), shape='record')\n", 556 | " if n._op:\n", 557 | " dot.node(name=str(id(n)) + n._op, label=n._op)\n", 558 | " dot.edge(str(id(n)) + n._op, str(id(n)))\n", 559 | "\n", 560 | " for n1, n2 in edges:\n", 561 | " dot.edge(str(id(n1)), str(id(n2)) + n2._op)\n", 562 | "\n", 563 | " return dot" 564 | ], 565 | "metadata": { 566 | "id": "9wJYCA--mdXl" 567 | }, 568 | "execution_count": 20, 569 | "outputs": [] 570 | }, 571 | { 572 | "cell_type": "code", 573 | "source": [ 574 | "Value??" 575 | ], 576 | "metadata": { 577 | "id": "EDeU_JVImqTv" 578 | }, 579 | "execution_count": 21, 580 | "outputs": [] 581 | }, 582 | { 583 | "cell_type": "markdown", 584 | "source": [ 585 | "Для отслеживания, как считаются градиенты в примере ниже, временно \"пропатчим\" функции `__add__`, `__mul__`, `relu` и `backward` из класса `Value` так, чтобы видеть, в логах как применяются операции" 586 | ], 587 | "metadata": { 588 | "id": "5eRoT_-QIXV1" 589 | } 590 | }, 591 | { 592 | "cell_type": "code", 593 | "source": [ 594 | "old_relu = Value.relu\n", 595 | "old_backward = Value.backward\n", 596 | "old_add = Value.__add__\n", 597 | "old_mul = Value.__mul__\n", 598 | "\n", 599 | "def new_relu(self):\n", 600 | " out = Value(0 if self.data < 0 else self.data, (self,), 'ReLU')\n", 601 | "\n", 602 | " def _backward():\n", 603 | " self.grad += (out.data > 0) * out.grad\n", 604 | " print(f\"\"\"\n", 605 | " Считаем производную relu:\n", 606 | " входной градиент = {out.grad}\n", 607 | " локальный градиент = {int(out.data > 0)}\n", 608 | " выходной градиент = {out.grad} * {int(out.data > 0)} = {self.grad}\n", 609 | " \"\"\"\n", 610 | " )\n", 611 | " out._backward = _backward\n", 612 | "\n", 613 | " return out\n", 614 | "\n", 615 | "def new_mul(self, other):\n", 616 | " other = other if isinstance(other, Value) else Value(other)\n", 617 | " out = Value(self.data * other.data, (self, other), '*')\n", 618 | "\n", 619 | " def _backward():\n", 620 | " self.grad += other.data * out.grad\n", 621 | " other.grad += self.data * out.grad\n", 622 | "\n", 623 | " print(f\"\"\"\n", 624 | " Считаем производную умножения.\n", 625 | " Для умножения локальный градиент по отношению к какому-либо из входов будет равен значению оставшегося входа:\n", 626 | " входной градиент = {out.grad}\n", 627 | " локальный градиент для первого множителя = {other.data}\n", 628 | " локальный градиент для второго множителя = {self.data}\n", 629 | " выходной градиент для первого множителя = {out.grad} * {other.data} = {self.grad}\n", 630 | " выходной градиент для второго множителя = {out.grad} * {self.data} = {other.grad}\n", 631 | " \"\"\"\n", 632 | " )\n", 633 | " out._backward = _backward\n", 634 | "\n", 635 | " return out\n", 636 | "\n", 637 | "\n", 638 | "def new_add(self, other):\n", 639 | " other = other if isinstance(other, Value) else Value(other)\n", 640 | " out = Value(self.data + other.data, (self, other), '+')\n", 641 | "\n", 642 | " def _backward():\n", 643 | " self.grad += out.grad\n", 644 | " other.grad += out.grad\n", 645 | "\n", 646 | " print(f\"\"\"\n", 647 | " Считаем производную сложения.\n", 648 | " Относительно каждого из входов локальный градиент в вершине суммирования будет равен 1.\n", 649 | " входной градиент = {out.grad}\n", 650 | " локальный градиент для первого слагаемого = 1\n", 651 | " локальный градиент для второго слагаемого = 1\n", 652 | " выходной градиент для первого слагаемого = 1 * {out.grad}\n", 653 | " выходной градиент для второго слагаемого = 1 * {out.grad}\n", 654 | " \"\"\"\n", 655 | " )\n", 656 | " out._backward = _backward\n", 657 | "\n", 658 | " return out\n", 659 | "\n", 660 | "def new_backward(self):\n", 661 | "\n", 662 | " # topological order all of the children in the graph\n", 663 | " topo = []\n", 664 | " visited = set()\n", 665 | " def build_topo(v):\n", 666 | " if v not in visited:\n", 667 | " visited.add(v)\n", 668 | " for child in v._prev:\n", 669 | " build_topo(child)\n", 670 | " topo.append(v)\n", 671 | " build_topo(self)\n", 672 | "\n", 673 | " # go one variable at a time and apply the chain rule to get its gradient\n", 674 | " self.grad = 1\n", 675 | " print(\"\"\"\n", 676 | " Начинаем обратное распространение.\n", 677 | "\n", 678 | " На каждом шаге метода обратного распространения будем считать выходной градиент как входной * локальный.\n", 679 | " Входной градиент - градиент с прошлого шага метода обратного распространения.\n", 680 | " Локальный градиент - значение производной элементарной функции на данном шаге.\n", 681 | " Входной градиент на первом шаге = 1 (так как df/df = 1).\n", 682 | " \"\"\"\n", 683 | " )\n", 684 | " for v in reversed(topo):\n", 685 | " v._backward()" 686 | ], 687 | "metadata": { 688 | "id": "Y26ugyrQIujv" 689 | }, 690 | "execution_count": 22, 691 | "outputs": [] 692 | }, 693 | { 694 | "cell_type": "code", 695 | "source": [ 696 | "Value.relu = new_relu\n", 697 | "Value.backward = new_backward\n", 698 | "Value.__add__ = new_add\n", 699 | "Value.__mul__ = new_mul" 700 | ], 701 | "metadata": { 702 | "id": "8FolW8ZEItGl" 703 | }, 704 | "execution_count": 23, 705 | "outputs": [] 706 | }, 707 | { 708 | "cell_type": "code", 709 | "source": [ 710 | "x = Value(1.0)\n", 711 | "y = (x * 2 + 1).relu()\n", 712 | "y.backward()\n", 713 | "draw_dot(y)" 714 | ], 715 | "metadata": { 716 | "colab": { 717 | "base_uri": "https://localhost:8080/", 718 | "height": 748 719 | }, 720 | "id": "qxvDyeJnmdVU", 721 | "outputId": "80ea37fc-7308-4b16-cc0a-c54130a115dd" 722 | }, 723 | "execution_count": 24, 724 | "outputs": [ 725 | { 726 | "output_type": "stream", 727 | "name": "stdout", 728 | "text": [ 729 | "\n", 730 | " Начинаем обратное распространение.\n", 731 | " \n", 732 | " На каждом шаге метода обратного распространения будем считать выходной градиент как входной * локальный.\n", 733 | " Входной градиент - градиент с прошлого шага метода обратного распространения.\n", 734 | " Локальный градиент - значение производной элементарной функции на данном шаге.\n", 735 | " Входной градиент на первом шаге = 1 (так как df/df = 1).\n", 736 | " \n", 737 | "\n", 738 | " Считаем производную relu: \n", 739 | " входной градиент = 1\n", 740 | " локальный градиент = 1\n", 741 | " выходной градиент = 1 * 1 = 1\n", 742 | " \n", 743 | "\n", 744 | " Считаем производную сложения.\n", 745 | " Относительно каждого из входов локальный градиент в вершине суммирования будет равен 1.\n", 746 | " входной градиент = 1\n", 747 | " локальный градиент для первого слагаемого = 1\n", 748 | " локальный градиент для второго слагаемого = 1\n", 749 | " выходной градиент для первого слагаемого = 1 * 1\n", 750 | " выходной градиент для второго слагаемого = 1 * 1\n", 751 | " \n", 752 | "\n", 753 | " Считаем производную умножения.\n", 754 | " Для умножения локальный градиент по отношению к какому-либо из входов будет равен значению оставшегося входа:\n", 755 | " входной градиент = 1\n", 756 | " локальный градиент для первого множителя = 2\n", 757 | " локальный градиент для второго множителя = 1.0\n", 758 | " выходной градиент для первого множителя = 1 * 2 = 2\n", 759 | " выходной градиент для второго множителя = 1 * 1.0 = 1.0\n", 760 | " \n" 761 | ] 762 | }, 763 | { 764 | "output_type": "execute_result", 765 | "data": { 766 | "image/svg+xml": "\n\n\n\n\n\n%3\n\n\n\n133697222993440\n\ndata 3.0000\n\ngrad 1.0000\n\n\n\n133697222984752ReLU\n\nReLU\n\n\n\n133697222993440->133697222984752ReLU\n\n\n\n\n\n133697222993440+\n\n+\n\n\n\n133697222993440+->133697222993440\n\n\n\n\n\n133697222984752\n\ndata 3.0000\n\ngrad 1.0000\n\n\n\n133697222984752ReLU->133697222984752\n\n\n\n\n\n133697222996032\n\ndata 2.0000\n\ngrad 1.0000\n\n\n\n133697222996032->133697222993440+\n\n\n\n\n\n133697222996032*\n\n*\n\n\n\n133697222996032*->133697222996032\n\n\n\n\n\n133697222996608\n\ndata 2.0000\n\ngrad 1.0000\n\n\n\n133697222996608->133697222996032*\n\n\n\n\n\n133697222995264\n\ndata 1.0000\n\ngrad 2.0000\n\n\n\n133697222995264->133697222996032*\n\n\n\n\n\n133697222996416\n\ndata 1.0000\n\ngrad 1.0000\n\n\n\n133697222996416->133697222993440+\n\n\n\n\n\n", 767 | "text/plain": [ 768 | "" 769 | ] 770 | }, 771 | "metadata": {}, 772 | "execution_count": 24 773 | } 774 | ] 775 | }, 776 | { 777 | "cell_type": "markdown", 778 | "source": [ 779 | "Вернем старые реализации функций обратно, чтобы при обучении модели не утонуть в логах" 780 | ], 781 | "metadata": { 782 | "id": "vUbNUKBfJtJs" 783 | } 784 | }, 785 | { 786 | "cell_type": "code", 787 | "source": [ 788 | "Value.relu = old_relu\n", 789 | "Value.backward = old_backward\n", 790 | "Value.__add__ = old_add\n", 791 | "Value.__mul__ = old_mul" 792 | ], 793 | "metadata": { 794 | "id": "APZHmn_1JyhR" 795 | }, 796 | "execution_count": 25, 797 | "outputs": [] 798 | }, 799 | { 800 | "cell_type": "code", 801 | "source": [ 802 | "Neuron??" 803 | ], 804 | "metadata": { 805 | "id": "M_Mb3eVnmq_W" 806 | }, 807 | "execution_count": 23, 808 | "outputs": [] 809 | }, 810 | { 811 | "cell_type": "code", 812 | "source": [ 813 | "random.seed(1337)\n", 814 | "n = Neuron(2)\n", 815 | "x = [Value(1.0), Value(-2.0)]\n", 816 | "y = n(x)\n", 817 | "y.backward()\n", 818 | "\n", 819 | "dot = draw_dot(y)\n", 820 | "dot" 821 | ], 822 | "metadata": { 823 | "colab": { 824 | "base_uri": "https://localhost:8080/", 825 | "height": 357 826 | }, 827 | "id": "lEw9JEWkmdSv", 828 | "outputId": "ef4a2c98-bbed-4b53-83d2-10add880869e" 829 | }, 830 | "execution_count": 26, 831 | "outputs": [ 832 | { 833 | "output_type": "execute_result", 834 | "data": { 835 | "image/svg+xml": "\n\n\n\n\n\n%3\n\n\n\n133696956211808\n\ndata 0.1024\n\ngrad 1.0000\n\n\n\n133696956212384ReLU\n\nReLU\n\n\n\n133696956211808->133696956212384ReLU\n\n\n\n\n\n133696956211808+\n\n+\n\n\n\n133696956211808+->133696956211808\n\n\n\n\n\n133696956214928\n\ndata 0.0665\n\ngrad -2.0000\n\n\n\n133696956208352*\n\n*\n\n\n\n133696956214928->133696956208352*\n\n\n\n\n\n133696956212384\n\ndata 0.1024\n\ngrad 1.0000\n\n\n\n133696956212384ReLU->133696956212384\n\n\n\n\n\n133696956218000\n\ndata 0.0000\n\ngrad 1.0000\n\n\n\n133696956208112+\n\n+\n\n\n\n133696956218000->133696956208112+\n\n\n\n\n\n133696956213920\n\ndata 0.2355\n\ngrad 1.0000\n\n\n\n133696956213920->133696956208112+\n\n\n\n\n\n133696956213920*\n\n*\n\n\n\n133696956213920*->133696956213920\n\n\n\n\n\n133700378863280\n\ndata 1.0000\n\ngrad 0.2355\n\n\n\n133700378863280->133696956213920*\n\n\n\n\n\n133696956208352\n\ndata -0.1331\n\ngrad 1.0000\n\n\n\n133696956208352->133696956211808+\n\n\n\n\n\n133696956208352*->133696956208352\n\n\n\n\n\n133696956217616\n\ndata -2.0000\n\ngrad 0.0665\n\n\n\n133696956217616->133696956208352*\n\n\n\n\n\n133696956212096\n\ndata 0.2355\n\ngrad 1.0000\n\n\n\n133696956212096->133696956213920*\n\n\n\n\n\n133696956208112\n\ndata 0.2355\n\ngrad 1.0000\n\n\n\n133696956208112->133696956211808+\n\n\n\n\n\n133696956208112+->133696956208112\n\n\n\n\n\n", 836 | "text/plain": [ 837 | "" 838 | ] 839 | }, 840 | "metadata": {}, 841 | "execution_count": 26 842 | } 843 | ] 844 | }, 845 | { 846 | "cell_type": "markdown", 847 | "source": [ 848 | "#### MLP для бинарной классификации" 849 | ], 850 | "metadata": { 851 | "id": "1dv25IPTm3BT" 852 | } 853 | }, 854 | { 855 | "cell_type": "code", 856 | "source": [ 857 | "X, y = make_moons(n_samples=100, noise=0.1)\n", 858 | "\n", 859 | "y = y*2 - 1 # make y be -1 or 1\n", 860 | "# visualize in 2D\n", 861 | "plt.figure(figsize=(5,5))\n", 862 | "plt.scatter(X[:,0], X[:,1], c=y, s=20, cmap='jet')" 863 | ], 864 | "metadata": { 865 | "id": "SxlwcBdg7G66", 866 | "colab": { 867 | "base_uri": "https://localhost:8080/", 868 | "height": 466 869 | }, 870 | "outputId": "ac117ff4-d572-4271-847a-4b612339a9cc" 871 | }, 872 | "execution_count": 27, 873 | "outputs": [ 874 | { 875 | "output_type": "execute_result", 876 | "data": { 877 | "text/plain": [ 878 | "" 879 | ] 880 | }, 881 | "metadata": {}, 882 | "execution_count": 27 883 | }, 884 | { 885 | "output_type": "display_data", 886 | "data": { 887 | "text/plain": [ 888 | "
" 889 | ], 890 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAcsAAAGwCAYAAADG505FAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjguMCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy81sbWrAAAACXBIWXMAAA9hAAAPYQGoP6dpAAByRElEQVR4nO3dd1wT9/8H8NfdBQIoU3ZFBVFwbxHrFgVXpbZ1Vq2rrXXU2trKr3W1/dZqrbWOltq69x514EBxIiqioiKKIiqyHBBABJK73x9oakwCBEguhPfz8cijzc33geSdz93n8/4wgiAIIIQQQohWrNgBEEIIIcaOkiUhhBBSAkqWhBBCSAkoWRJCCCEloGRJCCGElICSJSGEEFICSpaEEEJICShZEkIIISWgZEkIIYSUgJIlIYQQUgK9JsuTJ0+iX79+cHd3B8Mw2L17d7Hb79y5Ez169ICTkxNsbGzg7++PQ4cOqWwze/ZsMAyj8vL19dXjVRBCCKnqJPo8eG5uLpo1a4bRo0djwIABJW5/8uRJ9OjRAz/99BPs7OywatUq9OvXD1FRUWjRooVyu0aNGuHo0aPK9xKJbpfB8zwePXoEa2trMAyj076EEEJMgyAIyM7Ohru7O1i2hLajYCAAhF27dum8X8OGDYU5c+Yo38+aNUto1qxZuWJ58OCBAIBe9KIXvehFL+HBgwcl5g29tizLi+d5ZGdnw8HBQWX57du34e7uDgsLC/j7+2Pu3LmoVauW1uPk5+cjPz9f+V54OdHKgwcPYGNjo5/gCSGEGDWZTAYPDw9YW1uXuK1RJ8sFCxYgJycHAwcOVC7z8/PD6tWr4ePjg5SUFMyZMwcdO3bEtWvXtF7w3LlzMWfOHLXlNjY2lCwJIaSKK83jOEYQDDOfJcMw2LVrF4KDg0u1/caNGzFu3Djs2bMHAQEBWrfLzMxE7dq1sXDhQowZM0bjNm+2LF99m8jKyqJkSQghVZRMJoOtrW2pcoFRtiw3b96MsWPHYtu2bcUmSgCws7ND/fr1kZCQoHUbqVQKqVRa0WESQgipIoxunOWmTZswatQobNq0CX369Clx+5ycHNy5cwdubm4GiI4QQkhVpNeWZU5OjkqLLzExEZcvX4aDgwNq1aqFkJAQJCcnY+3atQCKbr2OHDkSv//+O/z8/JCamgoAsLS0hK2tLQDgq6++Qr9+/VC7dm08evQIs2bNAsdxGDJkiD4vhRBCSBWm15blxYsX0aJFC+UYyalTp6JFixaYOXMmACAlJQX3799Xbr98+XLI5XJMmDABbm5uytfnn3+u3Obhw4cYMmQIfHx8MHDgQNSoUQPnzp2Dk5OTPi+FEEJIFWawDj7GRJeHuoQQQkyTLrnA6J5ZEkIIIcaGkiUhhBBSAkqWhBBCSAkoWRJCCCEloGRJTIpCwYsdAiHEBFGyJJWeQsFj/vwzcHf/FRLJD/Dy+h3Ll0ejCnb0JoToiVGWuyNEF59/HoY//riAV7nx3r1MfPLJPjx9mofp0zuIGxwhxCRQy5JUag8fylQSJQDl///ww0nk5BSIExghxKRQsiSVWmTkA2i72/r8eSGuXEk1bECEEJNEyZJUanZ2FuVaTwghpUHJklRqXbrUgYtLNbCs6uStHMegaVMXNGxINYMJIeVHyZJUamZmHLZvHwgrKzOwLAMzMxYMAzg4WGLjxgGlmgGdEEJKQr1hSaXXoUMt3Lv3Odatu4p79zLRoIEjhg5tAmtrmvCbEFIxKFkSk1CjhhWmTGkndhiEEBNFt2EJIYSQElCyJIQQQkpAyZIQQggpASVLQgghpASULAkhhJASULIkhBBCSkDJkhBCCCkBJUtCyig/X460tBzI5TThNCGmjpIlITp68UKOL74Ig739PLi6/goXlwWYN+80eJ4mmybEVFEFH0J0NHz4LuzcGadMjkWTTIcjL0+O2bO7iBscIUQvqGVJiA5u3nyM7dtvaGxFzp9/BtnZ+SJERQjRN0qWhOjgwoVkrevy8uSIj39iwGgIIYZCyZIQHTg7Vyt2vZOTlYEiIYQYEiVLQnTQvbsX3N2twXHqk0136VIHtWvbiRMYIUSvKFkSogOJhMXevYNhZ2ehfA8Anp72WLs2WMTICCH6RL1hCdFRq1buuH//C+zYcQP372ehUSNn9O1bX5k4CSGmh5IlIWVgZWWG4cObiR0GIcRA6KswIYQQUgJKloQQQkgJKFkSQgghJaBkSQghhJSAkiUhhBBSAkqWxKRlZb3A1atpePo0T+xQCCGVGCVLYpLy8+WYMOEAnJ0XoFmzULi4LMCoUbuRm1sgdmiEkEpIr8ny5MmT6NevH9zd3cEwDHbv3l3iPhEREWjZsiWkUim8vb2xevVqtW2WLVuGOnXqwMLCAn5+fjh//nzFB08qtc8+24/Q0IsoKFAAAORyHmvXXsWIEbtEjowQUhnpNVnm5uaiWbNmWLZsWam2T0xMRJ8+fdC1a1dcvnwZU6ZMwdixY3Ho0CHlNlu2bMHUqVMxa9YsXLp0Cc2aNUNgYCDS09P1dRmkkklNzcHq1VfUptHieQE7d97E7duVY2aQHTtuoFWr5bCw+BHe3ouxeHEUTTBNiEgYQRAM8tfHMAx27dqF4OBgrdt888032L9/P65du6ZcNnjwYGRmZiIsLAwA4OfnhzZt2mDp0qUAAJ7n4eHhgUmTJmH69OmlikUmk8HW1hZZWVmwsbEp+0URo3T8eCK6dVurdf3OnQPx7rsNDBiR7pYvj8Ynn+wDyzIqCXLChDZYurS3iJERYjp0yQVG9cwyMjISAQEBKssCAwMRGRkJACgoKEB0dLTKNizLIiAgQLkNIW+9Vfw/+po1jfsLUkGBAiEh4QCg1pL8448LuHcvU4SoCKnajCpZpqamwsXFRWWZi4sLZDIZ8vLy8PjxYygUCo3bpKamaj1ufn4+ZDKZyouYrvr1a6Bz59pq02hJJCyaNXNB69buIkVWOjduZGjtvSsIwMmTSQaOiBBiVMlSX+bOnQtbW1vly8PDQ+yQiJ5t2vQemjRR/VJVt649du8eDIZhtOxlHKpXNy92vbV18esJIRXPqGYdcXV1RVpamsqytLQ02NjYwNLSEhzHgeM4jdu4urpqPW5ISAimTp2qfC+TyShhmjg3N2tcuvQxTp26j/j4x/DyskfXrp5gWeNOlADg7e2Ali3dcOVKKhSK/27DMgxgbS1FYKC3iNERUjUZVcvS398f4eHhKsuOHDkCf39/AIC5uTlatWqlsg3P8wgPD1duo4lUKoWNjY3Ki5g+hmHQqVNtjBvXCt27e1WKRPnKmjXBsLW1AMMU3T7mOAYSCYuNGwfAyspM7PAIqXL02rLMyclBQkKC8n1iYiIuX74MBwcH1KpVCyEhIUhOTsbatUU9Fz/99FMsXboUX3/9NUaPHo1jx45h69at2L9/v/IYU6dOxciRI9G6dWu0bdsWixYtQm5uLkaNGqXPSyFG6MULOXheMMnk0bixM27fnoS1a68gNjYNtWrZYtSoFqhVy1bs0AipmgQ9On78uABA7TVy5EhBEARh5MiRQufOndX2ad68uWBubi54eXkJq1atUjvukiVLhFq1agnm5uZC27ZthXPnzukUV1ZWlgBAyMrKKuOVETHduvVY6NNng8CycwRgttCx40rh/PmHYodFCKlkdMkFBhtnaUxonGXllZqag8aN/0Bm5gvl8zyOY2BuzuHixY/RsKFThZ1LEAQoFAIkEqN6WkEIqSCVdpwlISUJDb2okigBQKEQUFjI45dfzlTIOTIzX+Czz/ajevW5MDP7AX5+f+PIkTsVcmxCSOVEyZJUKmfOPFBJlK/I5XyFjD8sLFSge/c1WL48Gs+fFwIALl5MQVDQBhw9erfcxyeEVE6ULEml4uRkpVZsACgaVuHkVK3cx9+7Nx6XLqkO2XhVRee7746V+/iEkMqJkiWpVD76qLnGlqUgAGPHtiz38U+fvg8zM/U/C54XEBWVXO5C5vn5cvzyyxk0arQMHh6/YdSo3YiPf1yuYxJC9M+oihIQEh39CPv23QLLMggO9lWrwtOzZ118911H/PjjKWULU6EQMHJkM4we3aLc57ezs9CaEKtXN0d5iv/wvIB+/TYhPDxReY7162OxfXscIiPHoHFj57IfnBCiV9QblnrDGgWeFzB+/H4sXx4NiYSBIBQlwa++8sf8+T3UStTFxWVg5844FBby6NOnHtq0eatC4khIeAofnyXgedXlHMfgs8/aYPHiXmU+9v79t9C37ya15RzHoG/f+ti9e3CZj00I0Z0uuYCSJSVLo7Bu3RWMGLFb47o9ewbjnXd8DBbLX39dxGefHQDDACzLoLCQR5s27jhyZDhsbS3KfNyJEw/gr7+iIZfzauvMzTnk539XnrAJITrSJRfQbVhiFFasiFGbuxEoanWtXn3ZoMnyk09aIyDACxs3xiIz8wU6dqyNvn3rl3u8pVTKaV1nbq59na7u3cvE3r3x4HkBffrUQ716NYrdvrBQAQAwM6u4GAgxNZQsiVHIyMjV+KxQoRCQnp5r8Hjq1nXAjBmdK/SYH3zQCAsXnlNbznEMBg5sVCHn+PHHk5g587jytvUXXxzCtGntMW9egNqt7OvX0zFt2hEcOlQ0hrR373r45Zce8PV1rJBYCDEl1BuWGIVOnWprbLlxHIMOHWqJEFHFa9euJiZP9gNQdF0MUzTkxcPDFv/7X7dyH//w4TuYMeM4BKHoGfCrLx+//HIW27ffUNn23r1MtG+/EocP31Fue/DgbbRvvwIPH9J8r4S8iZIlKVZW1gssWnQOAwduw6ef7kNk5AO9nGfqVH9IpZzKGEqOY2BjI8XEiW31ck4xLFoUiLCwYRg6tAn69fPBr7/2xOXLn8DVtXq5j/3339Eax6CyLIO//opWWfbbb5F4/rxArRKSTJaPJUuiyh0LIaaGbsMSrR48yEL79iuRnCwDwzDKD92ffuqGkJCOFXquevVq4NSpUfjqqyM4diwRAGBlZQZLSzPMnh2Bb755u8Rnb5UBwzAIDPTWy5yUKSk5Gseg8ryAR4+yVZZFRCRBLtd82/vEifJXQiLE1FDLkmj11VeHkZKSrbyt96oX5//93zG9DKRv0cINR44MR1BQXTAMkJNTgEePsrFmzWW0arUc16+nV/g5TYmf31tgNfxFSyQs2rWrqbLM0dFK4/yeHMegRg0rfYVISKVFyZJoVFiowM6dNzW2VDiOwbZtNzTsVX6HDiUgLOwOBKGoKg8AyOUCnj8vxLffUrm54tSt64A3B4IxTNHv68svVSdHHzWqudYOVaNGNddjlIRUTpQsiUYKhaBxPCBQdCsxL69QL+c9eDBBY0cfhULAgQO39XJOU3DoUAImTjygliw5jsWuXYPQqJFqdaChQ5tgzJiiikcSCav8mX/2WRu8914Dg8RMSGVCzyyJRhYWErz9tgciIx+qtUDkcl4vz9wAaKzL+t86Ggeozf/+dwoMw+DNGiMKBY+bNx+jV696KstZlsE//7yDTz5phT174sEwQHCwL1q1cjdk2IRUGpQsiVbz5/dAly6rwTBQ3o5lGKBfPx907Kif4Rzvv99Q72MRS6JQ8HjyJA+2tlJIpZXjT+TSpRSNt1VZlkFMTKrW/dq0eavCSgUSYsroNizRqn17D5w7NxbvvOODGjUsUb9+Dcyf3wPbt3+gNsBdoeCxdOl5NG78Bxwd56NPnw04c+a+zuf09/dQDhV5NQyCZRm4u1vjxx+7lv+iiiEIAn7//Rzc3RfCxWUB7O3nYdKkA8p5LY1ZcUNP3NzKPyyFkKqOasNSbdgKMWrUbqxZcwVAUcecV4lu376hCArS7ZatIAg4eDAB69dfRWbmC3TtWgdjx7aEvb1lhcf9uvnzz+Cbb46qLGNZBkFB3ti/f6hez11ev/0WiS+/PKz2zJJlGdy48Rl8fKgqDyFvokLqJaBkWbGuXk1Ds2ahassZBmjQwAnXro1Xa4m+LirqIX7++QzOn0+Gu7s1Pv20FUaPblHsPhXtxQs5XFwWQCbL17g+JuYTNG/uarB4dKVQ8Bg7di9Wr74Chin6wiKVclix4h0MG9ZU7PAIMUpUSJ0Y1OHDdzQWQRcE4MaNDKSl5Wq9TXjoUAL69NkIhmEgl/NIScnG2LGPEBOTiqVLexsifADAnTtPtSZKADh/PtmokyXHsVi1KhjTpr2NY8cSUa2aGfr394WDg35b44RUFZQsSblZWkrUemG+TttsG4Ig4PPPw8DzgnL/V4dZtuwCJk1qa7DbhzVqWClbZJo4O1czSBzl1bChExo2dBI7DEJMDnXwIeX27rsNtFaDCQjw0vqsMTk5G/HxTzQmKIaBcjYMQ3B1rY5evbzVaquyLANHR0v06qWfoTKEkMqBkiUpN3d3ayxZ0gtA0QD3V5Mm29tbYtky7bdSS5rDsSLneCyNf/55B/XrF9WffTVI39raHHv2DKk0Q0gIIfpBHXyog0+FiYlJwYoVMUhLy0Xr1m4YO7ZliXVGO3RYiXPnHqqV1eM4Bg8fTq2Q2Th0IZfzOHjwNq5cSUPNmjZ4//2GqF7d3KAxEEIMg3rDloCSpfGIjU1Dp06rIZPlg+cFSCQs5HIeS5f2woQJpjM1F1BUjD4jIxfVq5ujWjVKwISIjXrDkkqjSRMX3LjxGZYvj0Z0dApcXatj7NiWaNu25KoyCgWPiIh7SE7ORosWrmjSxMUAEZfNhg1X8e23x5CUlAWJhMXAgQ2xaFEQnJx07zh08OBt/PzzGcTGpsHDwxaTJ7ct81AbuZxHTEwKWJZB8+au4Dh6MkOIJtSypJZlpXTjRgb69t2IxMRM5bKgIG9s3fo+rK2l4gWmwZYt1zB48A6VZRzHoEEDJ8TEfKKxcLw269ZdwYgRu5VDdV714J02rT3mz++hU1w7d8Zh/Pj9SE/PBQC89ZY1li/vh96965WwJyGmQZdcQF8jiUZyOY/09FwUFirEDkVNYaECQUHrcf9+lsryI0fuYPLkMJGi0kwQBMycGYE3G30KhYBr19Lx77/xpT5WYaECX355GACUY1pffdVdsOAsHj6UlfpYUVEP8cEH25SJEgAePcpG//6bERubVurjEFJVULIkKhQKHj/+eBLOzr/AxWUBHB1/wbffhhtV0gwLS8CDBzK1TkEKhaAskWcsnj8vxK1bmofHmJmxOH8+udTHiot7jIyM5xrXCULRl4XSWrTonNpwn1cxLl58vtTHIaSqoGRJVHzzzVHMnHkcz54VJRyZLB8//3wG48fvFzmy/9y/n6XWUntFLueRmppj2ICKYWEhgZWVmcZ1CoWgU7EDS8viuxgsXXoeCoXmOUjfdPVqusb5SuVyHteupZc6JkKqCkqWROnJk+dYvDhKrRXE8wJWroxRu+0plkaNnLVW2rGyMkOtWraGDagYHMdizJgWasUOgKKxnEOHNin1sby9HdCkibPWLwqXLqVi166bpT6Wtpjq1rUvdUyEVBWULInS1atpKCzU3DIRhKI5E41B58610bKlm9qHPcMAn3/up7UlJ5affuqODh2K5v/kOAYMU1QCcMuW9+HiUvpxpAzDYPXqYK3rOY4pdbKcNKmt2m1soOg2/IQJbUodEyFVBSVLolTSMAYnp+ILDBgKwzA4eHAYevSoq1wmlXKYOtUf33+v3zkvy6J6dXMcPz4Sx4+PxPffd8Uff/RBcvJUBAf76nysli3dir11W9rO7QEBXli6tBcsLP67tWtlZYYVK96Bv7+HznERYupo6AgNHVESBAGtWi3H1atpKq0OjmNQp44dbt2apLEGrJgePpQhJSUb9evXgK2thdjhGMSkSQfw558XNbYMN216D4MHNy71sbKyXuDYsUSwLINu3TyNbtgNIfpEFXxKUNWTZUZGLlavvoxbt57A29sBH33UXHk7MCHhKbp1W4MHD2TKajrOztVw9Ohwox70X5WkpGSjbdt/kJKSDYVCUD7D7NrVE2Fhw2BmZtiauoRUVpQsS1CVk2VU1EP06LEOubmFyoHtFhYSHDr0ofK5Wn6+HHv2xOPmzceoW9ceAwY0gKWlcT0HrOrS03OxeHEU9u+/BSsrMwwZ0gTjxrWkgu+E6ICSZQmqarLkeQHe3ouRlJSlMlEzyzJwc6uOpKQpVO6MEFJlUAUfotHFi4+QmJipkiiBoiSanJyNM2ceiBQZIYQYN70ny2XLlqFOnTqwsLCAn58fzp/XXh2kS5cuYBhG7dWnTx/lNh999JHa+qCgIH1fhknIzs4v13pCKtLTp3k4eTIJN25kiB0KISXS6wOOLVu2YOrUqQgNDYWfnx8WLVqEwMBAxMfHw9nZWW37nTt3oqCgQPn+yZMnaNasGT744AOV7YKCgrBq1Srle6mUevCVRuvW7rCwkODFC7naOjMzFu3a1RQhKlLV8LyAkJCjWLQoCgUFRWUUW7Vyw+bN78Pb20Hk6AjRTK8ty4ULF2LcuHEYNWoUGjZsiNDQUFhZWWHlypUat3dwcICrq6vydeTIEVhZWaklS6lUqrKdvT1VHCkNW1sLzJzZCQDUqsBMn94BOTkFOHr0LhISnooQHakq5s07jfnzzyoTJQBcvpyKbt3WID9f/YscIcZAb8myoKAA0dHRCAgI+O9kLIuAgABERkaW6hgrVqzA4MGDUa2a6iDsiIgIODs7w8fHB+PHj8eTJ0+KPU5+fj5kMpnKq6qaPr0DVq58B76+jjAzY1G/vgN+/z0IcXEZ8PT8HT16rEO9eksQGLgejx9rLtpNSFkpFDwWLFD/+1coBDx4IMPu3aWrQESIoektWT5+/BgKhQIuLqpj81xcXJCamlri/ufPn8e1a9cwduxYleVBQUFYu3YtwsPDMW/ePJw4cQK9evWCQqF9Voy5c+fC1tZW+fLwqLoVShiGwahRLXDjxgQUFMxAfPwknD37ALt23VSptxoefhfvv79VvECJSXr27AWePs3TuE4iYXHz5mMDR0RI6RjtoKwVK1agSZMmaNu2rcrywYMHK/+/SZMmaNq0KerWrYuIiAh0795d47FCQkIwdepU5XuZTFalE+brkpNl2Lr1ulphcoVCwIkTSbh6NQ1Nm1IxAlPz7Fkejh69C6Co9J29vaVBzmtnZwEbGylkMvXOZHI5Dy8veqRCjJPeWpaOjo7gOA5paaoTyaalpcHV1bXYfXNzc7F582aMGTOmxPN4eXnB0dERCQkJWreRSqWwsbFReZEid+480zqDBwDEx9M3fVPzxx8X4Ob2KwYO3I6BA7fDze1X/PHHBYOcWyJhMXFiG7Vn5hzHwMnJCu+919AgcRCiK70lS3Nzc7Rq1Qrh4eHKZTzPIzw8HP7+/sXuu23bNuTn5+PDDz8s8TwPHz7EkydP4ObmVu6YqyJPTzutUz4BoN6JJiYi4h4mTDiA/Pz/Hlvk5yswYcIBRETcM0gMs2d3wahRzVX+3dWqZYsjR4Yb3YwxhLyi196wU6dOxd9//401a9YgLi4O48ePR25uLkaNGgUAGDFiBEJCQtT2W7FiBYKDg1GjRg2V5Tk5OZg2bRrOnTuHe/fuITw8HP3794e3tzcCAwP1eSkmy8PDFsHBvmrTXUkkLPz9a6JFC/oSYkqWLj2vcR5LjmOwbJn2MdAVycyMw4oV/ZGY+Dl27BiIEyc+QkLCZDRrVvwdJ0LEpNdnloMGDUJGRgZmzpyJ1NRUNG/eHGFhYcpOP/fv3wfLqubr+Ph4nD59GocPH1Y7HsdxuHr1KtasWYPMzEy4u7ujZ8+e+OGHH2isZTmsWtUfQ4bswMGD/93KbtfuLWzbNlDEqIg+3L37TMs8lgISEp4ZNJbate1Qu7adQc9JSFlRbVh6fqkUH/8YN28+hpeXPc0wYqJGjdqN9etjIZerTvItkbD48MOmWLWqv0iREWJ4uuQCo+0NS/RPEAQcPXoX69ZdxbNneejQoRbGjWsFBwfD9Iwkhvf55+2wfn0sGAbKjl2vnh1+/rmfeIERYuSokHoV9n//F46ePddj48ZY7Nt3G//3f8fQpMmfuH8/S+zQiJ40b+6KPXsGo2bN/75F16xpgz17BqN5c3pmSIg2dBu2it6GvXIlFc2b/6W2nOMYfPBBQ2za9L4IURFDUSh4xMamAwCaNHGmqdlIlUS3YUmJtm+/AYmEVXt2pVAI2LEjDjwvgGWLGVNCKjWOY6klSYgOKFlWUa8XsX6TXM6j6IYDJUtTp1DwiIlJBc8LaNHCFWZmnN7PKQgCCgoUMDfnwBQ3yJcQI0L3XqqoXr3qqbUqgaLbsAEBXnRbrgo4ePA2atdehDZt/oaf3z94662F2Lbtut7Op1DwmD//DNzcfoWFxf9Qq9YiLF16HlXwSRCphOgTsYrq3Lk23n3XV6WKikTCwNycw88/B2jfkZiEa9fS8c47m/HoUbZyWUbGcwwatB2RkQ/0cs6pUw9j+vSjSEvLBQA8fCjDpEkHMWfOCb2cj5CKRMmyimIYBlu2vI9Fi4LQrJkLPDxsMGRIE1y8+DFatqSqPaZu6dKiaj1vNuo4jsFvv52r8PM9epSNZcvOa6xDPG/eGWRlvajwcxJSkeiZZRVmZsZh8mQ/TJ5M4+uqmmvX0jXehpfLBcTGpmnYo3zOn0/WWDkIAF68kCMmJhVdutSp8PMSUlGoZUlIFeTt7QCJRP3Pn+MY1K1b8cXz7ewsil1PhTCIsaNkSUgV9NlnbaBQqLcsFQpBL3caOnashZo1bdSGI3Ecg0aNnNCkiXOFn5OQikTJkpAqqG3bt7B6dTCqVftvSiwLCwmWLOmFnj3rVvj5OI7Fjh0DYW1tDoYBzMxYMAxgb2+JzZvfpyEkxOhRBZ8qWsGHEADIySnAsWOJ4HkBXbvWga1t8bdLyysz8wU2bozFnTtP4evriMGDG8PammYMIuLQJRdQsqRkSQghVZIuuYBuwxJCCCEloGRJCCGElICSJSGEEFICSpaEEELUCIKAmJUrEdqsGebZ22NVp064tX+/2GGJhpIlIYQQNeEhIdg7ZgzSYmPxIjMTD86cwaa+fXF5zRqxQxMFJUtCiMHIZPkIC0vAsWOJxU4TR8QlS07G2V9+KXrzcsCEwBcVsTjy1VdQFBSIFZpoqDYsIcQgliyJwjffHEVenhwAUKOGJVavDkbfvvVFjoy86V5EhDI5vun548dIv34dbi1aGDgqcVHLkhCid//+G4/Jk8OUiRIAnj7Nw7vvbkF8/GMRIyOamFkWX6u3pPWmiJIlIUTvFi48B45TLWn3qhxKaOhFESIixakbGAjz6tWBN8oQMhwHp0aNUMPHR6TIxEPJkhCid7duPdY4RZdcziMh4akIEZHimFerhuA1a8ByHBiOA8OyYFgWZlZWCF69ukrW8qVnloQQvfP1dURaWq5awpRIGNSvX0OkqEhxGgwYgM+uX8elFSuQlZQE58aN0XLsWFR3dRU7NFFQbViqDUuI3h04cBt9+mxUWcYwgETC4vr1z1CvHiVMYnhUG5YQYlR6966HP//sg+rVzZXLXFyq499/h1CiJJUCtSypZUmIweTmFiAqKhnm5hzatasJiYS+rxPx6JIL6JklIWUk8DzuHDmCjBs3YFurFnz69QNnbl7yjlVYtWrm6NbNU+wwCNEZJUtCyiA7JQXrevRAxvXrYFgWAs+jupsbhh8+DOfGjcUOjxBSwegeCCFlsGv4cDyOjwfwXxmw3PR0bOzbF7yCyrgRYmooWRKio8ykJCSGh0OQy1WWCwoFspKScC8iQpzACCF6Q8mSEB3lpKYWvz4lxUCRGKfnzwsRG5uGtLQcsUMhpMJQsjRRsbFp2L37JuLiMsQOxeQ4+vqCk0q1rndr2dKA0RgPnhcwZ04EnJ1/QdOmoXBz+xX9+29Cenqu2KERUm6ULE1MenouOnVahaZNQ/Huu1vQsOEf6NFjLZ4+zRM7NJNhYWsLv8mT1etmsix83nkHTg0bihSZuH766RRmzz6B3NxCAEW1Xw8cuI3AwPXg+So3Qo2YGEqWJub997ciMvKhyrLjx+9h+PBdIkVkmrrPnYtOM2bA3NoaAMCZm6PluHF4b9MmkSMTR36+HAsWnFVbLpcLuHw5FcePJ4oQFSEVh4aOmJDY2DScOnVfbblCIeDAgdu4e/cZvLzsRYjM9LAch65z5qBjSAiyU1JQzcmpaJaGKurRo2xkZeVrXMeyDK5eTUP37l4GjoqQikMtSxNy9+6zcq0nmikKCvDw3Dkknz8P/o0esBILC9h7elbpRAkAjo5WMDPT/HHC8wJq1qRKWaRy03uyXLZsGerUqQMLCwv4+fnh/PnzWrdd/XLql9dfFhYWKtsIgoCZM2fCzc0NlpaWCAgIwO3bt/V9GZVCcbM3MAxQr56DAaMxDde3bcPCt97CCn9//OPnh99q1cKtffvEDsvoWFtLMXx4M7U5KzmOgZOTFd55p+rNf0hMi16T5ZYtWzB16lTMmjULly5dQrNmzRAYGIj09HSt+9jY2CAlJUX5SkpKUlk/f/58LF68GKGhoYiKikK1atUQGBiIFy9e6PNSKoUGDZzQo4eXxg+sd9/1Re3aduIEVkk9iIzE9kGD8PzxY+WynNRUbHn3XaRdvSpiZMZp0aJAtVJ2zs7VEBb2IaRSeuJDKje9JsuFCxdi3LhxGDVqFBo2bIjQ0FBYWVlh5cqVWvdhGAaurq7Kl4uLi3KdIAhYtGgRvvvuO/Tv3x9NmzbF2rVr8ejRI+zevVuflyKa/Hw55s49hbp1F8PBYR6CgzcjOvqR1u23bHkfgYHeyvcMAwQH+2L16mADRGtazi1aBJbjVBe+nHfg/JIlIkRk3KytpTh8eDiioz/G8uV98e+/Q5CUNAUtW7qJHRoh5aa3r3sFBQWIjo5GSEiIchnLsggICEBkZKTW/XJyclC7dm3wPI+WLVvip59+QqNGjQAAiYmJSE1NRUBAgHJ7W1tb+Pn5ITIyEoMHD9bX5YhCEAS8++4WhIUlvPqMxr59t3DgwG1ERHyE9u091Paxt7fE/v1DkZj4DHfvPkO9ejVQq5atgSM3DenXrqk9owQAXi5H+vXrIkRUObRs6UYJkpgcvbUsHz9+DIVCodIyBAAXFxekaqmA4uPjg5UrV2LPnj1Yv349eJ5H+/bt8fBh0VCIV/vpckwAyM/Ph0wmU3lVBseOJeLgwf8SJVDUs5XnBUyffrTYfT097dG9uxclynKo4e0N5s2WJQBWIoGDt7eGPQghpsqoesP6+/tjxIgRaN68OTp37oydO3fCyckJf/31V7mOO3fuXNja2ipfHh7qLTJjdOTIXY3z/SkUAk6duo+CAirYrU9tJ0+GoKEoOq9QoM2ECSJERAgRi96SpaOjIziOQ1pamsrytLQ0uLq6luoYZmZmaNGiBRISEgBAuZ+uxwwJCUFWVpby9eDBA10uRTSWlhJom5vbzIxV68hDKpZX9+7o/ccfkFhaKpeZVauG4NWrUdPPT8TICCGGprdkaW5ujlatWiE8PFy5jOd5hIeHw9/fv1THUCgUiI2NhZtb0fMPT09PuLq6qhxTJpMhKiqq2GNKpVLY2NiovCqDgQMbQaFQT5Ycx2DgwEbgOKO6MWCS2owfj69SUzF4zx4M+fdffJWWhmYjRogdFiHEwPTan3vq1KkYOXIkWrdujbZt22LRokXIzc3FqFGjAAAjRozAW2+9hblz5wIAvv/+e7Rr1w7e3t7IzMzEL7/8gqSkJIwdOxZAUU/ZKVOm4Mcff0S9evXg6emJGTNmwN3dHcHBwfq8FFE0aOCEn37qhv/7v2OQSFjwvABBKBrgPX9+D7HDqzKkNjbweecdscMghIhIr8ly0KBByMjIwMyZM5GamormzZsjLCxM2UHn/v37YNn/WkfPnj3DuHHjkJqaCnt7e7Rq1Qpnz55Fw9cKU3/99dfIzc3Fxx9/jMzMTHTo0AFhYWFqxQtMRUhIR3Tr5ok1a67g2bMXePttD4wc2QzW1uqzXqSn5+LkySRYWZmhWzdPWFjQ2DZCCKkIjKDtoZgJk8lksLW1RVZWVqW5JVscQRAwa1YE5s49DbmcBwDY21tg/foB6N27nsjRVQ5Z9+/j+tatyM/ORp0uXVCnSxcwDD0TJsSU6ZILKFkaIFkmJWXi778v4c6dZ6hXzwHjxrWEh0fFDelYvfoyRo3ao7KMYQCJhEV8/ER4elLx9OJE//039n/6KYCiabZ4uRxeAQEYvHcvzF7r3EMIMS265ALqIaJnR4/ehY/PUvz882ls3XodP/10Cj4+SxERca/CzrFo0bk3p1aEIBQVsP7nn0sVdh5T9PjmTez75BMIPA+B55VFCBKPHcOp//1P4z4CzyNfJoPA84YMlRAiIkqWelRYqMCHH+5EQYFCWUxAoRCQn1+0XKGomA/bxMRMaLo/IAhF64h2V9atUy9ph6KEeOnvv1WW8QoFTv30E35xdsbPtrZY4OKC0z//TEmTkCqAkqUenTnzAGlpuWqJjOcFJCdnIyoquULO06CBI1hW/fkawxStI9q9XiT9TXnPVKc0O/zllzj23XfIe/JEuW94SAgOf/WVXmMkhIiPkqUe5eUVlmt9aU2f3gE8r5qRWZaBpaUZxo5tWSHnMFU127XTWP+VYVmVwgM5aWm4sGwZNDXhzy9ZgtyMDL3GSQgRFyVLPfL394BUqn6LDwCsrMzQtu1bFXKe4GBf/PlnH9ja/jecpG5dexw9OhxubtYVcg5T1XjwYDi8UQOWYVkIgoDOs2crl6VcuqQxqQJFhdVToqP1HSohRESULPXIzs4C33/fFQCUt0lf/ffHH7tqHCtZVp9+2hopKV/i7NnRuHz5E8THT4SfX80KO76pMrO0xKhTp9Bo4ECwkqJxqc6NG2PYgQPw6t5duZ2lQ/ETZ1s50u1uQkwZDR0xwNCRzZuvYcGCs7hz5xnq16+Br77yxwcfNNL7eYlu5Pn5UOTnQ6rh34QgCFjq44Nnd++qFFdnOA4O3t6YEBdH4zIJqWRonGUJTK0oATGMtKtXsTYgAM8zMsCamYEvLISVkxNGHjsG58aNxQ6PEKIjXXIB1UMjpJRcmjbF54mJuL51K54mJKBGvXpo+MEHMK9WTezQCCF6RsmSEB2YV6uGFi8nAiCEVB3UwYcQQkiZCTyvdd5dU0LJkhBCiM7uRURgxdtv43uJBHOrV8f+zz5TK+RhSihZiuzIkTto334FJJLv4eT0C0JCjlZYsQJCCNGHpJMnsTYgAMnnzgGCgMLnzxG9fDnWdOkCRUGB2OHpBSVLER04cBuBgesRFZUMhULA48fPMX/+WfTrt6lK3NYghFROx2fMAARBpS6yoFAg7epVxO3cKWJk+kPJUkTTpx8FAJVSdTwvIDw8EceP3xMpKkIIKd79M2c0TiDAmpkh6dQpvZ+fl8uREReHzKQkvZ/rFUqWIpHJ8hEbm65xthCJhMWJE/cMHhMhFUUu57FrVxxmzDiGP/+8gGfP8sQOiVQgqbWWMpqCAAs7O72eO3bjRvxWqxb+aNgQv9epg7/btkVabKxezwlQshSNVMrBzEzzj5/nBdjYVFwpPEIMKTU1B02b/okBA7Zi3rwzmDDhAGrW/A1Hj94t13EzMnJx8+Zj5OdrrtFLDKf5qFEq9ZRf4eVyNP3wQ72d9/bBg9g5bBhyUlKUy1IuXcLqzp31PpkBJUuRSKUSfPBBQ3Cc5qm1Bg2iijCkcvr00324datoGrPCQh6CUDTDzoABW5Cbq3vnj/T0XPTvvwkuLgvQoMEyuLgswC+/nKHn+iLqMns23Fu3BlB065XhOIBhEPT773Bq0EBv5z09d65akhYUCuRnZSFm5Uq9nRegogSiWrgwEBcvpuDWrSeQSIpmuhAE4K+/+qJmTSrDRyqfp0/zsHdvvNrjBUEAsrMLsGdPPIYObVLq4ykUPAIC1uLGjQzlMbOy8vH110dhZsZhypR2FRg9KS2pjQ1Gnz6NW/v3415EBCxsbdF4yBA4+vjo9byply+r1GZWYhikX72q13NTshSRi0t1XLnyKbZtu47IyIdwdLTC8OFNUa9eDbFDI6RMMjNfaHwODxTNuJORkavT8Q4duoPY2HSN63766RQmTmwLiYRukImBlUjg278/fPv3N9g5bWrWxOObNzXOK2tdU7+zLFGyFJmFhQTDhzfD8OHNxA6FkHLz8LCBk5MVMjKeq63jeQH+/h46He/KlVRwHAOFQv3DMSPjOTIycmnO1iqk7cSJODBhgurCl7P9tBwzRq/npq9khJAKY2bGYc6cLmrLWZZBz5510aaNu07Hc3e31pgogaJOcnZ2FiUeIyrqISZM2I+hQ3dg2bLzyM7O1ykGYjxaf/op2k6cqEyQACCxsMB7GzeiRv36ej03TdFFU3QRUuFWrLiEOXNO4MEDGayszDB2bAvMnRsAKyszAMCTJ8/x66+R2LEjDizL4IMPGmLqVH+15JeTUwAPj98gk+WrjEfmOAZjx7ZEaGjfYuP46adT+PbbY8o+ATwvoE4dO5w5M5papJXYs7t3kXjsGMyqVUP9Pn00zkFbGjSfZQkoWRKif4IgICsrH9WqmcHM7L8ejE+f5qFt279x716mstXIcQzq1auBqKixasOmzp59gHfe2YQnT/LAMEWPq3r08MLOnYNQvbq51vPfuJGBRo3+UFvOcQyGDGmCdeveraArJZWVLrmAbsMSQvSCYRjY2VmoJEoAWLIkComJmSq3VxUKAbduPcFff11UO0779h54+HAqtm//AMuW9cb582Nx+PDwYhMlAGzdel3j0CyFQsCWLddUWqqElIQ6+BBCDGrv3lsaExXPC9i79xamTXtbbZ2FhQTvvddQp/Pk5RWCYRgA6ucqLOShUPBgWfWB9YRoQi1LQohBaatcxTCAuXnFfSQFBHhBLtdQv5Rl0LFjLbUWLyHFoWRJyBsUhYVIvXwZGXFxVCVGDwYNavR6Z0YVH3zQqMLO0727FwID64Jl/zsZxzHgOAY//xxQYechmmU/eoSDkydjYc2aWFS7Ng5Pm4bnT56IHVaZUQcf6uBDXnN1wwYc+uILPH9ZZ9KxQQP0X7UKNf38RI7MdOTlFaJ797WIjHyoTJqCAHTpUgdhYcMglVbc06H8fDl+/TUS//xzCZmZL9CpU23MmNEJrVrpNoSF6CYnLQ3LW7VCTloaBHlRLV+G42Dv5YVxFy7AwtZW5AiLUG/YElCyJJrcOXIE63v2VFnGsCwklpaYePMmbPRcIaQqyc+XY926q9iz5yYYhsGAAQ0wdGgTmJvTrVFTcHT6dJxdsECtNB3Dsuj+8894e9o0kSJTRb1hCSmDM/PmqRdp5nnIX7xA9PLlIkVlmqRSCcaObYl//x2KvXuH4KOPmlOiNCG39u3TWMNV4HkkHDwoQkTlR8mSkJfSrl7V+geefu2aCBERUjmZWVpqXM6wLCTSyjn9ICVLQl6yq1MHDKv+J8FyHGxr1xYhIkIqp8ZDh0JTLy6B59Fo8GARIio/SpaEvOQ3eTIE/o2hBgwDgefRatw4cYIipBJqM348anfsCKCoNfnq8Ub9fv3QdNgwMUMrMypKQMhLTYYNw+P4eJyeO1d5O9bMygrvrFgBp4a6DYgnpCqTWFhg+JEjuL5tG279+y8YloXvu++iwYABYLnK+WyaesNSb1jyhuyUFCQeOwaJhQW8AwNhXr262CERQvRAl1xALUtC3mDt5lZpbxURQvSDnlkSQgghJdB7sly2bBnq1KkDCwsL+Pn54fz581q3/fvvv9GxY0fY29vD3t4eAQEBatt/9NFHYBhG5RUUFKTvyyCEEFKF6TVZbtmyBVOnTsWsWbNw6dIlNGvWDIGBgUhPT9e4fUREBIYMGYLjx48jMjISHh4e6NmzJ5KTk1W2CwoKQkpKivK1adMmfV5GhcjPlyM+/jEeP34udiiEEEJ0pNcOPn5+fmjTpg2WLl0KAOB5Hh4eHpg0aRKmT59e4v4KhQL29vZYunQpRowYAaCoZZmZmYndu3eXOS5DdvARBAG//HIWP/10CllZ+WAYIDjYF3/91RdOTtX0em5CCCHaGUW5u4KCAkRHRyMg4L/q/izLIiAgAJGRkaU6xvPnz1FYWAgHBweV5REREXB2doaPjw/Gjx+PJyVUss/Pz4dMJlN5Gcpvv53DN98cRVZWPoCigtF798ajZ8/1NPksIYRUEnpLlo8fP4ZCoYCLi4vKchcXF6SmppbqGN988w3c3d1VEm5QUBDWrl2L8PBwzJs3DydOnECvXr2g0FCm7JW5c+fC1tZW+fLw8CjbRelILucxd+5pteUKhYDLl1Nx9Ohdg8RBCCGkfIx26MjPP/+MzZs3IyIiAhYWFsrlg18rldSkSRM0bdoUdevWRUREBLp3767xWCEhIZg6daryvUwmM0jCTE3N0fqMkuMYxMSkoGfPunqPgxBCSPnorWXp6OgIjuOQlpamsjwtLQ2urq7F7rtgwQL8/PPPOHz4MJo2bVrstl5eXnB0dERCQoLWbaRSKWxsbFRehuDgYKl1JgWFQsBbb1FBBEIIqQz0lizNzc3RqlUrhIeHK5fxPI/w8HD4+/tr3W/+/Pn44YcfEBYWhtatW5d4nocPH+LJkydwc3OrkLgrkpWVGUaMaAaOUy0ozLIM7O0t8O67viJFRgghRBd6HToydepU/P3331izZg3i4uIwfvx45ObmYtSoUQCAESNGICQkRLn9vHnzMGPGDKxcuRJ16tRBamoqUlNTkZOTAwDIycnBtGnTcO7cOdy7dw/h4eHo378/vL29ERgYqM9LKbOFC3uic+eiGSteFeG3t7fA/v1DUa2auYiREWKaBEFAQYH2PgyElIVen1kOGjQIGRkZmDlzJlJTU9G8eXOEhYUpO/3cv38f7GtTIv35558oKCjA+++/r3KcWbNmYfbs2eA4DlevXsWaNWuQmZkJd3d39OzZEz/88AOkRjpHmrW1FEePjsD588m4ePERXFyqo2/f+rCw0M+PPjU1B9HRj+DoaIW2bd8Co2GaHEJMUV5eIWbNisBff0VDJsuHr68jZs/ujEGDGosdGjEBVEjdRAqpKxQ8pkw5hD//vACFouhXWq+eA3bsGIgmTVxK2JuQyk0QBPTtuxFhYXeUQ7IYpmio1po1wRgxopnIERJjZBTjLIlhzZ17GsuWnVcmSgC4e/cZundfi+fPC0WMjBD9i4pKxoEDCSpjl181A/7v/8KhUPBa9iSkdChZmgCFgsfChZF48x6BQiEgI+M5tm27Lk5ghBjImTP3wbKaHzkkJ2fj0aNsA0dETI3RjrMkpZeTU4Bnz15oXGdmxiIh4amBIyLEsOztLaHtiRLLMrC2Ns4+DZWFwPO4FxGBtNhY2NSsifp9+0JipP1E9IWSpQmwtpaiRg1LPHmSp7ausJBHvXo1RIiKEMMZMKABJk48gBcv5Cp3WDiOQb9+PrCzs9C+MylWbkYG1gcGIjUmBgzLQuB5VHNxwYeHDsG1WdV5Fky3YU0AyzKYOtUfb3Z85TgGLi7V8P77DcUJjBADsbOzwObN70MiYcGyDMzMij7avLzs8ccfvUWOrnLbO2YM0mJjARS1MAHg+ePH2NinD3i5XMzQDIpalibim2/expMnz7F48XnI5UX/oOvXr4Ht2wfCyspM5OgI0b933vFBUtIUbNgQi9TUHLRq5YYBAxpAKqWPubLKTknBrX378GaHCEGhQHZyMu4cPox6vavGlxH6V2QiOI7Fr78GYvr0DoiJSYWjoxVatHClcZakSnFzs8ZXX7UXOwyTkZuerpYoX5edkqL8f0VBAW7t24fMpCQ4NWyIuj16gGFN5+YlJUsT4+RUjYqzE0IqhEPdupBYWkKep94fAgDcWrYEAKRfu4Z1PXsiJyVF+VzTqVEjDD9yBNZGWIq0LEwn7RNCCKlQ5tWrw//LL9WWMxwHrx494NaiBXiFApv69StqheK/55qP4+Oxe8QIg8arT9SyJARFf+C39u1D3M6dEBQK1OvbFw3few+shP5ESNXWdc4csBIJIn/9FQXZ2WAlEjT98EMELV4MALgXEYHMe/fU9hPkctw9ehSZSUmwq13bwFFXPPokIFUer1Bg+8CBiNu5E6xEAkEQcHX9ekR37YphBw9WufFkhLyOYVl0mTULHb75BrLkZFRzcoL0tdJwOa89t9QkJzXVJJIl3YYlVd61TZsQt3MnAICXyyEoimasuBcRgYuhoWKGRojRkFhYwKFuXZVECfz33FITTiqFo69pTEVIyZJUedc2bdLca08QELt+veEDIqQScWrYED7vvKP+N8Qw8Js8GRa2tuIEVsEoWZIqrzAvT9kpQW3d8+cGjoaQyue9TZvQctw4cOZFc/SaW1uj03ffofvcuSJHVnHomSWp8uoGBiLpxAm1hMlwHLyryIBrQsrDzMoKfUND0XPBAuSmp8Pa3R0SC9MqMUgtSwNKSsrEpEkH4OOzFG3a/I0lS6JQWEgzuout9SefwK5OHTAcp1zGcBysHB3hP3WqiJERUrmYV68Oey8vk0uUALUsDeb27Sfw8/sHMlk+FAoBDANERz/CoUN3sHfvEK3TCxH9s7Czw5hz53Dqf//D9S1bwCsUaDBgADp++63JDKg2Rk+ePMc//1zCxYspcHGphtGjW6BlS/p5E+PECNrmtTFhusyOrc2jR9m4efMxPDxsSjWrx+DB27F9+w2VyZlf2b9/KHr3rlemOAipjG7deoK3316Jp0/zIAgCOI6FXM5j2bLe+OyzNmKHR6oIXXIB3YbVUV5eIUaM2AUPj9/Qvfta1K+/FN27r0FaWk6x+/377y2NiVIiYbFv3y19hUuIUZowYT+ePcsDzwsQBCiL/0+efJAmaiZGiZKljiZPPogNG2LB8/8lvhMnktCv3yatk88CgESi/TarREK/BlJ1PHuWh6NHEzV+eeR5Abt2xYkQFSHFo09pHTx58hyrV19RSZQAoFAIuHDhEaKikrXu+957DTUmTLmcx4ABDSo8VkKMVX6+9k5tDMMgL6/qzJFIKg9KljpITMxU3i7SJD7+sdZ1P/7YDW5u1sqOPBxX9N9Ro5qjc+fKXwqKkNJycamGBg0c1SYrB4paljRrDjFG1BtWB7Vq2YJlGbWW5SteXvZa93V3t8bly58iNPQijh1LhI2NFMOGNcG77zagOSdJlcIwDH77LRB9+mwEAGXvcEEAPvqoGZo2dRE5QkLUUW9YHXvDDhu2E1u2XFN53iKRMGjQwAlXrnxKiY+QUjp79gF++ukUzp17CDe36vjkk9YYP741OI5ueFVGAs8j8dgxPLl1C/ZeXvDq0QPsa2OXjZEuuYBaljoKDe2DnJwC7N0br1zWrJkrdu0aRImSEB20b++BffuGih0GqQBZDx5gQ1AQMm7cwKvbBA7e3vjw8GHYe3qKHV6FoJZlGcdZ3r79BNeupaN2bTu0aOFa7kT5/Hkhfv31LNatu4rc3EIEBXkjJKQDvL0dynVcQgjRtxXt2+PRhQvg5f91zmIkEjg3aoRPYmKMtiGhSy6gZFnGZFmRCgsV6NJlNc6dS1Y+D5VIWFSrZobz58ehfv2Six4QQnSTk1OA1NQcuLtbw8rKTOxwKq2MGzfwR6NGWtePu3gR7q1aGTCi0qOiBJXMjh1xOHv2oUrHIbmcR05OAb7//oSIkRFiel68kGPChANwdJyPevWWwNFxPr7++gjVaS4jWbL2IXMAkF3C+sqCnlkagUOH7kAiYSCXq4/f3L//tkhRkYr2/MkTJIaHg5VI4NWjB6TW1mKHVCWNGrUbW7feUH45zcuTY8GCs8jJKcAff/QROTrdCIKAe8eP42FUFKo5OaHh++/Dws7OoDE4N24MhmU1T3PHMHBu0sSg8egLtSyNgLk5C0DzPX1zc+PuTUZK5+yCBfjVzQ3bBw3C1vfewwIXF1xes0bssKqcu3efYfPm62rDvwQB+PvvS0hPzxUpMt3ly2RY1bEj1nbvjuMzZuDfjz/Gwrfewu2DBw0ah7WbG5qPGqU2+TPDsmg0aJDJdPChZGkEBg5spLHYAccxGDbMNL6VVWW39u3DkWnTwBcWKpfJ8/KwZ9QoJF+4IGJkVc+VK6la18nlPK5fTzdgNOVzZNo0PDx3DgAgKBSAIKAwLw9b33sPeU+fGjSW3suWoc2ECeCkUgAAZ26OluPGof/KlQaNQ58oWRqBbt08MW5cSwBFCZJhinpf16tXA99+21Hk6Eh5RS1erDJX5issx+FiaKgIEVVdbm7F3/ouab2xkOfn4/KaNUVJ8nWCAPmLF7i2ZYtB45FIpei1eDGmZWRgQlwcpmVkoG9oKMwsLQ0ahz7RM0sjwDAM/vqrL4KDfbFp0zU8f16I7t09MXJkM1SrZi52eKScnt29q/6hBoCXy5GZmChCRFWXn99baNDAEbduPVErLNKmzVvw9XUUMbrSK8zNhSI/X+M6luOQmy5OC1lqbQ2pr68o59Y3SpZGgmEY9O5dj+a1NEEuTZsi8949tYTJSCRwbtxYpKiqJoZhsGfPYPTosQ5JSVnK8pV16zpgy5b3xQ6v1Czs7WFbuzaykpLU1vFyOd5qQ3OCVjQaZ2kE4yyJabt/5gxWdexY1IvkJYZlwUok+PTqVTj6+IgYXdUkl/M4ePA27t59Bh8fR/To4VXpyuxdXrMGez76SGUZw3FwadoU4y5cMPpSc8aAyt0RYkRqvf02Pti6FQcmTkRuWhoAwLZWLbyzYgUlSpFIJCz69avcP/vmI0dC4HkcnzED2cnJYCUSNB48GIGLFlGi1ANqWVLLkhgIL5cj7epVsC9vv77Z1Z6QshB4Hrnp6TC3toZ5tWpih1OpUMuSECPESiRwa9lS7DCIiWFYFtVdXcUOw+Tp/avtsmXLUKdOHVhYWMDPzw/nz58vdvtt27bB19cXFhYWaNKkCQ4cOKCyXhAEzJw5E25ubrC0tERAQABu36YqN4QQQvRHr8lyy5YtmDp1KmbNmoVLly6hWbNmCAwMRLqWbs1nz57FkCFDMGbMGMTExCA4OBjBwcG4du2acpv58+dj8eLFCA0NRVRUFKpVq4bAwEC8ePFCn5dCCCGkCtPrM0s/Pz+0adMGS5cuBQDwPA8PDw9MmjQJ06dPV9t+0KBByM3Nxb59+5TL2rVrh+bNmyM0NBSCIMDd3R1ffvklvvrqKwBAVlYWXFxcsHr1agwePLhUcdEzS0IIIUYx60hBQQGio6MREBDw38lYFgEBAYiMjNS4T2RkpMr2ABAYGKjcPjExEampqSrb2Nraws/PT+sxCakIL7Ky8OjiRcgePhQ7FEL0rjAvDzGrVuHfTz5B+Lff4nF8fMk7mTi9dfB5/PgxFAoFXFxcVJa7uLjg5s2bGvdJTU3VuH1qaqpy/atl2rbRJD8/H/mvVbuQyWSlvxBSpfEKBcJDQhC1eLGyYop3UBD6r16N6m/8OyTEFGSnpGBVx454ducOWIkEgiDg9Ny56Ld8OVqOHSt2eKKpEn3X586dC1tbW+XLw8ND7JBIJRExaxbOLligUlrs7tGj2NCrF6rgqCtSBRz64gtk3rsHoGi406si7fs++aRK31nRW7J0dHQEx3FIezkI+5W0tDS4aunm7OrqWuz2r/6ryzEBICQkBFlZWcrXgwcPdL4eUvUU5uXh3KJFKpV3gKIPkNSYGNyLiBAlLkL0Rf7iBeJ27NBYyxgArm/dauCIjIfekqW5uTlatWqF8PBw5TKe5xEeHg5/f3+N+/j7+6tsDwBHjhxRbu/p6QlXV1eVbWQyGaKiorQeEwCkUilsbGxUXoSURPbgAQpztcxvyDBIf62XNiGmQJ6fD14u17iOYVnkZ2cbOCLjodeiBFOnTsXIkSPRunVrtG3bFosWLUJubi5GjRoFABgxYgTeeustzJ07FwDw+eefo3Pnzvj111/Rp08fbN68GRcvXsTy5csBFBVBnjJlCn788UfUq1cPnp6emDFjBtzd3REcHKzPSyFVUDUXF7ASieYPD0GAXe3ahg+KED2ysLWFc+PGSL9+XeMdlTpduogTmBHQa7IcNGgQMjIyMHPmTKSmpqJ58+YICwtTdtC5f/8+2NdKfrVv3x4bN27Ed999h//7v/9DvXr1sHv3bjR+bWaGr7/+Grm5ufj444+RmZmJDh06ICwsDBYWFvq8FFIFWdjaoumHH+LKunUqt6UYjkN1V1d49+olYnSE6EfAvHnY2LcvGJaFwBdNSs+wLLx69EDtTp1Ejk48VBuWbsmSYhTk5GDbwIFIOHhQucyuTh0M2bcPzo0aiRgZIfpzNzwcJ2bPxsOoKFg6OKDluHHo9O23kJhYo0SXXEDJkpIlKYW0q1eRevkyrN3dUadrV5rVgRATQIXUCalgLk2bwqVpU7HDIISIpEqMsySEEELKg1qWhBBCDEoQBCSdOIG74eEws7JCo4ED4VC3rthhFYuSJSGEEIOR5+djy4ABSDhwQFlO79i33yJw4UK0mzJF7PC0otuwhBBCDObsL7/gTlgYANVyeoe++AIpMTEiR6cdtSwJIUYjJiYFu3ffhCAA77zjg9at3cUOiVSwS3//rRy/+TpWIsGVNWvg1qKFCFGVjJIlIUR0giBg8uQwLF16HhJJ0Q2vH344iXHjWiI0tC9YlhE5QlJR8p4907hcEATkPXli4GhKj27DEkJEt3NnHJYuPQ8AkMt5yOVFLY+//76ETZtixQyNVDCPt98Go2GcssDzqNm+vQgRlQ4lS0L0TOB5XPrnH/zdpg1+8/DA1g8+wKPoaLHDMiqrVl0Gx6m3HlmWwcqVlw0fUCUhCAIenD2LuF27lNNqGbvOM2eCYRgwr5U6ZTgOdrVro9nw4SJGVjxKloTo2f7PPsO/48bhUXQ0ZA8f4uauXVjRrh0Sjx8XOzSj8fjxcygU6sXEeF5ARoaWmV+quIy4OCzz9cXKt9/G1gED8LuXF3Z++CHkr829aow8/P0x/MgRuLVuDaDoWWXDDz7AqNOnYV69usjRaUfPLAnRo/Tr1xH9119Fb15WlhQUCggsi7DPP8enV66AYeh5XKdOtXHx4iO1hMlxDLp0qSNOUEZMUVCAdT16ICc19b+FgoBrmzahmrMzAhcuFC+4UqjTpQvGRUWh8PlzsBIJOHNzsUMqEbUsCdGjhLAwldtNSjyP9NhY5KanGz4oIzR5sh9sbKQqt2I5joGVlRm++KKdiJEZp/h//0V2crLaJM0Cz+NiaCgK8/JEikw3ZlZWlSJRApQsRaVQ8Pj333hMm3YYP/54Enfvau4lRsSVm56O2I0bcX3rVrzIytJp35I+CDgzs/KEZjJq1rTBmTOjERTkDZZlwDBAjx5eOHNmNDw97cUOz+g8u3NHYycZAJDn5eF5RoaBIzJ9dBtWJDJZPnr2XIeoqGSYmbHgeQGzZkXgjz9645NPWosdHnnp9M8/4/iMGcoJoCUWFui9bBlajB5dqv19g4NxSENVEobjULtjR1g6OFRkuJVagwZO2LdvKAoKFBAEAVIpfTxp4+jrq9aqfMXc2hrVXs4ZTCoOtSxFMnPmcVy8+AgAUFjIQ6EQwPMCxo/fj4SEpyJHRwAgfu9ehIeEKBMlAMhfvMDesWORfP58qY5h6+GBHr/8AqCoIwNQNJGu1MYGvZYurfigTYC5OUeJsgT1eveGfd266q1LhoHf5MmQSKXiBGbCKFmKZNWqyxp7/7Esg/Xrr4oQEXnT+SVLNN7qYjkOF0NDS30c/6lTMfrMGTQdMQL1evdGpxkzMOHGDZo8mpQZK5FgxNGjcG/TRrmMYVm0GD0aXWbPFi8wE0Zf30QgCAKyszV372ZZBs+eVY6H86bu2d27Gm918XK5zmPaPNq3h4cRD7gmlUO+TAZeLoelgwPs6tRB0KJF2BIcjJzUVAg8jytr1sCyRg0EzJ2ruWMZKTP6aYqAYRi0a1dTYwmvwkIeHTrUEiEq8ibXFi2Ut05fx0gkOk0EzSsUSL1yBenXrkEQ1O8mGMrdo0exY9gwrOvRA8dnzkR2SoposRDdPI6Px7oePfCzrS3m16iB5a1a4ebevVjXo4dKj2peLsfZ+fMRtWSJiNGaJkYQ869XJDKZDLa2tsjKyoKNjY0oMYSH30XPnusBFA28Boq6yjdu7IwLF8bBzExzTzdiOA8iI7GqQ4eiBPfyz4RhWXDm5vj06lXUqFevxGPE7dqFgxMnIvtR0fNpey8v9PvnH3h27arX2N8UMXs2TsyZA4bjICgUYDgOFra2GH3mDBx9fQ0aC9FNTloa/mjYEC+yspR3Ol61Gl//t/k6m1q18EVSkkHjrIx0yQXUshRJ9+5eCAsbhlat3AAAlpYSjB3bEsePj6REaSQ8/P0xcMcOWLv/N/OFvZcXhoWFlSpRPoiMxLb331dpwT27dw8bevXCk1u39BKzJk9u3cKJOXMAQPlhKygUeJGVhTAjnj+QFIlevhwvMjNVHgkIPF/sXQrZ/fsaZ/YgZUfPLEXUo0dd9OhRF4WFCnAcSzMrGCHf4GDU79cPGdevg5VI4NigQakr7kT++ivAssBrvWnB8+AVCpxfuhS9Fi/WU9Sq4nbtUrYoXycoFLhz+DAKnz+HmZWVQWIhukuOitKc+IpJlnaenvTMsoJRsjQC1JI0bizH6fSM8pXUy5chvJ4oXxLkcqRdNVyPZ76wUPtKQQCvZbweMQ7VnJ3BSiQqQ5heedVb+80vQu2nTTNIbFUJffUgRE/svbw0Dz2RSGDn6WmwOOr17q2xVy/DsqjZvj2k1tYGi4XorsXo0RoTJQB0mD4d9l5eyvechQU6z56N1p9+aqjwqgxKloToSdtJkzQPPVEo0Gb8eIPF4dayJVqMGVP05uWtuVfFqwN//dVgcZCyqdWhAwLmzQNeTmv16gtYi9Gj0fX77zExPh7jLl7E8KNH8VVKCrrMmlXqRwW8XI4Lf/6J5a1bY5GnJ3aPGoWMuDh9Xk6lRb1hReoNS6qGM/Pn49i335a5XF5xBEFA0okTuL5tGxQFBfAOCoJv//4ah7sIPI/Lq1fj0t9/Iyc9HbU7dED7r7+mwgiVSOa9e4jbubPod92rF1ybNSvX8QRBwI4hQ3B969ZXC8BIJODMzDD69Gm4tWxZAVEbN11yASVLSpZEjwRBwK19+3Bh2TLwCgWajRiBJkOGaExouh53//jxiP7rL+WxeLkctTt3xodhYZBYWFRE+MSE3T99Gqs6dlRbznAc6nTtihFHjogQlWHR0BFCjIAgCDg4aRI2v/MOEsPDkRQRgd0jRmBtQEC5p1BKOHhQOU8mL5crW673T53CuUWLyhs6qQQKcnNxNCQEC1xc8KNUijXduiHp1KlS73/74EHNdyEUCiSGh0NRXMewKoiSJSF6cufwYVxYtgxAxSe02I0bNXYeEngeV9auLdexifETeB4be/fG2fnzkZueDkVBAZJOnsSarl2RePx4qY7BSiRax2oyLEuTkr+BkiUhehK7YQMYLc8Pr6xZU65jF+TkaB10XpCdXa5jE+OXcOgQkk6eVPk3ICgUgCAgPCSkVMdo+N57mntJcxx8g4PL/ajA1FCyJERPCnJytM45WN6E5tmtm8bljESCuj17luvYxPjdO35ca0eu5KgoKAoKSjyGS9OmeHv6dAD/jddkWBZWjo7KaeXIfyhZEqIndbTUf2UlEniVM6E1HzUKDm/MZ8hwHMytrNChlC0LUnmZW1trvYXKSaWlbhV2/+knDAsLQ+NBg1A3MBBd5szB+NhY2BtwHHBlQb1hqTcs0ZP87Gwsb9kSzxIT/yuAzXEws7LCxxcvokb9+uU6fm5GBk7MmYNrmzZBUViIen36oMvs2XD08amI8IkRe3L7Npb6+KiVvGM4Ds1GjED/lStFiqxyoaEjJaBkSQwlNyMDEbNnFyW0/Hx49+6NrnPmwKlhQ7FDI5Vc1OLFCPv8c5WhQ46+vvjo5ElUc3ISObrKgZJlCShZEkJMQfq1a7iybh1ePHsGj7ffRuNBg2iMrQ4oWZaAkiUhhBAqSkAIIYRUIEqWhBBCSAkoWRJCCCEl0FuyfPr0KYYNGwYbGxvY2dlhzJgxyMnJKXb7SZMmwcfHB5aWlqhVqxYmT56MrKwsle0YhlF7bd68WV+XQQghhEBv9YyGDRuGlJQUHDlyBIWFhRg1ahQ+/vhjbNy4UeP2jx49wqNHj7BgwQI0bNgQSUlJ+PTTT/Ho0SNs375dZdtVq1YhKChI+d7Ozk5fl0EIIYTopzdsXFwcGjZsiAsXLqB169YAgLCwMPTu3RsPHz6Eu7t7qY6zbds2fPjhh8jNzYXk5VgihmGwa9cuBAcHlzk+6g1LCCFE9N6wkZGRsLOzUyZKAAgICADLsoiKiir1cV5dgOSN0k0TJkyAo6Mj2rZti5UrV2ot+/RKfn4+ZDKZyosQQggpLb3chk1NTYWzs7PqiSQSODg4IDU1tVTHePz4MX744Qd8/PHHKsu///57dOvWDVZWVjh8+DA+++wz5OTkYPLkyVqPNXfuXMyZM0f3CyGEEEKgY8ty+vTpGjvYvP66efNmuYOSyWTo06cPGjZsiNmzZ6usmzFjBt5++220aNEC33zzDb7++mv8UkKF/JCQEGRlZSlfDx48KHeMhBBCqg6dWpZffvklPvroo2K38fLygqurK9LT01WWy+VyPH36FK6ursXun52djaCgIFhbW2PXrl0wMzMrdns/Pz/88MMPyM/Ph1Qq1biNVCrVuo4QQggpiU7J0snJCU6lKNDr7++PzMxMREdHo1WrVgCAY8eOged5+Pn5ad1PJpMhMDAQUqkUe/fuhUUpahxevnwZ9vb2lAwJIYTojV6eWTZo0ABBQUEYN24cQkNDUVhYiIkTJ2Lw4MHKnrDJycno3r071q5di7Zt20Imk6Fnz554/vw51q9fr9IRx8nJCRzH4d9//0VaWhratWsHCwsLHDlyBD/99BO++uorfVwGIYQQAkCP4yw3bNiAiRMnonv37mBZFu+99x4WL16sXF9YWIj4+Hg8f/4cAHDp0iVlT1lvb2+VYyUmJqJOnTowMzPDsmXL8MUXX0AQBHh7e2PhwoUYN26cvi6DEFIMXqFAwsGDyIiLg72nJ3zeeQecubnYYenFk1u3EL18OZ7duYMaPj5o9cknNElyFUKzjtA4S0LKJOvBA6wLCMCTW7fAcBwEhQLV3dww/MgRODdqJHZ4FerWvn3Y8u67EAQBgkIBhuPAmZlh6IED8OzaVezwSBmJPs6SEGL6dgwdiqd37wIABIUCAJCbno7N/ftD4HkxQ6tQ8vx87B45ErxCobxOQaGAoqAAu0eMAP9yGTFtlCwJITp7mpCAB6dPQ5DLVZYLCgWe3bmD+6dPixRZxUs6eRJ5T58Cb9yEE3gesocP8ejiRZEiI4ZEyZIQorOcEoqLlLS+MpG/eFHsekV+voEiIWKiZEkI0ZlTo0bFduRxa9nSgNHoV60OHcBpG8NtYwP318p6EtNFyZIQojNLe3u0mTgRYBiV5QzLotGgQXB4o0d7ZWZpb4+uP/wAoOj6AIDhOABAwPz5MLOyEi02Yjh6GzpCCDFtPebPh3n16ohatAj5MhkkFhZo+fHH6DFvntihVbi3p02DvacnIhcuxLM7d+DYoAHaf/UV6vftK3ZoxEBo6AgNHSGkXOT5+chNS4OVoyO1skilQkNHCKmk4nbuRGjz5vie47DA1RUnvv8eisJCscMqlkQqhW2tWpQo35CfnY0r69bh3KJFeBAZWeJUgsS40W1YQozElbVrsXvkyKLngIKA3LQ0nJgzBxk3buD9zZvFDo/o4M6RI9g6YAAKcnLAsCwEnodXQAAG7d4N82rVxA6PlAG1LAkxArxCgfCQkKI3r7VABJ7H9S1bkHb1qkiREV3lPX2KLcHBKMjNBQBlgYbE48dxdPp0MUMj5UDJkhADy370CHG7duFeRISy+kvmvXvIfvRI8w4Mg6STJw0YISmPa1u2oDAvT72IgUKBmBUrjP62OtGMbsMSYiC8QoFDX3yBC8uWKVsbNjVr4oPt24svyC0IkFJHtEojJyUFLMeBf6O6EQDI8/JQmJsLzs7O8IGRcqGWJSEGEvnrrzi/dKlK3dTsR4+wvmdPsGZm8AoIUI7fU2IYmFlZwad/fwNHS8rKrWVLjYkSAGxr1YLU1tbAEZGKQMmSEAMQBAGRCxdqrC+an52N2I0b0Xf5cli7uQEAWDMzMBwHViLBgA0bYEEfsJVG/b594dSwofoXHwCdZs4E80YhB1I50G1YQgyALyxEblqaxnWsRIJnd+/C3tMTE27exLVNm5ASEwNrd3c0GzECth4eBo6WlAcrkWDEsWM48NlnuLl7NwSeRzVnZ3SZMwctx4wROzxSRpQsCTEA1swMNh4ekD14oLaOl8vh6OsLADCvVg0tx441dHikglV3ccHAHTuQ9+wZ8rOyYFOzJlgJfdxWZnQblhADYBgGb3/9tfpyjoOVoyMaDx4sQlTaCTyP9OvXkX79uknNTWlolvb2sKtThxKlCaBkSYiBtJkwAZ1nz4bEwkK5zNHXFyOPHYPU2lrEyFTdOXIEi7298WfjxvizcWP87uWFhEOHxA6LEFFRbVjqkk8M7EVWFlIvX4alvT2cmzQxqg4fabGxWN6yZdH4z1cfDQwDluMw7uJFuDZrprZPQW4uEo8dA19YiDpdu8LS3t7AURNSNrrkAro3QIiBWdjaok7nzmKHoVHU778X/c/r36Ff/n/UokXov2qVyvaxmzZh38cfoyAnBwDASaXoPncu/L/4wiDxlpbA8wDDGNUXE1K50G1YQohS6uXLGscI8nI5UmJiVJalxMRg57BhykQJAIr8fByeOhW39u3Te6ylkXTyJFZ27IjvJRLMtbbG/gkTkPfsmdhhqUm9fBlHQ0JwcPJk3NyzR1nZiRgPSpaEECU7T0+NnVEYjlOrMnQxNBSshrGEDMf910IVUdKpU1jbvTsenj0LCAIKc3MR/ddfWNO1KxQFBWKHpxQxZw7+atECkQsW4GJoKLYEB2NN164ofP5c7NDIayhZEkKU2owfr7FlKSgUaDNhgsqyZ3fuaN326Z07eouxtI7PmAFBEFR68woKBdKuXEHcrl0iRvafh+fO4cTs2QCKWu/8y7qxD86cwWkTnES7MqNkSYiRu3/6NLYNHIjQ5s2x88MPkXzhgt7O5dmtG4IWLwZrZqZcxpqZIfC33+AVEKCyrVOjRlpboc6NG+t0XkVhIS78+Sf+adcOfzRqhLAvvkCWhjGpunhw5gwEDbczWTMzoylMf3X9eo0/Q4HncXnlynIfXxAEpF6+jFv79yMzKancx6vKqIMPIUYsZtUq7B09GqxEAl4uR8b167i2aRPe37IFDd9/Xy/n9Js0CU2GDsWdw4cBQUDdnj1h5eiotl2bzz5DdGiocr7GVwSeh/+XX5b6fALPY+t77/33nFMQ8Dg+HlfXrsXY8+fhULduma7DvHp1vMjM1HBCwWjKB+bLZFonhc6Xycp17Kz797H1vffw6OLFogUMg8aDB+OdFStgZmlZrmNXRdSyJMRIFeTk4OCkSQCgvN3Jy+UQeB77x4/X61RPVjVqoMmQIWgydKjGRAkAjj4+GLp/P6zfeku5zLJGDQzYsEGn3r4JYWG49e+/Rb1uXyYOQaHAi6wsRMyaVeZraD5qlMb6rLxcjqYffljm41ak2p06aWz9MhyHOl27lvm4As9jfVAQUi9ffm2hgOtbtuDQlCllPm5VRsmSECOVdPIkCl9OIPym548fI/n8eQNHpM4rIACfJyZi3MWLGHPuHL589AhNhgzR6Ri39u3TfCtSocDN3bvLHFuXOXPg1qoVgP8K0wNA4G+/walhw1IfpyAnBwlhYbhz5AjkL16UOR5Nmgwdiho+PipJ/VUB/c7l+KKQePw4HsfFqT1TFngeMatWaW5xk2LRbVhCKiljGTPIchzcXyalsmBY7d/ZNbUMS0tqbY0xZ87g1r59uBcRAamNDZoMHaqsw1sa0X//jUNffKH80mJhZ4e+y5ej0QcflDmu15lZWWH06dM49t13iN2wAYV5efDs3h3dfvwRbi1agJfL8SIrCxa2tjqVzHuakKB1HV9YCNnDh7CgOTV1QhV8qIIPMVIFubn41dVVZRwjAIBhYOXoiKnJyeBe64hTWd0ND8e6NzoPAUWJstmIEehfAR1dyiLx2DGs7d5dbTnDshh38SLcWrRQLrt79ChO/fQTUqKjUd3dHW0++wxtJ0wo9ouAJoIggGEY8HI5Tv7vfzi3aBHyMzNhYWcHvylT0Om77zQO11GL/fhxrO3WTeM6ztwcX6WnG81zWzHpkgvoNiwhRsq8WjX0Wrq0qNzcy1YFK5GAYRj0DQ01iUQJFPXAbTpiBID/WpkMy6K6qyu6/vCDaHGdW7RIY8uWYVlcWLpU+f7Gjh1Y17Mnkk6eRL5Mhifx8QibPBn7Pv1U53O+ulsQNmUKTsyZg/yXt0tfZGbixJw5OFTKykh1OnfW2FuZYVm0GD2aEmUZUMuSWpbEyD04exZRS5bgya1bcG7UCH6TJ8O9dWuxw6pQAs/jxvbtuLp+PfKzsuAZEIA248dr7VxkCEt9ffEkPl7julodOmDUqVMQeB6/e3khS8uwjAk3b8LRx0en8+akpmJhzZqah71IJPji4UNUd3Ep8ThZDx5g+6BBeBgZWbSAYdD0ww/Rb/lylWL+VRnVhiXEhHi0bw+P9u3FDkOvGJZFo4ED0WjgQLFDUXL09cXThAS1pMVKJHB82UEo8949rYkSDIPE8HCdk2VKTIzGRAkU9eRNjYmBd1BQicex9fDAmLNnkX79OmQPH8K5USPY1KypUyzkP3QblhBCNGj3xRfqc3kyDARBQNuX1YwkxY1XFASYWVnpfN6SWtNWTk46Hc+5USN4BwZSoiwnSpaEVBL5MhmSTp5E2tWrWgeyk4pTp3NnBK9eDelrvUatatTAwB074NK0KQDA2s0NHh06aHy2yZmbw+edd3Q+r3vr1nD09VU7JsNxcGrYEG4tW+p8TFJ+9MySnlkSIycIAk7973849b//Kcf5OTVqhPc2bYJLkyYiR2f65C9e4OG5c2A4DjXbtVPrWJURF4dVHTvixcvZTBiWBc/zCF69Gs2GDy/TOTNu3MDa7t2Rk5qqrN5U3dUVI44dg1ODBuW+JlJEl1xAyZKSJTFyF0NDsX/8eJVlDMfBws4On9+9Cyn9Gxbd8ydPELNyJVJjYmDt7o4Wo0frVPhAk8K8PMTt3ImnCQmoUa8eGgwYQB1zKhglyxJQsiSVySJPT2Tdu6e+gmHQ548/0LoMQxQIITTOkhCTwSsUmhMlinplZsTFGTYgQqoovSXLp0+fYtiwYbCxsYGdnR3GjBmDnDcrkbyhS5cuYBhG5fXpG9+a79+/jz59+sDKygrOzs6YNm0a5Brm1CPEFLAcB2t3d43reLkc9l5eBo6IkKpJb8ly2LBhuH79Oo4cOYJ9+/bh5MmT+Pjjj0vcb9y4cUhJSVG+5s+fr1ynUCjQp08fFBQU4OzZs1izZg1Wr16NmTNn6usyCBFdOw1VWxiWhXm1akYzewYhpk4vyTIuLg5hYWH4559/4Ofnhw4dOmDJkiXYvHkzHj16VOy+VlZWcHV1Vb5ev498+PBh3LhxA+vXr0fz5s3Rq1cv/PDDD1i2bBkKCgr0cSmEiM5/6lT4TZ6sMpSguqsrPjx8GFY1aogYGSFVh16SZWRkJOzs7ND6tZJcAQEBYFkWUVFRxe67YcMGODo6onHjxggJCcHz589VjtukSRO4vFbqKTAwEDKZDNevX9d6zPz8fMhkMpUXIZUFw7II+v13fHH/Pgbu2IER4eGYkpQED39/sUMjpMrQS7m71NRUODs7q55IIoGDgwNSU1O17jd06FDUrl0b7u7uuHr1Kr755hvEx8dj586dyuO6vFET8dX74o47d+5czJkzp6yXQ4hRsHZ3R4MBA8QOQ+/Sr13D+WXL8DguDg7e3mgzYYLKDB8lEQQBCWFhiNuxA7xcDu9evdBgwACTKTxPxKFTspw+fTrmzZtX7DZx5eid9/ozzSZNmsDNzQ3du3fHnTt3ULdu3TIfNyQkBFOnTlW+l8lk8PDwKPPxCCGlU5CTg6z791HdzQ2W9vYlbn9r3z5sefddAEUdmB6cOYPLq1bhvU2bSlU3VuB57BoxArEbNoCVSCAAuLJmDWp37oxhBw/CrLjydIQUQ6dk+eWXX+Kjjz4qdhsvLy+4uroiPT1dZblcLsfTp0/h6upa6vP5+fkBABISElC3bl24urri/Buzw6elpQFAsceVSqWQSqWlPi8hpHwUhYUIDwnBhWXLIH/xAqxEgibDhqHXkiWQWltr3WfvmDHgFQrg5fBvXi4HGAb/fvwx6vfrV2Kyi9u1C7EbNvy370tJJ0/i/NKleHvatAq6QlLV6PTM0snJCb6+vsW+zM3N4e/vj8zMTERHRyv3PXbsGHieVybA0rh8+TIAwM3NDQDg7++P2NhYlUR85MgR2NjYoGE5q2UQQirO4S+/ROTChcryfLxcjqvr12PH4MFa90k+fx656enKRKkkCMjPykLSyZMlnvfa5s0a67RCEHB1/XqdroGQ1+mlg0+DBg0QFBSEcePG4fz58zhz5gwmTpyIwYMHw/3lmLHk5GT4+voqW4p37tzBDz/8gOjoaNy7dw979+7FiBEj0KlTJzR9WbS4Z8+eaNiwIYYPH44rV67g0KFD+O677zBhwgRqORJiJPKePsXF0FC1pCcoFLh94ADStXTG0zYtVWnXA0Bhbq7W7QpLGOdNSHH0Ns5yw4YN8PX1Rffu3dG7d2906NABy5cvV64vLCxEfHy8srerubk5jh49ip49e8LX1xdffvkl3nvvPfz777/KfTiOw759+8BxHPz9/fHhhx9ixIgR+P777/V1GYQQHT2OjwdfWKh1fdqVKxqXv9W2LSxem+HjdWZWVqjVsWOJ5/YKCAAYRm05w3Hw7t27xP0J0YZqw1JtWEIqVGZSEn6vU0fr+o9OnEDtTp00rruybh12jxwJhmUhKBRgOA6CQoE+f/5Zqhq4+TIZlrdqhWeJicoW5qui859cugTbWrXKdE3ENFFtWEKIaOxq10bdnj01zsfoUK8eanXooHXfZsOHY+Tx46jfpw8c6tWDd1AQPjx8uNTF4qU2Nhh99izaTpyIas7OsHRwQLPhwzHuwgVKlKRcqGVJLUtCKlxOWho29u6NlEuXlMvsPD3xYVgYatSvL2JkhPxHl1ygl6IEhJCqrbqLC8ZdvIj7p04h48YN2L5sbbKaeqoSUglQsiSElJvA87h98CBu7t4NCALq9+uH+n37onanTlqfTxJSmVCyJISUC69QYMfgwbixfTtYSdFHSsyKFajXpw8G7dpFZeaISaAOPoSQcondsAE3tm8HUFR84FXlnNsHDiBm5UoxQyOkwlCyJISUy9UNG8Cwmj9Krq5bZ+BoCNEPSpaEkHIpzM2FwPPqKwQBBdnZhg+IED2gZEkIKRevHj00tiwZjkPdoCARIiqZIAiaEzwhWlCyJISUS9sJE2Dt7q5ShIDhOFg5OqLdlCniBaaBLDkZu4YPx/8sLPCDmRnWBQaqjAUlRBtKloSQcrFydMTYqCi0HDsWlg4OsLC3R/ORIzHu/HlYv5wxyBi8yMzEyvbtEbtpExQFBRB4Honh4VjZoYPW4u6EvEIVfKiCDyFVQuTChTj81Vdqs6EwEgkaDxyIAS/nwSRVB9WGJYSQN9w/dUrjckEuR+Lx4waOhlQ2lCwJIVWChZ2d1nJ7Fvb2Bo6GVDaULAkhVULT4cOVBRNUMAxajB5t+IBIpULJkhBSJXh264YOISEAAFYiUZbmq9e7N/wmTRIzNFIJUAcf6uBDSJWSevkyrm/bBvmLF6jXqxc8u3XTWoGImDaaoosQQrRwbd4crs2bix0GqWTo6xQhhBBSAkqWhBBCSAkoWRJCCCEloGRJCCGElICSJSGEEFICSpaEEEJICShZEkIIISWgZEkIIYSUgJIlIYQQUgJKloQQQkgJKFkSQgghJaiStWFf1Y6XyWQiR0IIIUQsr3JAaeYTqZLJMjs7GwDg4eEhciSEEELElp2dDVtb22K3qZJTdPE8j0ePHsHa2hoMw4gai0wmg4eHBx48eGAS04WZ0vXQtRgnuhbjVBmvRRAEZGdnw93dHWwJ07RVyZYly7KoWbOm2GGosLGxqTT/wErDlK6HrsU40bUYp8p2LSW1KF+hDj6EEEJICShZEkIIISWgZCkyqVSKWbNmQSqVih1KhTCl66FrMU50LcbJlK5FkyrZwYcQQgjRBbUsCSGEkBJQsiSEEEJKQMmSEEIIKQElS0IIIaQElCxF8L///Q/t27eHlZUV7OzsSrWPIAiYOXMm3NzcYGlpiYCAANy+fVu/gZbC06dPMWzYMNjY2MDOzg5jxoxBTk5Osft06dIFDMOovD799FMDRaxq2bJlqFOnDiwsLODn54fz588Xu/22bdvg6+sLCwsLNGnSBAcOHDBQpCXT5VpWr16t9juwsLAwYLTanTx5Ev369YO7uzsYhsHu3btL3CciIgItW7aEVCqFt7c3Vq9erfc4S0PXa4mIiFD7vTAMg9TUVMMErMXcuXPRpk0bWFtbw9nZGcHBwYiPjy9xP2P+e9EVJUsRFBQU4IMPPsD48eNLvc/8+fOxePFihIaGIioqCtWqVUNgYCBevHihx0hLNmzYMFy/fh1HjhzBvn37cPLkSXz88ccl7jdu3DikpKQoX/PnzzdAtKq2bNmCqVOnYtasWbh06RKaNWuGwMBApKena9z+7NmzGDJkCMaMGYOYmBgEBwcjODgY165dM3Dk6nS9FqCo0srrv4OkpCQDRqxdbm4umjVrhmXLlpVq+8TERPTp0wddu3bF5cuXMWXKFIwdOxaHDh3Sc6Ql0/VaXomPj1f53Tg7O+spwtI5ceIEJkyYgHPnzuHIkSMoLCxEz549kZubq3UfY/57KROBiGbVqlWCra1tidvxPC+4uroKv/zyi3JZZmamIJVKhU2bNukxwuLduHFDACBcuHBBuezgwYMCwzBCcnKy1v06d+4sfP755waIsHht27YVJkyYoHyvUCgEd3d3Ye7cuRq3HzhwoNCnTx+VZX5+fsInn3yi1zhLQ9drKe2/PbEBEHbt2lXsNl9//bXQqFEjlWWDBg0SAgMD9RiZ7kpzLcePHxcACM+ePTNITGWVnp4uABBOnDihdRtj/nspC2pZVgKJiYlITU1FQECAcpmtrS38/PwQGRkpWlyRkZGws7ND69atlcsCAgLAsiyioqKK3XfDhg1wdHRE48aNERISgufPn+s7XBUFBQWIjo5W+ZmyLIuAgACtP9PIyEiV7QEgMDBQ1N8BULZrAYCcnBzUrl0bHh4e6N+/P65fv26IcCucsf5eyqN58+Zwc3NDjx49cObMGbHDUZOVlQUAcHBw0LqNqf1eqmQh9crm1fMKFxcXleUuLi6iPstITU1Vuz0kkUjg4OBQbFxDhw5F7dq14e7ujqtXr+Kbb75BfHw8du7cqe+QlR4/fgyFQqHxZ3rz5k2N+6Smphrd7wAo27X4+Phg5cqVaNq0KbKysrBgwQK0b98e169fN7pJBkqi7fcik8mQl5cHS0tLkSLTnZubG0JDQ9G6dWvk5+fjn3/+QZcuXRAVFYWWLVuKHR6AolmbpkyZgrfffhuNGzfWup2x/r2UFSXLCjJ9+nTMmzev2G3i4uLg6+troIjKrrTXUlavP9Ns0qQJ3Nzc0L17d9y5cwd169Yt83FJ6fn7+8Pf31/5vn379mjQoAH++usv/PDDDyJGVrX5+PjAx8dH+b59+/a4c+cOfvvtN6xbt07EyP4zYcIEXLt2DadPnxY7FIOiZFlBvvzyS3z00UfFbuPl5VWmY7u6ugIA0tLS4ObmplyelpaG5s2bl+mYxSnttbi6uqp1IJHL5Xj69Kky5tLw8/MDACQkJBgsWTo6OoLjOKSlpaksT0tL0xq7q6urTtsbSlmu5U1mZmZo0aIFEhIS9BGiXmn7vdjY2FSqVqU2bdu2NZrENHHiRGVHvpLuQBjr30tZ0TPLCuLk5ARfX99iX+bm5mU6tqenJ1xdXREeHq5cJpPJEBUVpdI6qCilvRZ/f39kZmYiOjpaue+xY8fA87wyAZbG5cuXAUDli4C+mZubo1WrVio/U57nER4ervVn6u/vr7I9ABw5ckQvvwNdlOVa3qRQKBAbG2vQ30FFMdbfS0W5fPmy6L8XQRAwceJE7Nq1C8eOHYOnp2eJ+5jc70XsHkZVUVJSkhATEyPMmTNHqF69uhATEyPExMQI2dnZym18fHyEnTt3Kt///PPPgp2dnbBnzx7h6tWrQv/+/QVPT08hLy9PjEtQCgoKElq0aCFERUUJp0+fFurVqycMGTJEuf7hw4eCj4+PEBUVJQiCICQkJAjff/+9cPHiRSExMVHYs2eP4OXlJXTq1MngsW/evFmQSqXC6tWrhRs3bggff/yxYGdnJ6SmpgqCIAjDhw8Xpk+frtz+zJkzgkQiERYsWCDExcUJs2bNEszMzITY2FiDx/4mXa9lzpw5wqFDh4Q7d+4I0dHRwuDBgwULCwvh+vXrYl2CUnZ2tvJvAoCwcOFCISYmRkhKShIEQRCmT58uDB8+XLn93bt3BSsrK2HatGlCXFycsGzZMoHjOCEsLEysS1DS9Vp+++03Yffu3cLt27eF2NhY4fPPPxdYlhWOHj0q1iUIgiAI48ePF2xtbYWIiAghJSVF+Xr+/Llym8r091IWlCxFMHLkSAGA2uv48ePKbQAIq1atUr7neV6YMWOG4OLiIkilUqF79+5CfHy84YN/w5MnT4QhQ4YI1atXF2xsbIRRo0apJP3ExESVa7t//77QqVMnwcHBQZBKpYK3t7cwbdo0ISsrS5T4lyxZItSqVUswNzcX2rZtK5w7d065rnPnzsLIkSNVtt+6datQv359wdzcXGjUqJGwf/9+A0esnS7XMmXKFOW2Li4uQu/evYVLly6JELW6V8Mn3ny9in/kyJFC586d1fZp3ry5YG5uLnh5ean87YhJ12uZN2+eULduXcHCwkJwcHAQunTpIhw7dkyc4F+j6Rre/IyqbH8vuqIpugghhJAS0DNLQgghpASULAkhhJASULIkhBBCSkDJkhBCCCkBJUtCCCGkBJQsCSGEkBJQsiSEEEJKQMmSEEIIKQElS0IIIaQElCwJIYSQElCyJIQQQkpAyZIQQggpwf8DXrMrxF5YTPcAAAAASUVORK5CYII=\n" 891 | }, 892 | "metadata": {} 893 | } 894 | ] 895 | }, 896 | { 897 | "cell_type": "code", 898 | "source": [ 899 | "X[0], y[0]" 900 | ], 901 | "metadata": { 902 | "colab": { 903 | "base_uri": "https://localhost:8080/" 904 | }, 905 | "id": "1qxsjISDk5cg", 906 | "outputId": "4b287cd6-d093-4dd2-b42b-abb53f112fba" 907 | }, 908 | "execution_count": 28, 909 | "outputs": [ 910 | { 911 | "output_type": "execute_result", 912 | "data": { 913 | "text/plain": [ 914 | "(array([ 1.58202308, -0.44581483]), 1)" 915 | ] 916 | }, 917 | "metadata": {}, 918 | "execution_count": 28 919 | } 920 | ] 921 | }, 922 | { 923 | "cell_type": "code", 924 | "source": [ 925 | "MLP??" 926 | ], 927 | "metadata": { 928 | "id": "EU_zgjaVkfq9" 929 | }, 930 | "execution_count": 28, 931 | "outputs": [] 932 | }, 933 | { 934 | "cell_type": "code", 935 | "source": [ 936 | "Layer??" 937 | ], 938 | "metadata": { 939 | "id": "MjTtPjPi9GB-" 940 | }, 941 | "execution_count": 29, 942 | "outputs": [] 943 | }, 944 | { 945 | "cell_type": "code", 946 | "source": [ 947 | "Module??" 948 | ], 949 | "metadata": { 950 | "id": "zLu5jABPkqEK" 951 | }, 952 | "execution_count": 30, 953 | "outputs": [] 954 | }, 955 | { 956 | "cell_type": "code", 957 | "source": [ 958 | "model = MLP(2, [16, 16, 1]) # 2-layer neural network\n", 959 | "print(\"number of parameters\", len(model.parameters()))" 960 | ], 961 | "metadata": { 962 | "colab": { 963 | "base_uri": "https://localhost:8080/" 964 | }, 965 | "id": "8uWxWtpSj-Gr", 966 | "outputId": "7b2d1cc5-136f-4bb6-bc4e-f0c06017ca4d" 967 | }, 968 | "execution_count": 29, 969 | "outputs": [ 970 | { 971 | "output_type": "stream", 972 | "name": "stdout", 973 | "text": [ 974 | "number of parameters 337\n" 975 | ] 976 | } 977 | ] 978 | }, 979 | { 980 | "cell_type": "code", 981 | "source": [ 982 | "model.layers" 983 | ], 984 | "metadata": { 985 | "colab": { 986 | "base_uri": "https://localhost:8080/" 987 | }, 988 | "id": "CDFlsn00j-JO", 989 | "outputId": "6138f302-c7e4-4d9c-926e-0401ace43f19" 990 | }, 991 | "execution_count": 30, 992 | "outputs": [ 993 | { 994 | "output_type": "execute_result", 995 | "data": { 996 | "text/plain": [ 997 | "[Layer of [ReLUNeuron(2), ReLUNeuron(2), ReLUNeuron(2), ReLUNeuron(2), ReLUNeuron(2), ReLUNeuron(2), ReLUNeuron(2), ReLUNeuron(2), ReLUNeuron(2), ReLUNeuron(2), ReLUNeuron(2), ReLUNeuron(2), ReLUNeuron(2), ReLUNeuron(2), ReLUNeuron(2), ReLUNeuron(2)],\n", 998 | " Layer of [ReLUNeuron(16), ReLUNeuron(16), ReLUNeuron(16), ReLUNeuron(16), ReLUNeuron(16), ReLUNeuron(16), ReLUNeuron(16), ReLUNeuron(16), ReLUNeuron(16), ReLUNeuron(16), ReLUNeuron(16), ReLUNeuron(16), ReLUNeuron(16), ReLUNeuron(16), ReLUNeuron(16), ReLUNeuron(16)],\n", 999 | " Layer of [LinearNeuron(16)]]" 1000 | ] 1001 | }, 1002 | "metadata": {}, 1003 | "execution_count": 30 1004 | } 1005 | ] 1006 | }, 1007 | { 1008 | "cell_type": "code", 1009 | "source": [ 1010 | "# посчитаем параметры\n", 1011 | "(2 * 16 + 16) + (16 * 16 + 16) + (16 * 1 + 1)" 1012 | ], 1013 | "metadata": { 1014 | "colab": { 1015 | "base_uri": "https://localhost:8080/" 1016 | }, 1017 | "id": "yRxUBd6Jj-Ln", 1018 | "outputId": "64ed55ce-6258-45e3-fb22-ec4d2da5a44f" 1019 | }, 1020 | "execution_count": 31, 1021 | "outputs": [ 1022 | { 1023 | "output_type": "execute_result", 1024 | "data": { 1025 | "text/plain": [ 1026 | "337" 1027 | ] 1028 | }, 1029 | "metadata": {}, 1030 | "execution_count": 31 1031 | } 1032 | ] 1033 | }, 1034 | { 1035 | "cell_type": "code", 1036 | "source": [ 1037 | "def loss(batch_size=None):\n", 1038 | " Xb, yb = X, y\n", 1039 | " inputs = [list(map(Value, xrow)) for xrow in Xb]\n", 1040 | "\n", 1041 | " # forward the model to get scores\n", 1042 | " scores = list(map(model, inputs))\n", 1043 | "\n", 1044 | " # svm \"max-margin\" loss\n", 1045 | " losses = [(1 + -yi * score_i).relu() for yi, score_i in zip(yb, scores)]\n", 1046 | " data_loss = sum(losses) * (1.0 / len(losses))\n", 1047 | " # L2 regularization\n", 1048 | " alpha = 1e-4\n", 1049 | " reg_loss = alpha * sum((p * p for p in model.parameters()))\n", 1050 | " total_loss = data_loss + reg_loss\n", 1051 | "\n", 1052 | " # also get accuracy\n", 1053 | " accuracy = [(yi > 0) == (score_i.data > 0) for yi, score_i in zip(yb, scores)]\n", 1054 | " return total_loss, sum(accuracy) / len(accuracy)\n", 1055 | "\n", 1056 | "total_loss, acc = loss()\n", 1057 | "print(total_loss, acc)" 1058 | ], 1059 | "metadata": { 1060 | "colab": { 1061 | "base_uri": "https://localhost:8080/" 1062 | }, 1063 | "id": "s5zk3OeMj-OJ", 1064 | "outputId": "b65ed918-bc38-4933-b37b-2b6cc7dfd90a" 1065 | }, 1066 | "execution_count": 32, 1067 | "outputs": [ 1068 | { 1069 | "output_type": "stream", 1070 | "name": "stdout", 1071 | "text": [ 1072 | "Value(data=0.41454652413193865, grad=0) 0.82\n" 1073 | ] 1074 | } 1075 | ] 1076 | }, 1077 | { 1078 | "cell_type": "code", 1079 | "source": [ 1080 | "for k in range(100):\n", 1081 | " # forward\n", 1082 | " total_loss, acc = loss()\n", 1083 | "\n", 1084 | " # backward\n", 1085 | " model.zero_grad()\n", 1086 | " total_loss.backward()\n", 1087 | "\n", 1088 | " # update (sgd)\n", 1089 | " learning_rate = 1.0 - 0.9 * k / 100\n", 1090 | " for p in model.parameters():\n", 1091 | " p.data -= learning_rate * p.grad\n", 1092 | "\n", 1093 | " if k % 1 == 0:\n", 1094 | " print(f\"step {k} loss {total_loss.data}, accuracy {acc*100}%\")" 1095 | ], 1096 | "metadata": { 1097 | "colab": { 1098 | "base_uri": "https://localhost:8080/" 1099 | }, 1100 | "id": "27ZNbO4fj-RF", 1101 | "outputId": "e2008ae3-6998-4e2a-f357-0dd79b5fa8fc" 1102 | }, 1103 | "execution_count": 33, 1104 | "outputs": [ 1105 | { 1106 | "output_type": "stream", 1107 | "name": "stdout", 1108 | "text": [ 1109 | "step 0 loss 0.41454652413193865, accuracy 82.0%\n", 1110 | "step 1 loss 0.7863805046215181, accuracy 81.0%\n", 1111 | "step 2 loss 0.461961145267714, accuracy 83.0%\n", 1112 | "step 3 loss 0.5085892319645842, accuracy 83.0%\n", 1113 | "step 4 loss 0.31049052687261214, accuracy 86.0%\n", 1114 | "step 5 loss 0.2689947571730255, accuracy 89.0%\n", 1115 | "step 6 loss 0.24554659072506863, accuracy 89.0%\n", 1116 | "step 7 loss 0.23622330038400666, accuracy 90.0%\n", 1117 | "step 8 loss 0.21720946703312552, accuracy 90.0%\n", 1118 | "step 9 loss 0.21400559443772385, accuracy 91.0%\n", 1119 | "step 10 loss 0.20939260057894807, accuracy 91.0%\n", 1120 | "step 11 loss 0.3300210916807542, accuracy 92.0%\n", 1121 | "step 12 loss 0.38174928707618516, accuracy 87.0%\n", 1122 | "step 13 loss 0.4508064668364713, accuracy 85.0%\n", 1123 | "step 14 loss 0.24799516110878045, accuracy 89.0%\n", 1124 | "step 15 loss 0.19302863848938961, accuracy 91.0%\n", 1125 | "step 16 loss 0.18158693713880056, accuracy 93.0%\n", 1126 | "step 17 loss 0.18371866789300473, accuracy 94.0%\n", 1127 | "step 18 loss 0.17646529149266213, accuracy 92.0%\n", 1128 | "step 19 loss 0.2118644238610899, accuracy 95.0%\n", 1129 | "step 20 loss 0.23740708747640143, accuracy 90.0%\n", 1130 | "step 21 loss 0.2424213728622781, accuracy 91.0%\n", 1131 | "step 22 loss 0.16352206202435898, accuracy 93.0%\n", 1132 | "step 23 loss 0.14055706166037016, accuracy 94.0%\n", 1133 | "step 24 loss 0.12641650696039422, accuracy 95.0%\n", 1134 | "step 25 loss 0.11779662914897572, accuracy 96.0%\n", 1135 | "step 26 loss 0.11656157360300687, accuracy 96.0%\n", 1136 | "step 27 loss 0.10693241862855216, accuracy 97.0%\n", 1137 | "step 28 loss 0.10958624114174169, accuracy 96.0%\n", 1138 | "step 29 loss 0.10134992156481276, accuracy 98.0%\n", 1139 | "step 30 loss 0.10930078228361885, accuracy 95.0%\n", 1140 | "step 31 loss 0.08885676934639257, accuracy 98.0%\n", 1141 | "step 32 loss 0.0822624174703037, accuracy 98.0%\n", 1142 | "step 33 loss 0.07952622026383123, accuracy 97.0%\n", 1143 | "step 34 loss 0.08377693512335735, accuracy 98.0%\n", 1144 | "step 35 loss 0.09834976703042793, accuracy 97.0%\n", 1145 | "step 36 loss 0.0805034897187587, accuracy 98.0%\n", 1146 | "step 37 loss 0.0882343743831684, accuracy 98.0%\n", 1147 | "step 38 loss 0.07944486631852585, accuracy 98.0%\n", 1148 | "step 39 loss 0.10287087028218089, accuracy 96.0%\n", 1149 | "step 40 loss 0.11136981231707056, accuracy 96.0%\n", 1150 | "step 41 loss 0.10982494830215679, accuracy 95.0%\n", 1151 | "step 42 loss 0.05744935437701046, accuracy 98.0%\n", 1152 | "step 43 loss 0.04896980932498546, accuracy 98.0%\n", 1153 | "step 44 loss 0.04452824806589162, accuracy 98.0%\n", 1154 | "step 45 loss 0.03857659354607201, accuracy 99.0%\n", 1155 | "step 46 loss 0.03629101324493242, accuracy 100.0%\n", 1156 | "step 47 loss 0.03618323133012296, accuracy 99.0%\n", 1157 | "step 48 loss 0.04728429210924266, accuracy 99.0%\n", 1158 | "step 49 loss 0.03248749654352533, accuracy 100.0%\n", 1159 | "step 50 loss 0.030335357997559816, accuracy 100.0%\n", 1160 | "step 51 loss 0.028156244389769505, accuracy 100.0%\n", 1161 | "step 52 loss 0.029230812235729126, accuracy 100.0%\n", 1162 | "step 53 loss 0.027270821931031152, accuracy 100.0%\n", 1163 | "step 54 loss 0.024973498876801573, accuracy 100.0%\n", 1164 | "step 55 loss 0.02558550342438838, accuracy 100.0%\n", 1165 | "step 56 loss 0.024083011912084673, accuracy 100.0%\n", 1166 | "step 57 loss 0.0255029133464586, accuracy 100.0%\n", 1167 | "step 58 loss 0.021453874084932222, accuracy 100.0%\n", 1168 | "step 59 loss 0.019916045963472316, accuracy 100.0%\n", 1169 | "step 60 loss 0.020874485829553507, accuracy 100.0%\n", 1170 | "step 61 loss 0.02352369573570373, accuracy 100.0%\n", 1171 | "step 62 loss 0.03326503825041317, accuracy 99.0%\n", 1172 | "step 63 loss 0.019791643375603657, accuracy 100.0%\n", 1173 | "step 64 loss 0.021290815578892223, accuracy 100.0%\n", 1174 | "step 65 loss 0.030884203516995862, accuracy 100.0%\n", 1175 | "step 66 loss 0.018927539525833106, accuracy 100.0%\n", 1176 | "step 67 loss 0.018852402065537233, accuracy 100.0%\n", 1177 | "step 68 loss 0.02078797948389191, accuracy 100.0%\n", 1178 | "step 69 loss 0.01563093592456672, accuracy 100.0%\n", 1179 | "step 70 loss 0.01905328666327437, accuracy 100.0%\n", 1180 | "step 71 loss 0.01619222407425494, accuracy 100.0%\n", 1181 | "step 72 loss 0.024471109661991345, accuracy 100.0%\n", 1182 | "step 73 loss 0.01564616679328279, accuracy 100.0%\n", 1183 | "step 74 loss 0.019398658398722304, accuracy 100.0%\n", 1184 | "step 75 loss 0.020862785805321207, accuracy 100.0%\n", 1185 | "step 76 loss 0.014047328210772154, accuracy 100.0%\n", 1186 | "step 77 loss 0.01944617841459297, accuracy 100.0%\n", 1187 | "step 78 loss 0.021606808339660943, accuracy 100.0%\n", 1188 | "step 79 loss 0.014148971740402144, accuracy 100.0%\n", 1189 | "step 80 loss 0.014910996431333663, accuracy 100.0%\n", 1190 | "step 81 loss 0.01721007109292912, accuracy 100.0%\n", 1191 | "step 82 loss 0.012366212108223178, accuracy 100.0%\n", 1192 | "step 83 loss 0.012257445936183161, accuracy 100.0%\n", 1193 | "step 84 loss 0.012186318425017391, accuracy 100.0%\n", 1194 | "step 85 loss 0.01257294527319086, accuracy 100.0%\n", 1195 | "step 86 loss 0.012632904962825906, accuracy 100.0%\n", 1196 | "step 87 loss 0.01381341587243698, accuracy 100.0%\n", 1197 | "step 88 loss 0.013181444719459269, accuracy 100.0%\n", 1198 | "step 89 loss 0.012112123429310276, accuracy 100.0%\n", 1199 | "step 90 loss 0.011991619681594404, accuracy 100.0%\n", 1200 | "step 91 loss 0.01191484101087392, accuracy 100.0%\n", 1201 | "step 92 loss 0.011841435415422583, accuracy 100.0%\n", 1202 | "step 93 loss 0.011891835891202848, accuracy 100.0%\n", 1203 | "step 94 loss 0.011849492283132935, accuracy 100.0%\n", 1204 | "step 95 loss 0.011772922324668656, accuracy 100.0%\n", 1205 | "step 96 loss 0.011713712499624353, accuracy 100.0%\n", 1206 | "step 97 loss 0.011658024588577119, accuracy 100.0%\n", 1207 | "step 98 loss 0.011630878060508679, accuracy 100.0%\n", 1208 | "step 99 loss 0.011662977441662272, accuracy 100.0%\n" 1209 | ] 1210 | } 1211 | ] 1212 | }, 1213 | { 1214 | "cell_type": "code", 1215 | "source": [ 1216 | "h = 0.25\n", 1217 | "x_min, x_max = X[:, 0].min() - 1, X[:, 0].max() + 1\n", 1218 | "y_min, y_max = X[:, 1].min() - 1, X[:, 1].max() + 1\n", 1219 | "xx, yy = np.meshgrid(\n", 1220 | " np.arange(x_min, x_max, h),\n", 1221 | " np.arange(y_min, y_max, h)\n", 1222 | ")\n", 1223 | "Xmesh = np.c_[xx.ravel(), yy.ravel()]\n", 1224 | "inputs = [list(map(Value, xrow)) for xrow in Xmesh]\n", 1225 | "scores = list(map(model, inputs))\n", 1226 | "Z = np.array([s.data > 0 for s in scores])\n", 1227 | "Z = Z.reshape(xx.shape)\n", 1228 | "\n", 1229 | "fig = plt.figure()\n", 1230 | "plt.contourf(xx, yy, Z, cmap=plt.cm.Spectral, alpha=0.8)\n", 1231 | "plt.scatter(X[:, 0], X[:, 1], c=y, s=40, cmap=plt.cm.Spectral)\n", 1232 | "plt.xlim(xx.min(), xx.max())\n", 1233 | "plt.ylim(yy.min(), yy.max())" 1234 | ], 1235 | "metadata": { 1236 | "colab": { 1237 | "base_uri": "https://localhost:8080/", 1238 | "height": 447 1239 | }, 1240 | "id": "ByZKLTnPj-Tf", 1241 | "outputId": "5754caba-bbfc-4104-97c8-ff0b3717c9bf" 1242 | }, 1243 | "execution_count": 34, 1244 | "outputs": [ 1245 | { 1246 | "output_type": "execute_result", 1247 | "data": { 1248 | "text/plain": [ 1249 | "(-1.5978882018302847, 2.1521117981697153)" 1250 | ] 1251 | }, 1252 | "metadata": {}, 1253 | "execution_count": 34 1254 | }, 1255 | { 1256 | "output_type": "display_data", 1257 | "data": { 1258 | "text/plain": [ 1259 | "
" 1260 | ], 1261 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAi8AAAGdCAYAAADaPpOnAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjguMCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy81sbWrAAAACXBIWXMAAA9hAAAPYQGoP6dpAABuwElEQVR4nO3dd3hc5Z02/vuc6SNpRr3LqpZ7ww3bFAMG4xACCSGksJQQ8sILSVizm8X5ZZMl5fXuJlnS2BBICOCEwJIEkiWEYFwwBndbuMu2bKv3MqPp5ZzfHyPJljRVmtEU3Z/r0gUanZl5NBqfc89Tvo8gy7IMIiIioiQhxrsBRERERJFgeCEiIqKkwvBCRERESYXhhYiIiJIKwwsRERElFYYXIiIiSioML0RERJRUGF6IiIgoqSjj3YBokyQJbW1tyMjIgCAI8W4OERERhUGWZQwODqK4uBiiGLxvJeXCS1tbG8rKyuLdDCIiIpqA5uZmlJaWBj0m5cJLRkYGAODCT/4PMnSaOLeGiIho+nmlsgYAIGQVh30fu8WGf1p7z8h1PJiUCy/DQ0UZOg0MeoYXIiKiqbSlqhY6AEJ28N6TQMKZ8sEJu0RERBQVW6pqAUw8uISL4YWIiIgmbaqCC8DwQkRERJM0lcEFYHghIiKiSZjq4AIwvBAREdEExSO4AAwvRERENAHxCi4AwwsRERFFKJ7BBYhxeNm8eTOWL1+OjIwM5Ofn4/bbb0d9fX3I+7322muYPXs2tFotFixYgLfeeiuWzSQiIqIwxTu4ADEOL++99x4eeeQR7N27F1u3boXb7cZNN90Eq9Ua8D4ffvghPve5z+GBBx7AkSNHcPvtt+P222/H8ePHY9lUIiIiCiERggsACLIsy1P1ZN3d3cjPz8d7772Ha665xu8xd911F6xWK958882R26688kosXrwYzzzzTMjnMJvNMBqN6Hn2q6ywS0REFCWxDi52ixWPLPs0TCYTDAZD0GOndM6LyWQCAGRnZwc8Zs+ePVi3bt2o29avX489e/bEtG1ERETkX6L0uAybsr2NJEnCY489hjVr1mD+/PkBj+vo6EBBQcGo2woKCtDR0eH3eKfTCafTOfK92WyOToOJiIgo4YILMIU9L4888giOHz+OV155JaqPu3nzZhiNxpGvsrKyqD4+ERHRdJWIwQWYovDy6KOP4s0338SOHTtQWhr8BSgsLERnZ+eo2zo7O1FYWOj3+E2bNsFkMo18NTc3R63dRERE012iBRcgxuFFlmU8+uijeP3117F9+3ZUVlaGvM+qVauwbdu2Ubdt3boVq1at8nu8RqOBwWAY9UVERESTs6WqNiGDCxDjOS+PPPIIXn75Zfz5z39GRkbGyLwVo9EInU4HALjnnntQUlKCzZs3AwC+9rWv4dprr8WPfvQj3HLLLXjllVdw8OBBPPvss7FsKhEREQ1J5OACxLjn5Re/+AVMJhPWrl2LoqKika9XX3115Jimpia0t7ePfL969Wq8/PLLePbZZ7Fo0SL84Q9/wBtvvBF0ki8RERFFR6IHFyDGPS/hlJDZuXPnuNvuvPNO3HnnnTFoEREREQWSDMEF4N5GREREhEsri5IBwwsREdE0l6hLogNheCEiIprGki24AAwvRERE01YyBheA4YWIiGhaStbgAjC8EBERTTvJHFwAhhciIqJpJdmDC8DwQkRENG2kQnABGF6IiIimhVQJLgDDCxERUcpLpeACMLwQERGltFQLLgDDCxERUcpKxeACMLwQERGlpFQNLgDDCxERUcpJ5eACMLwQERGllFQPLgDDCxERUcqYDsEFYHghIiJKCdMluAAML0RERElvOgUXgOGFiIgoqU234AIwvBARESW96RRcAIYXIiKipLWlqnbaBReA4YWIiCgpTdfgAjC8EBERJZ3pHFwAhhciIqKkMt2DC8DwQkRElDSGVxZNdwwvRERESWA6LokOhOGFiIgowTG4jMbwQkRElMAYXMZjeCEiIkpwDC6jMbwQERFRUmF4ISIioqTC8EJERERJheGFiIiIkgrDCxERESUVhhciIiJKKgwvRERElFQYXoiIiCipMLwQERFRUmF4ISIioqQS0/Cya9cu3HrrrSguLoYgCHjjjTeCHr9z504IgjDuq6OjI5bNJCIioiQS0/BitVqxaNEiPP300xHdr76+Hu3t7SNf+fn5MWohERERJRtlLB98w4YN2LBhQ8T3y8/PR2ZmZvQbREREREkvIee8LF68GEVFRbjxxhvxwQcfBD3W6XTCbDaP+iIiIqLUlVDhpaioCM888wz++Mc/4o9//CPKysqwdu1aHD58OOB9Nm/eDKPROPJVVlY2hS0mIiKiqRbTYaNIzZo1C7NmzRr5fvXq1WhoaMBTTz2FLVu2+L3Ppk2bsHHjxpHvzWYzAwwREVEKS6jw4s+KFSuwe/fugD/XaDTQaDRT2CIiIiKKp4QaNvKnrq4ORUVF8W4GERERJYiY9rxYLBacO3du5PsLFy6grq4O2dnZmDFjBjZt2oTW1la89NJLAIAf//jHqKysxLx58+BwOPCrX/0K27dvxzvvvBPLZhIREVESiWl4OXjwIK677rqR74fnptx777144YUX0N7ejqamppGfu1wuPP7442htbYVer8fChQvx7rvvjnoMIiIimt4EWZbleDcimsxmM4xGI3qe/SoMes6FISKi5LalqhZCdmm8mxFzdosVjyz7NEwmEwwGQ9BjE37OCxEREdHlGF6IiIgoqTC8EBERUVJheCEiIkpQ02W+S6QYXoiIiBIQg0tgDC9EREQJhsElOIYXIiKiBMLgEhrDCxERUYJgcAkPwwsREVEC2FJVG+8mJA2GFyIiojgbDi7sdQkPwwsREVEcMbhEjuGFiIgoThhcJobhhYiIKA4YXCaO4YWIiGiKMbhMDsMLERHRFGJwmTyGFyIioinC4BIdDC9ERERTgMElehheiIiIYozBJboYXoiIiGKIwSX6GF6IiIhihMElNpTxbgARRW6g1Y6z27rRfmIQAFAwOx21N+Qha4Y+zi0jomEMLrHD8EKUZM682439LzZBEAFZ8t022OHA2e09uOJzpZj7sYL4NpCIGFxijMNGREmk4+Qg9r/YBOBScLn8/w//vgUtdaY4tIyIhjG4xB7DC1ESOfnXDghB/tUKInDyzY6paxARjcLgMjUYXoiShOSR0XbMPKrHZSxZArrqLXDZvVPXMCICwOAylRheiJKE1yMBcpjHuoIkHCKKOgaXqcXwQpQklBoRWmPoOfZqvQKadM7FJ5oqDC5Tj+GFKEkIgoDadfkQhCDHiMDM63MhKoIcRERRx+AytRheiJLI7PX5MBRr/U7aFUQgPU+DubcURvU5ey9YcXZnDxre74W11xXVxyZKdluqahlc4oB9y0RJRK1T4KZ/nYWDW5pxcW8f5KF5uYIIzFieheX/UBa1IaP+Rhs+fO4i+hvtl24UgBnLMrHygXJo0nj6oOmNwSV+ePYhSjKaNCXWPFSJpZ8vRc95GyDLyKlMgy5TFbXnMLXa8ffv1Y+f+CsDzYcGMNjpxPpvzYZSw85bmp4YXOKLZx6iJKU1qFC62IjSJZlRDS4AUPdaG7wuye+ybFkC+pvsaHi/J6rPSZQsGFzij+GFiEZxmNxoPjwQtJ4MAJzd1j01DSJKIMMriyi+GF6IaBRrryusejKWbk7epemFS6ITB8MLEY2i0inCOk6p5emDpg8Gl8TCsw8RjZJRqIGhWBv0GEEEKq7MnqIWEcUXg0vi4WojohTgcUloeK8HZ97txmCXEwq1iPIVWZi9Ph+ZpbqIHksQBCy4vQgf/PeFAAcAolLArBvzo9ByosTG4JKYGF6IkpzL7sW7m8+g74Jt5DbJ40XDrh40vN+Da75ajbIrMiN6zMpV2bD1unDk1VYIIi5N3hUApVrE2o01yCjQTKrd5nYHzmzrRmudCZJXRk6VHrXr8lEwOx1CsDLCRFOEwSVxxXTYaNeuXbj11ltRXFwMQRDwxhtvhLzPzp07ccUVV0Cj0aCmpgYvvPBCLJtIlPQOvNiE/kbbuNtlCZC9wPs/Ow9bf+STa+d9vBCf+ME8zF5fgII56ShaYMAVnyvFJ3+yAIVzMybV5gt7+vC/T5xA/dYuDHY6Ye1xofngAN79f2dwcEszZDnMHSiJYoTBJbHFtOfFarVi0aJF+OIXv4hPfepTIY+/cOECbrnlFjz00EP43e9+h23btuFLX/oSioqKsH79+lg2lSgp2U1uXNzTF3RZs+yVcXZHDxZ9qjjixzcUarH089E9efc32vDhMxfGtXn4+/qt3TAW61C7Li+qz0sULgaXxBfT8LJhwwZs2LAh7OOfeeYZVFZW4kc/+hEAYM6cOdi9ezeeeuophhciP7rqLSHrscgy0H7UNKHwEgun3+kKecyJv3Zg5vW5EEQOH9HUYnBJDgk152XPnj1Yt27dqNvWr1+Pxx57LOB9nE4nnE7nyPdmszlWzSOKm/bjZtRv7ULXGQsEQUDBnHTMvqkAkje84RXJG+MGRqD5UOgCeNYeF8ztDhhLIptsTDQZDC7JI6HCS0dHBwoKCkbdVlBQALPZDLvdDp1u/Ils8+bNePLJJ6eqiURTSpZlHH6lFafe6hw1cbb54ACa9g9g1vrQK34EEcitTvP7M0uPE2e3dePi3n54nBIyCjSovSEPFVdmQ1TGptfD6w4vcIV7HFE0MLgkl6Sv87Jp0yaYTKaRr+bm5ng3iShqLu7px6m3OgFgVG/FyPyQv3chs0wHIci/ZFkCZt4wfv5Ix8lB/O+/nMDJtzph7XHBOehBT4MVH/7yIt799zPwOCbeXSNJMjpPD6JxXx86Tg1Cki4FEWOJFgiRi0SlgPQ89YSfnygSDC7JJ6F6XgoLC9HZ2Tnqts7OThgMBr+9LgCg0Wig0UxuySZRPJnaHTi/qweWbhdUegXKV2ShcG4GBFHAyb92+C70ATohBBFQaESo9Qq4bN7RwzFD91t4RzGyykb/+3GY3dj5X+d8vRuXP/bQ/3efseDAb5ux6ksVEf8+F/b04cgrLbD1uUdu02WqsPjOYlRfk4tZ6/Kw57nGgPf3FcDLgjotoU5PlKIYXJJTQp0dVq1ahbfeemvUbVu3bsWqVavi1CKi2JEkGQe3NOPMu92+ISEZEATg3I4eZFfqseb/VKC/yR70MWQJ6D1nxSd+MA9HX29H474+yEMdJsZiLebfWojKNTnj7tewqxcelxQwFMkycH53L5bcVQptRvinibM7urHv+aZxt9sH3NjzXCPcdgm1N+Thwod96Dg5OO75BdG3W/biO0sCPofL7oXT7IE6XQENAw5NAoNL8orpv3yLxYJz586NfH/hwgXU1dUhOzsbM2bMwKZNm9Da2oqXXnoJAPDQQw/h5z//Ob7+9a/ji1/8IrZv347/+Z//wV//+tdYNpMoLo7+sQ1n3vXtzDzcYzJc3qS/0YZdPzsf9mPps9W46uFKrLinDNY+N5QaEel56oDF3lrrTCE3X5S9QOfJQZSvzAqrDS67Fwd/G3zY9vDvW1C5OhvXbaxB3R/acHZ7NzzOoV9eAEqvyMSyu8ugzx4/ZNTfbMex19tGTfgtXmTAgtuLkFeTPnKc1yXh/O5enNneDUuXCyqdiIpV2ahdl4f0XPbSkg+DS3KLaXg5ePAgrrvuupHvN27cCAC499578cILL6C9vR1NTZc+pVVWVuKvf/0r/vEf/xE/+clPUFpail/96ldcJk1xIUky2j4y4fzuXjhMHuiyVKi6KgfFCwyTXsLrsnlx8m+dAX8uS4Cp1QGVToTbHnxpTlqeGkq1b9KLOk0Z1nCL1xViuc/wcZ7wjgOAi3v64HUFT0SSJOP8B72Yc3MBln6+FAs/VYTeBiskr4zMMh30Wf7nuXSdsWDbv5+B5JVHDY21HzOj/ZgZ1z5WjdIlmXBZPXj338+i76JtZNjMbffi1N86cebdblz/zzXInzW5AnuU/Bhckl9Mw8vatWuDVsr0Vz137dq1OHLkSAxbRRSa0+LB9h+eQ2+DdWSVjyACjXv7kVebhusenwm1Przdl/1pOTIAKYzVNKJSgCBc6pEZRwBmT2CPoawKPfoabSGXLI+dKxPMYIcDgkKAHGT5tiD6tgUYptIqUDjPEPRxJUnG+z8/73f10XD7d//3Bdzxs4XY+3wT+puGqg3Lo4/zuCTs+NE5fPKpBZxPM40xuKSGpF9tRBQLu352Hn0XrAAuG9IZ+m/POSt2Px3+kI4/Los35IobAHAOeqHSK/yvJhKAvJlpqPWzkiiU2uvzggYXQQRyqvTImqEP+zGVGkWQlDXmuAi0Hh6Avd8d9BiPQ0L91i40HegP/HvJgNsh4fzuvoien1IHg0vqYHghGqPnvBWdJwcDXgRlCWg7ar70CR+Ay+pB04F+nN/di55z1pB78+hzVCHnnFx6bC/KlmZCob70z1WlFTF3QwFu+JfaUbeHK7tCj7m3FPj9mSACCpWIK79YHtFjll5hDF3t1wuULc2M6HGP/bk95DGCAmj7yBz6NZWB1qOmiJ6fUgODS2ph3ynRGM0H+kfvpOyHIAJNBwZgKNbiyCutOLO9e9QwkLFEi5X3zwg4v6JkkREqvQi3LfScEkEElFoRn356IUytDkAAMkt1I/NcJqL9hBkXPuz1+7OCORlYdncZMksjq26bU5mG/Nnp6D7jf8sCQfSFprxa/wXz/BnsdKLvYvAVV0BYHT4jJHf483gotTC4pA72vBCN4bZLIYd0BEGAy+7B7p9fwOl3usbNXzG1OfDu5rPoqrf4vf/FPX1h97zIEmBqcUClVSC3Og25VWmTCi5d9RZs/8FZ2Ac8438o+Ob7ZORPbFXONV+tvhR6hl/Dof8aCrW49h9rAq6A8uf87l6EdbgEFM4PPRF3OEDR9LKlqpbBJcUwvBCNkZ6vCTn8IUkyJI9vnx6/IUT2HXNgy/iaJ/Vbu0ZqnoRLqZv45OCxDv++BZAQsN39jXZc3DuxeSHaDCVufnI2rnqkEkXzDTCWaFE4NwOrH6rAx743B/pMVUSPZ+11hTU3SKEWMPeWAmSWaYOGHVkCZl7H3aqnEwaX1MRhI6IxqtZk48irLUGHIkSFAFufK/jw0lAQ6LtoG/m077J6cOjllojbNGN5ZsT38cfc7kBPgzX4QQJwdkcPqq/JndBzKJQiKq7MRsWV2RO6/+U06eGFtpq1uVCqFFj15Uq88916SB7J799l8Z3FMBRpJ90uSg4MLqmL4YVoDK1RhUV3FKPutbaAxyz+dDEa3u8N2UMDAINdzpHwcmFPX9g7QQO+YQ51mhKVa3JG6s60fWSG1yMhq0yHyqtyIqoya+11hT5IBizdztDHjdHXaMOpv3Wi+eAAvG4JGQVa1K7L8wWLMIe5vB4JjXv7cXZ7NwY7nRCVQujXWADmbigEAORU6LHh32bj0MvNaD8+OHKIPleFxXeUoOqq8dWGKTUxuKQ2hheiMVxWDzQGJYoXGdB5anBU4TV1mgKLPl2MWevy0XRwIKzHU2kvXbgHO5y+3ppw9jwUALVegRv+ZSYcA25s/+E5WLqcEBQAZN8k1cOvtGLFfTNQc214vSTqtPB6MjTpkZ0aGvf1Yfd/XwBwqSfK3OHAwd8248LuXqzbVAtViKEvt92Lbf95Fj3nrEH3cxpFAKqvzkFarq+4nSzLaDtqQudpCyD4tluADNh63Gja34+ypZkh20HJb3hlEaUuhheiIbIs49gb7Tj+lw5IHnlUcbqi+QbUXJeLkkVGKFS+MFK+Iss3BBPkIqvSiciffWkiqVIrhnVR1mWpMO+WQlRdlQ1ZBt7cdBIOs6/WyeXBR/LI2PurRmjSlWEtQc4u1yMtTw1rd5AeGAF+90MKxNLtxO7/vjC+h2To9+xrtOHgb5ux6sGKoI+z74Um9J63jrrvuKZdtgeULAHlK7Ow4r4ZIz8/8243Dr/SeqkJlz1O60cm7HzqHNZtqo1o0jAlFy6Jnh44YZdoyNE/tePon9oheXxXvMuL07UdNaP7rHUkuABA1TU5UKcFKCA3ZM7NBaOGTMqWZYU11HTtV6sxe30+1GlKnNvZA7vJHfh+AvDRH1pD1pYBAEEUsOhTxUF+DmjSFKhZG/58l7Pbu4P+XJaACx/0wTnoZ3XTEFu/C417+0K+NkqNiLJlmZh1Yz4+9r05uPqRqpG/idcloe4PgYf6ZAnoPGXxbQhJKYnBZfpgeKFpyeOS0H3Wgs7Tg3AOemA3uXH8L8GLoZ36WyesfZd6LDRpSqz7l9px2wQMh5nqa3Iw//aiUT/LqdCjYE56wMAjiEBebRpyqi8t5z3/fm/w3hoZGGhx+GrAhKHqqhws+WyJb1hFxKX/wrej87pvzIpoJ+m2o+bQq7O8MrrP+l82DgAdJwIXBbyc2y6hZJERy+4uQ3b56CXPrUdNcNuCj8cJ4tDrSSmHwWV64bARTStej4Sjf2rHmXe7RpYqCwogq0wfstCZIAAXdvdi/icuBZLsCj1u+68FuPBBL5oPDMDt8MJYosPM63ORW53md3jimq9UY9t/+jYPvHxoSpZ8xeeu/Vr1qPs5gvRYXC7c4wBg3i2FqLgyG+d29mCgxQ6FSkTJYiNmLM8c1bsUDkkKbwJysOO8ERSOO/Nut985PvYBd8i5MrIE2EJsNUDJh8Fl+mF4oWlD8sp4778a0HZ8dBl52QvfLsShCP5X66h1Csxal49Z68LbIFGTocTN356NliMDaNjVC1u/C/osFaquykXZ0kyIytGBR5+pCjrkcvlxkUjLUWPRHYGHkMKVW50Gc5sjZM/J2J6Sy0VSzffyjR0vpzWE3nJBEAGdMbLXKRIuuxeyJEOtV3BezRRhcJmeGF5o2rjwYR/ajpkn/gBy5KtwAhGVAmYsz8KM5Vkhj61em4uDv20OfGEWgJxKfdzql9Suy0fDe4GHYoYnPKfnBa7am1uThvR8NSxdoZdyB9rLqWSREUqtCI8jcIqSJaDqqsnXnxn1mLKM8+/34tTfOjHQ4gtWaXlqzL7JF2jHhlGKHgaX6YtzXmjaqN/aFVa11kBkCSiPQuG1SFVfnYOMfE3QicFLPlMydQ0aI6dCjwWfHBpKG/P6CqKvp2nF/TPG3/Hy4wQBax6qDPlcguhb5eWPUiNi4Zg5RmPvm1uThqL5hpDPEy5Z9q322vNcIwYum3Nk7Xbh0Mst2Plf50YmgFN0MbhMbwwvNG2YWuxh7yc0liD4dk3OKotss8JoUOkUuPEbtcip9A27CCIgKHwpQZ2mwLWPVaNwXvQuyGNtqaoNWTdj0aeKseahChiLL/X+iEoBVVflYMN35iA9N/ReSXkz0zHvE4XBDxKA2hsDl/evvDpn9B5HQxOSASB/dgau+6caCGL0ekIu7u1Hw66hXqex7y0ZaDtuxqm3O6P2fOTD4EIcNqJpQ1QK8Lonll6KFhiw5uHQPQOxos9W4+Z/m4Oe81a0fWSC5JGRWapD2bLIJ9hG4vLQsqWqFv9w/kzAYyvX5KBidTasvS54nBLSstURF4Rb/OliyB4ZJ9/qHLX1giD6emeufrQKmSX+A+TZHd048GIzJEkeqQcDyVdbZ+UD5aiMQa/Z6b93BZ8kLAOn3+nC3I8VRDU0TWcMLgQwvNA0UrLYiMZ9/WEtyR0hAGVXZOKar1UlxATM3CrfrtJTYexFQu5rCRlgBEEIq5cl2P2v+FwpZizPwpltXeg9b4OoFFC8yIja6/NGKumO1XSwH/uev7QJ5uUrx7wuCQdfakbR3AzfpN4okSUZvaH2iQJg73fD1u9GWo7/tlP4GFxoGMMLTRuzby7Axb39kd1JBkztjoQILlPJ30VCyC4NK8BEQ25NGnJrwuvpkmUZH/2hLWAPiCwBTosH53b2jFrmTsmFwYUuxzkvNG3kVqXhyi+W+y5yEWSRaZZbgl4khm9LpL1jTG1DBfpCFPJriHJxOkEUhmr5BD9On62CPit2y7OnAwYXGovhhaaVmrW5uOV7c5CeF14XviAChXMzQh+YIsK5SCRagHFZwivOF06tnEjNWp8XvLihAMy+KZ/zXSaBwYX8YXihaUedpoQl2MaEl5ElYOYNgVe3pJJILhKJFGB0WeEFUa9bgsMUveq6/c12HP1T8C0lihcaMHt9QdSec7phcKFAGF5o2mmtM4W9ZHrFvTMCrm5JJRO5SCRKgMnI1yBvZujhG69bxrv/cRZeTyQztv2zdDnxzvfqYely+j9AABZ+qghr/7GGReomiMGFgmF4oWnH45KCFnwbljszDbXrUr/XZTIXiUQJMEvuKgk9j0kGBprtaDk0MOnnO/5mBzwOb9CVa06LF6KCwWUiGFwoFIYXmnaMRdqQy6UFEcifmT41DYqjaFwkEiHA5M/KwMIw9mkShMlP3PV6JFzY3Rv8PSQDDe/1QA5z00q6hMGFwsHwQtNO0UIDdJmqoJ/UZQmouW78zsWpJJoXiUQIMOHUUZHlod2nJ8Ft9YZV7NDjlOAOss8SjcfgQuFieKFpRxQFrHyg3PdNgAAz79ZCGArjs9HhVIjFRSLeASac3aIFEUjLnlyxOKVOEdbyeUEhQKnhKTZSk3lPSl4ZPSedaN5lQ+teOxwD3ii2jBIJi9TRtFS62Ijr/3kmDv2u2VcjZIgmQ4kFtxVi1k35cWxdbMXy0+1UFrIbq2BOBnSZqqA9K7IEVF2TM6nnUapFlC7NRMvhgYBDR4IIlK/M4pyXCGypqp3Ue7LnpBNn/jQI16A8UrDwrADkL9Jg1h0ZUKj5t0glDC80bRUvMKBo81z0XbDB2uuCWq9A/qyMlF4dMhXd8vEKMKJCwOLPlGDPsxf9t0sEssv1KF2SOennmv+JQjQfHgj4c0EUMO8WLpGeKr2nnTj+kvnSKsLL/tv1kROuQQmLvmRkvZ0Uwj5NmtYEQUBOVRpmLM9C4TwDg0uUxGsIqfrqHKy4dwYUKt/fUVAIIyvLCuZk4Pqvz4xKb4jd5IEQZNrL4k8XI2uGftLPQ6HJsoxz/2sJujnmQIMbvafDq+1EyYE9L0TTQDwmQg73wEy12nV5qFiTjYt7+jDY4YRSI2LGskxklUcnTHicEj747/OBSwUJwMm3OjF7fUFKh+FEYW7ywN4TYmK0CLTvdyB37sQ3DaXEwvBClOLiuYJDyC7FFmDK57+odQrUXh+bGj0X9/bBbQ9ysZQBh9mDliMDmLE8KyZtoEuc4UzKlQB7HyfvphKGF6IUFo3g4uj3on2/A+YmNyACWTVqFC7TQp0W3qhzvAJMrPSet0JQAHKQa6GgENDTYGV4mQJKXXjvQ5WesyRSCcMLUYqKRnBp22fHmdctvm+Gxkn6z7px8R0r5t1tQM6c8LrhUynAhDfpU+bk0CmSWaWCUi/AYwtee6dgCYeMUgmjKKUsl9WDgWY7LD1OyEG3/k1dkwkuvfUunPnT0ETIy18+GZA8wPEtZljaw9+pWcgujfs2AhPhGPTg5N86sffXjTiwpRmadEXQXhfA1ytTOGf67EYeT6JSQPl1QeYziYDGKCJ/cerWbZqO2PNCKWew04G6P7ahaV//SB2OrBk6zL+tCOUrpkc3/mRrZgBA4zbrSL0Mv2Sg5X0bZn/GEHHbkqUHpn5rFw79rgWSJI+sWpK9uPT/fqa+CCKQnq9B4TyGl6lSerUObquEpp12COLQ32Xovas1ilj4pUwoNewJSyUML5RSTK12vP2d+nGb5vU32/H+z87D+rlSzP1YatffmExw8bplOE0SvE4J5sbgvSqyBHQddWL2Z8J//HgWsYvUhQ/7cOCl5pHvL+9tGe7IG7lQ4tL36jQl1j5WzWGjKSQIAqo2pKNwuRbt+x2wdXuhUAvInadB7jw1iwWmIIYXSikfPtfof7ffoYvN4d+3oPQKY8qW/p9ocHHbJDRus6H9gANeZ/hDbJIbkKXI5nckQ4CRJRl1r7UGOcD3n4K5Gei7aIPL6oUmQ4maa3Mw66Z86LMmtwUBTYw+V4nqj6X+hqo0RXNenn76aVRUVECr1WLlypXYv39/wGNfeOEFCIIw6kurTc0LDYXP3OHA/hea8Or/qcNv7zmEP33tKI690Q6n9VLvQH+TDb0N1qC7/QoicHZ7zxS0eOpNOLhYJRx+egAtH9gjCi4AoDaIE+phiPc+SKH0NFhh7QlR1Ezw1Xz5zDOL8YWXrsCd/70IS+4qZXAhmgIxDy+vvvoqNm7ciG9/+9s4fPgwFi1ahPXr16OrqyvgfQwGA9rb20e+GhsbY91MSmAdpwbx12+cxNkd3XDbvIAM2Prc+OhPbXjrm6dg6/NdZHov2EI+liz5lrqmmsmEgPNvW2Dv9Qae2xKIABRfOfEPFokcYByDYUxElgGHybePkhDOTo1EFDUxDy//9V//hQcffBD3338/5s6di2eeeQZ6vR7PP/98wPsIgoDCwsKRr4KC1J6jQIG57F6899Q5eD2y36EgW58Lu39xAQDCHtcWUmz8ezJLoj12CR2HnBEHF0EEtNkiSlbrIn7OUY+ToAFGlxl6h2oIgH6SO1QT0cTENLy4XC4cOnQI69atu/SEooh169Zhz549Ae9nsVhQXl6OsrIy3HbbbThx4kQsm0kJ7MIHvb5qpgEurrIEdJ22YKDZjoLZGb4VBsEIQNH8yFbHJLLJ1nKxdnpDLvv1J7NGhSUPZ0EVZoGwYBIxwORU6pFREKIuiAxUX5s7NQ0iolFiOmG3p6cHXq93XM9JQUEBTp8+7fc+s2bNwvPPP4+FCxfCZDLhhz/8IVavXo0TJ06gtHT8CdrpdMLpdI58bzabo/tLUNT1Ndow0GyHQiWgYE4GtIbAn3I7Tw4GX64LAALQftIMpUYBhUqA1xXgYAFQKAXUXJszqfYniokEF8kro/eUCx0HHHAMeMPee0epF1C1Pg0QfUXB9LnRPXUk2iReQRBwxedK8d6PG/z/XAQMxVpUrJweS++JEk3CrTZatWoVVq1aNfL96tWrMWfOHPzyl7/Ed7/73XHHb968GU8++eRUNpEmqK/Rhr2/akTfxUtzUwQFUHVVDpbfXQalVjHuPrKEsIY0Wg+b0HFyMODPBdFXGfXax6qDhqVkMZHg4rFLOPq8CeYmz6VAGOYImj5PgYKl2pHdmmMh0QJM2dJMrHmoAvteaILHIUFUCJBl3/Bl3sx0XPOVKijUrPNJFA8xDS+5ublQKBTo7OwcdXtnZycKCwvDegyVSoUlS5bg3Llzfn++adMmbNy4ceR7s9mMsrKyiTeaYmKg2Y53vlsPr3v0xBXZC5zf1YvBDifWPVE7ricgu1KP5sMDwQOMjKDBBQC0BhXWPTETxpLJzdFIBBMdKjr16iDMzUMTUYdfzzDnupgbPdjz/V7M/ZwB2bNiN88j0QJM5ZoclC3LQuP+fpjb7FCoRJRckYmciujsUE1EExPTjw1qtRpLly7Ftm3bRm6TJAnbtm0b1bsSjNfrxbFjx1BUVOT35xqNBgaDYdQXJZ5Dv2+B1y35XcYsy0BXvQWN+/vH/azm2tygS3EFEVDpxJGKp4HYB9wpMVF3osHF1u1B7ylX5CuKLuNxyDj2gsm3QWMMJdocGKVGRPXVOVhyVykWfqqYwYUoAcS8z3Pjxo147rnn8OKLL+LUqVN4+OGHYbVacf/99wMA7rnnHmzatGnk+O985zt45513cP78eRw+fBh33303Ghsb8aUvfSnWTaUYsfa60H7MHLz+igCc2TZ++bwuU4Urvzhj5JhR9xEBhUqE1qAK+tjD+pvskTQ74Uxmcm7PCVdYk5nFYH2xsi9oXtga+6XmiRZgiCixxHzOy1133YXu7m5861vfQkdHBxYvXoy33357ZBJvU1MTRPFShurv78eDDz6Ijo4OZGVlYenSpfjwww8xd+7cWDeVYsTS5Qx5jCwDgx3+j6u+Jhe6LDWOvd6G7rO+C6cgAjOWZ2Hhp4rxwTMXwmpHMpcIn+yqIq9LhiBcKmvvlxBGx4wM9J9xw2WRoE6P7WefRBtCIqLEMSUTdh999FE8+uijfn+2c+fOUd8/9dRTeOqpp6agVTRVlNrwLnJKrQhJkuFxSlBqRIiXDRcVLzCgeIEBtgE33FYPdJkqqNN8b9+SxUb0N9qC9r6ISgH5s5KzbPhkgwvgm3AbsndKBuQwN4l2W2MfXgAGGIoex4AXXXVOOM0SVHoB+Yu10OeOXyRAySHhVhtR6skq10OfrYKtL/BcCUEAVDoFXv3SEXjdMkSVgKo1OZh7S8GofYj0mSpgTAGxmWtzceJ/OyBL/vsNBAGoujoHmvTke7tHI7gAQO58DRRaC7yO4H0rSj3gCV2oGKq0qVtlwwBDkyFLMhresqJlt2/YeHgzzYtbbchfosGsOzJiuoqOYoPr/CjmRFHAvFuDrC4bGs7ob7LD6/ZdXCW3jIZdPXjrm6fQcy74HAt9thrXfK0aolIYPXF36HyUOzMNyz4/uYt/PEQruACAQiVg5m3Be54qbtSjaIUu+NwYAciqVU1Jr8uop02AOTCyLKPnvBVNB/rRccIMyTOJ2c80Zc6/bUXL+3bfmKg8tDv40J+uq86J068FX6k4EbIko++MC/V/HMTJl804/7bVtwUHRU3yfRSlpFR7Qx6s3S6cfKtz5JMPgNH1RsZcC2QJ8LolvPeTBnzyqQVBC6qVLjbilv83F2e2dqFxXz88LgmGQi1qb8hD5VXZUCiTK6dHM7gMK7xCC1EpoOGvFjgHLo0hqdIEVKxLQ/EqLVyDEtr3O+BxyMDYYSbB14tVsS4tam2KRDx7YNqOmnDwty0wtztGbtNkKLHg9iLMujEvor2NbANuNLzXA1OrAwq1gNIlmShZbEzqOVmJyjUo+YJLIDLQ/ZETlus9SC+MzuXQOSjh2PMDsLR5fee6ofNb0w4bSq/WofqWNO6FFQUMLzQlhiuWVqzOxtnt3ei7aINCJSI9X4Pz7/cGLf9vH3Cj5cgAZiwPXs3UWKTF8ntmYPk9M2LwG0ydWASXYfkLNcibr4bponto7F9EZrVq5MKpMSiw5P9k4tiLJjj6pJGeLFkClFoBcz5ngLE8fkX+hgPMVGo5PICdfirtOgc9OLilGc5BDxbdURzWY518qxNHXm259H4XgIb3epGer8ENX69BRsHEN7pMZrHqUes66gg+SR2+YaTOww6kf2zyc+Ikr4yjvxqAtcvXyzLyIW2oDS3v26HSCyi/Pj4fAFIJwwtNqexyPVbeXz7y/aGXWyAoEHR/HUEhoKveEjK8pIJYBpdhgiggsypwobm0QiVW/nM2+s640H/ODVkCMkqVyFugmXZzAySvjL3PNwZdhnXsjXZUXZ2DjHzfXkiyJPutTdTwfi8O/35M8Bp6XGuPE+98/wxu/Y95UOv8TyKVPDLajpth63NBk65EyUKD36rUySaW73nXYOhVdjJ8PTTR0HvKBWtH8OGhph02lF6lh0I9vf4tRRvDCyUBOeSnp1QwFcElXIIoIGe2BjmzQ2xOmOJaPzLBYQq+BEsQgeN/bofkldG0vx9etwx9jgq1N+Sjdl0e1DoFZEnGR39sDfgYwz2M59/vxeyb8sf9vGFXDw6/0grn4KW2KDUi5n+iEPNuLUzaYYhYv+fVGULonhcA6ozoDCt3HnGE3IvN6wJ6T7uQv3B6/9uarOSaCEApJ29mWshdjWWv77hUlkjBhS4xtzlCVm+WJaBhVy8u7ukbmXBu63Wj7rVWvP3t03AMetBz3gpbb4jKxDJwfnfvuJvP7ujGnucaRwUXAPA4JdS91oYjrwYORYlsKt7zeQu144pbjiVLQMGS6AzXua1yWFWsPbbo9PRMZwwvFFelSzKhNSoDrnARBN/EyLJlmVParqkUz+AieWVYOz2wtHtGLrx0iVIjht3rN66OjgwMdjqw99cX4bSEt9JkXEBxeHHod8Hn+Jz8aycs3aELQSaSqXrPazJElKwJsp+ZAOQtUCOtUIGB8y5ceMeK829b0H3cCckb+b8HbaYY1lVVk8lL72Rx2IjiSlQKuPar1Xj3P85A8sijLgCC6KuKe+3XqpJutVC44hVcJK+Mpp02tH5g931aBKDQCChaoUXFjXooNan5ekeqZLERB15qnvD9ZQloOWzCzOvyQh8sAGk5o+ciNR0cgMcZ/FO6IPp6fsKdNBxvU/2er/5YGmRJRuuHvpViwnAlaQnIW6hB+fV6HHyqH9ZO72UT1O1QpQuY+3kDsqrD34i0cLkWnUeCB0lVmoCsmbHb3HS6YHihuMurTcfHvjMHx//SgYv7+iF7ZQgKoHxFFubfWoTMsuTfCTqYeASX4y+Z0Vc/eqNGr1NGy247BhpcWPxQFpSa5JxHEU3peRrMWJGJ5oMDYe2f5ZcMOAbdyCzVYqDVEXhYQQZq1uaOummw0wlBIUAO0QswGMYWHIkgHmFdEAXM/EQGyq7Wo/OIY2SVXcESDZQ6EQd/3A+X1ffHvfxv7LbKOPprE5b830wYSsNbYZdZpUL2LBX6zrgD/p2rb0nnsvgoYHihhGAs0WHNw5W48oFyuGxeqPUKKNSp/el/S1VtXIaKOg450Hfa5f+HMmBp96JphxVVNyfndgrRtupLFbAPnEX3GevoGkXw9RyGVaxOFnDF58uw/Qdn/f5YEIHMMh3KV45eUacamuwblICAK5QSUbzmdWmzFOOWKF94xwqXRfIfNORLlXgX3m8M6zkEQcC8u42o/+MguuqcI7WRZMnXs1n98TQULp2ey+GjjeGFEopCLUKX4qEFiF9wAYDWD+zBV0TIQNteByrWpQUtDDhdqHQK3PiNWWg5PIBzO3tg6XJCk6FE5eocDLTbcfbd7pC9MrlVehhLdLj2q9Ujc2AEBQDJt4y3cJ4BVz1cCYVq9Hu/bFnm+OXVY8heYMbyzMn9klMgnu/5QNoPBOkJAwAZ6DvtimgjUoVawNzPGVC53oueE054nTK02QrkzddweXQUMbwQTbF4nsRlSQ5ZhwIAPHYZTrMEXXbyfKKPJVEhYMbyrHG1hgY7HTiztTvg/QQRyJuZDmOJb+izbFkmihcvRMshE0xtdijUIkoWG5FZ4n9oNCNfg/Irs9C0r9/vxGFB9O0dVjA3Y+K/3BRIxOACAG5LeGOBE9lFXZetQNnV+ok0i8LA8EJTzuuRfGP5ANILNCk7GdefuJ/EBYSsQzFy6PT5s0xYRoEWy+4uw8EtzeNeV0EE1GlKrHqwfNR9FEpxaHgovKKLqx4oh8viQfvxwZFhq+H/Gkt0uO7xmoSu8xL393wQSp0wMmE9mKnciJTCw/BCU8brknDsL+048243XFbfp391mgKzbszH/FsLp8Ucl3gTBAFZNSr0N7jH7110GV2OCI0xtf8e0TL7pnyk56px/C8d6GnwbSIqKgVUrsnGwtuLkZY7uZUlSq0C1399JjpPDuLcrl7Yel3QGpSoWJ2N0iWZCT35M5GDCwAULtOieZc9cJgXfJNwNVEqYkfRw/BCU8LrlrDtP8+i64xl1InCZfXi2J/b0Xl6EDd8fea4Mf9UkUhF6Mqu1qP/rCnoMaXX6BP603yiKb0iE6VXZMJucsPj8EJnVEW1dL8gCCicZ0DhPEPUHjPWEj24AEDJah3a9jngdQYuLlexjkM/iSg1rxSUcE6/0zUuuIyQga56C+qDzB1IZokUXAAge5YaFTcNnZAvPwMMZZXCZRoUr+SKiInQGVXIKNCmxJ5Dk5EIvYzh0GYqsPjLxpHtAQTx0nCpqALm3W0Iug8YxQ97XijmZFlG/TtdIWf112/twpwN+ZP+xO92eNF30QZZBrJKddBkxO9tnmjBZVjFDWkwlqvQ8oEdA+dckGXAUKZCyRodcuep2esSZ16XhMb9/eiqt0CWZeTWpKPyyqykCEWJ+p4PJKNEhSufyEbPCRcGGlyQvUB6qdJXB4bFGhMWwwvFnMvqha0vxL4uAKw9LrjtEtT6iZ2gfXu9tOLsjh54Xb4JHaJCQMWqLFzx+TJopzjEJPpJPKtGjawafqpMNF31Frz343O+5dRD186G93px+OVmXPVIFUoWhVdzJB4S/T0fiKgQkL9Qw80SkwhjJcVcJBMKxQm+I31zas6g/p2ukeAC+KrJXviwD39/8jScluC7A0dTsp7EKb5M7Q5s+88zcA5NaJelS0Xx3A4JO586h97z1ji2MLBEe89b2j3o+siBnlNOeJzctyvVsOeFYk6lUyC7Qo++RlvQWf05FfoJd4uf3dGD7rNWv48vS4Cl24ljf27Hsi+UTejxI5FoJ3FKHqfe6vBV7A0wNwwycPwvHbj2seqpblpQifSeNze7ceZ1Cyytlz6siCqgZI0OlTelJfTqLAofe15oSszZUBByzsucDQUTfvwz73YF/bksAed29sDrju1W9Il0EqfkIssyLnzQF7RarywBzYcH4HaEt0v1VEik97y5xY26ZwZgaRvdyyq5geaddpz6vRnyULU/W48XXR850H3MCddgbM8LFH3seaEpUbEqCz0NFtS/0z1qf5jh/5+9Ph/lV4ZXtGssWZJhbg+9MZ3HIcHW70ZGfmzGtRPpJE7Jx+uW4XWHs08S4LJ5oUqAybuJ9p4/+4YFkhcBPyh1H3Oh45ATnUccGDh3aR6eIAL5izSYeVs6lDp+pk8GDC80JQRBwLK7y1A034DT73Sh+4wFAJBfm45Z6wtQssgw8RUuAsLaeRcAFDHaqyfRTuLB2Hq8aP3Qjq4jDnicMrRZChSv1KJopZarK+JIoRKg0olw24P3AogKAZq0+J+6E+09b+3wYLA5xLw2ATjzp8FxWy3IEtBZ54S104MlD2dxD6IkEP9/ATRtCIKA0iWZKF2SGfXHLVlkQGudKXCXuwAYi7XQZYW3tX0kEu0kHkx/gwvHnjdBkjBSYdfe40XDW1a073dg8UOZEe/hQtEhCAKqr8lF/daugO9jQQTKV2bFPWQm4nve1hPGUJrs28gy0M8sbV6077ej9CoWpkt0PEtRXHkcXnickx9vnrOhIPjOvjIw7+OFUa9fkogn8UDcdgnHXzT7utXHvlYyYOv14vRrg/FoGg2Zs6EAKp3C775SgggoVCLm31Y09Q27TKK+56PVW9K61xGVx6HYYs8LTTlJktGwswen/t4Fc5vvRJFZpsOcm/NRdVUOBDHyk1DB7Awsv7cMB15s9junZu4tBahckx3NXyNhT+KBdB4eKoMeiAT0nXbB3uuFLif+8ymSjSzL6Dpt8e1vJPjek7nVaRE9RlqOGjf96yzs+kkDzO1OX4gRfL0F+mw1rv5KFYzF8at+nMjveWOlCgqNEPw9HgZHb+JMhqbAGF5o0voabTi7vRt9jTYoVCJKFxtRfU2u38q2kiRj98/Po+nAwKjbB1rs2PNcIzpPDWLVgxUTCjCz1uUjb2Y66rd2of34IGRJRt7MNMy6MR8FszMm+uv5lcgn8UD6z4YuFAgA/edc0OXoYtya1NLfaMP7T1+Aud3hCxwyIMtAdoUeVz9ahYyC8CeJZ5bocOt/zEPnKQs6Tw8CMpBbk4biBYYJ/buIlkR/zytUAkqv0qFxm21SjyOqON8lGTC80ITJsoy6/2nFiTc7R/V2dNVbcPSNdlz3eM240HB2W/e44OJ7MN9/zu/uQ8FcA6qvzplQm7LL9Vj1pYoJ3TdSiXoSD0QKY0IzgODDbwliS1Ut/uH8mXg3AwBg7nDgne/Xjwx/Xv769TfZ8PfvnsYt35sLXWb4860EQUDh3AwUzo1u6J6oRA8uwyrW6eHo86LziNM3KWL4byEAohpQp4lw9EsBVyMJIpA3n1WnkwHnvNCEnd3WgxNvdgIYc8GTfaX6t//gHKy9rks3yzJO/71rZANAvwTg9NudsWnwNJdRqgr+2g9JL07szzTDF9BE2fzv+J/b4XFKfkOfLAHOQY/vfZ+kkiW4AIAgCph9VwYWfdmI/AUapBUqYJihRNWGNKz6lxxU3ZwWvN4UgNKrOVk3GTC80IRIkozj/9se+AAZkDwSzmy7tFO00+LFYKczZLG6/iY73HYv7ANuuOwcf46W4hUh5koIGDnZJ7pECTAep4SLe/pDFpY7uyM5d0xPpuAyTBAEZFWrMffzBiz/x2xc8UgWZlyrhypNRP4iLSrXBwgnIjD38wakFyX++58YXmiC+httITdblCWgcV/fZTeEP5HuT187ij9+5Sj+58t1eOd79WitM020qTREm6VAzSfSfd+M7YERAYUKmH3XJOrtTLFECDDOQU9Yw3Euqzcm1Z1lWYapzYHuMxZYe1yh7xCBZAwu4fA7b0gAIAGOfn5YShaMmDQhHkd4J+LLC25pMpRIy1PD2h36JHv5/brPWrDjR+ew9AulmHPzxLcQIKB0tQ4ag4jGbVZY2oZO1AKQO1eNypvSkFYwuVOC5JVhuuCG2ypBbVDAWK6M6SRTIbsUcl9L3ObAqMLcAV2hEiBGuUBi04F+fPTHNphaLy3tLZiTgSWfKUFuTWSrnMaKdXCRZRmDzR5YOjwQlQKyalTQGGK/wq3nlBPn/+ZnY8uh/NnwVyv0BUrkzOK8l0TH8EITkh5GiX1BAAyFmsu+FzBnfT4O/rYlouca7pI/9LsWFM03ILOUK2EmI2++BnnzNbD3eeF1yFAbRajTJt8J27bPjgvvWOG2XOqJ0GaJqL4lHXkLYrMlAxDfAKPWK1C80ID24+agheUqVmdHtUfrzLvd2P9i07getK76Qbzz/Xpc//WZKJwzscm+sQ4u5hY36l8bhLXjsl4OwVeev/ZT6TEtwNe80+57zYJsENv8no3hJQlw2IgmJC1HjaL5GX6LaQ2TZWDmDXmjbqtdl4+SJcYJPacgYtQcGpocXbYC6cXKqASXpp02nPmTZVRwAQBHv4QTvzWj43BsC3/FcwgpaNG4oa0rJrPp6Fj2ATcObGnyfeOnzL3klfHhMxcgSROvdxKr4GJp86DumQFYO8cMz8hA10dOfPScyberdgx4nDJMF90h59wNNLjD22OK4orhhSbsis+XQVSK/quBCkD+rHRUrBxdGE5UCLj2a9VYdncZ0vMvfbrRZYXuBJQl+OpeUEJxmr04/7afrvjLnH3DAq8rtheEeAWY/Np0XP1oFRQqwfepXsDIvwmVVsT1j9cgsyR6vYUNu3pCXoBtfW60HzNH/NhbqmpjOsel4S0LJA/8t18GBps96KwLvcnqREQSiuQYBSiKHg4b0YRllemw/l9nYe+vG9F38VJhKEEBVF2Vg+V3l/kd5xcVAmavz8esm/LgsngBwRdKdv3kfMjnTJbJpNNJx8HQvSpep4zu404UXhHb6rDxGkKasTwLBXMzcH5XL3oarL5dimdloHJNdtR3f+5rtIda7QtB9K3aK1kUfi9nrIOLY8AbulCiALTttaNoWfTfJyqdAJVegNsW/NVTpQtQaHmeSXQMLzQp2RV6fOy7c9B30YaBFjtEpa+4ltYQuiCXIAgjVXjzZqaPKnTn93gRCVO0iy6x9XiDzyOA729n656alRzxCjCaNGVUh4cCEZUCBCH44j1ZRkQThGMdXADA0Rfexon2GJXnF0QBxat0aNxuCzrnpWSVjh+SkgCHjSgqsiv0qLoqBxVXZocVXMbSGVUoX5kVcg5N7Zg5NBR/CrUQsvadLEdv47xwpNry3ssVLzSEroIs+44Lx1QEFwBQaMO73Cg1sXuflF2tgz5P4b9YowikFShQejUXBCSDKQkvTz/9NCoqKqDVarFy5Urs378/6PGvvfYaZs+eDa1WiwULFuCtt96aimZSnC2/ZwYMhdpxJ5bhQLPy/hkwFMVvU7pUJ3lkdNY5cPT5ARz8aT+OvWhCz0kn5BATP3PnasK6mObOndoVHEJ2adyL2MVC+fIsaI3KgEF/uIcynHk2UxVcACC9UAFtVohLjgDkL47837jpohvHt5iw65vdeG9TNw7+pA/t++3javAodSKWPJyJgis0EC4bzRMUQOEVGix+KDOmq50oemL+V3r11VexceNGfPvb38bhw4exaNEirF+/Hl1d/stlf/jhh/jc5z6HBx54AEeOHMHtt9+O22+/HcePH491UynONOlKrP+32Vj4ySJojUMjmoLvE+SN36jFzOvY6xIrTrMXB3/Sj1O/H0TfGTcsrR70nnLh+ItmHHlmAB574HSSVaNCWqEi8NlEALJrVZOuITMRqRhgFGoR1//TTKi0Cr8BJqNAizX/tzLk40z16yKIAsqvD1J6X/AVSiy+MrLw0rrHjiO/GEDPSRckt2/o2dLuRf0fLTj2wvjVSyq9iDmfMWD1/5eDhQ8YsfABI1Z/Mwez7zRApWNwSRaCLEdQ9nQCVq5cieXLl+PnP/85AECSJJSVleErX/kKnnjiiXHH33XXXbBarXjzzTdHbrvyyiuxePFiPPPMMyGfz2w2w2g0oufZr8Kgj11tCYotWZbhcUpQKMWoF/eKhqn8xBprsizj0M8GYGn3XNrI7nICkDNbjQX3BZ786Rjw4qPnBmDvkS7Nfxn6b0apEgsfMEKlj9+FQe5rSZiNHKPFNuDG2W3dOL+7Fy6bF2nZKtRcl4fqa3JCThKOV/VcWZZx8V0bGt+1jd44EYBCI2DB/UZkVoY/7DzY6sahnw4EPkAAZlynR9X6yRXto6lht1jxyLJPw2QywWAIPuwZ049CLpcLhw4dwqZNm0ZuE0UR69atw549e/zeZ8+ePdi4ceOo29avX4833njD7/FOpxNO56WldWZz5MsDKfEIghD1VRrk38B5X09LQDLQe8oFa6cnYO+JNlOBZY9lo/uoEx2HHHBZJGgzRRQu0yF3nhqiIr4BVMguxRYgpQKMPlOFRXcUY9EdxRHdL55l/wVBQOWNachfqEHbPgcsbR6IKl84LrxCC2WEPR8tH9jHhaBRZKD1QzvKr9f7lrJTyohpeOnp6YHX60VBwegZ+AUFBTh9+rTf+3R0dPg9vqOjw+/xmzdvxpNPPhmdBhNNQz3HnSFXekEAek66gg79KFQCCpdqUbg0MeclpWKAiVSi7FeUVqDEzOF9tiah/4wrcHAZ4nXIsHZ4YCiLfCEBJa6kH+DbtGkTTCbTyFdzc3O8m0SUVLxOOXTdEMF3XCpItTkw4UqU4BJNISeKDx83idXXbruE9oMONL1nQ8dhBzwp8u8g2cW05yU3NxcKhQKdnZ2jbu/s7ERhYaHf+xQWFkZ0vEajgUbDuS1EE6XLVQSv2ArfRUKXm/zDePHeyDFeUjG4AEBGmQp9IXpfBAWgL4j8vStLvvk5ze/ZfFWBh+ZwiepBVKxLQ9k1rAcTTzHteVGr1Vi6dCm2bds2cpskSdi2bRtWrVrl9z6rVq0adTwAbN26NeDxRDQ5hWFUM1WogfyFqfEhYSq2EZBlGc0HB7D1/9Xj9w8cxu8fOILtPzyLtgmU7J+sVA0uAFCyWhd82EgECpZoJrSKqOEtKxq3DQUXYCTgSy7g/FtWNO20R/yYFD0xHzbauHEjnnvuObz44os4deoUHn74YVitVtx///0AgHvuuWfUhN6vfe1rePvtt/GjH/0Ip0+fxr/927/h4MGDePTRR2PdVKJpSWNQoOLGIEtYAdTcmh6yyJzbJqH5fRtOvWpG/R8G0XXUOa7ORqKIZYCRZRn7ftOE937SgK56355OXpeE9mNmbP/Ps6h7rTXqzxlIogYXySujr96FjsMO9J11Tfh9kl2rQtHKAOFb9O1qXvWxyOfWOPq9aNkdPJxc3GqF2xbmuBVFXcwLL9x1113o7u7Gt771LXR0dGDx4sV4++23RyblNjU1QRQvZajVq1fj5Zdfxje/+U184xvfwMyZM/HGG29g/vz5sW4q0bRVfr0eSp2Ii1ut8Fy294vaIKL6Y2koWBK8d6bjkAP1fxr0zS0Y2p+w/YADaoOIhV80Ir0o8XYiidUQUsN7vTi3owfA6DkZw/9//C8dyKlKQ9nSzKg9pz+JGlxa99px8e/WUXsMqTMEVG1Ij3iytyAIqP1kOtKLlWh+zwZHn+9FFtVA0TIdKtbpoZrArukdh0Lv1yVLQFed09f7Q1Mu5nVephrrvCQnj1NC08F+WLtdUKcpULY0E/rsqa3IGolUqvMyTJZlmC660X3MCcnjKz6XN18DQQze49J72oljvwkwHCIASq2A5RuzoDEk5pwZua8FQHRWIcmyjP994iTMbYEvfoLo28vrpm/OmvTzBZKowaXpPRvOvxV4B/KZt6ejZNXEwoAsy3D0SZA8MrRZikltR3H6D4PoPOQIvteaAii9SofqCfTskH8JU+eFKBxntnfj8O9b4HFIEBS+TzQHtjSj+pocrLh3BhSqpF8Ul/DMzW7UvzYIa+elZRnt+x0oWOLCzNszgu43c+EdW+CNGWXA45DRtteBypsSs1BYNHtgnGZP0OACDH1ir7dA8soxqX+TqMHFZZFw4e3AwQUAGt60oGCxJuJ6L4CvF0aXE52ArNKF/rvIEibUTooOvvIUV2d39mD/b5rgcfg+4she+C6CMtCwqxe7//sCUqxzMOEMtrpR98wArF1j1pPKQOcRJ47+eiDgnAR7r9dX4C7Yn0gOrxt+srwuGS6LNKH5E9GaAxPJc8fifZ2owcXW48WFv1tDLm2WPEDXR87gB02BvEXh7deVKpPYkxF7XihuvG4JR37fEvgAGWg+OICeBivyatg1GysNb1ohDYfGsWTA3OhB91Gn33kv4U5YjOXExoELLjTtsKGv3g3AV2a+aLkWM9bqoc4I//NZNHpgtJkqaA1KOMxBKhYDMBRpoFBG97NjIgYXU6MbDX+1wNwY/PW4nLUz/GNjxVCqQtZMFfrPuf3/uxCA/EWaqPX0UOTY80Jx01pngssWvHqUIPomQCaSVCpyZu/zYuB8gBP0MME3ydIfMcx5BbGa79Jx2IG6X5rQd9Y9cpvXKaPlQzsO/rQfjv7IqpNNtgdGFAXUrssbtzP6WLNuyp/Q4weSiMFl4LwLdb8cgLkpsjBianSHPmgKzPuCAcahfZaGN8Ac/m/ObDVmfTojTi0jgOGF4sjW5wp5kpclwNrnmpoGhSERLxKT4egL4+IuA44e/8ddfCf4HIZhRSuiv2WAY8CL+tcGfcFrbMeOBLgtEk6/Nhjx4042wMz9WCFyqtLgt36ZABQtMGDm2ujtkJ6I70lZknH6fwZ9Qy8Rjo5Z270he+pkSYa9zwtbj2fcrtHRotSJWPxlIxZ92YjCpVrkzFWjaIUWVzySifn3GrhXUpxx2IjiRp2mDHliE0RAk54Yb9NEvEhMliLIRNxRx2nHf86x93rRczx0sFSlCRMKL7Iko/e0C+YmNwABxkoVsmeqRlY/te93INi0EVkCBhrcsHV7oM+L7D00mSEkpUbEjZtqceyNdpzZ3g33UO+ixqDErBvzMe/jBVHbKT1R35P9DW44+ic2VChLQP85t9/5JLIko3WPAy3v20YeX6EVUHyl1rfcXxPdz+OCICCrWo2s6sRd+ThdJcZVgaal0iVGiCoBkjvwFUiWgIpV2VPYKv8S9SIxWRklSqgNIlzmYGtCfeP7Y/WedgZeZXQZY6Uq4gqn5mY3Tmwxw2mSRrrqm3b4io7Nu8eIjGIlTBdCDHcNMV2MPLwAkw8wS+4qwcJPFWGwywkBQEaBNmqhBUjs96S1wxPWeyMQf70pw705nUdGT+j1OmQ0v2dHX70LSx7KhNJP0KbUw78yxY06TYm5GwoC/lwQgaxyHYoXBl/vH2uJfJGYLEEUUH5DkOq6AqBQ+z7ZjuUNczQv0u1fbN0e1D07AOdQoJKlSwXeHAMS6n45AHufN+SQYzRMdghJoRKRWaKDsUQ3bYILAN/vOonRnDQ/exF1H3eNCy4jZMDa4cXFd20Tf1JKKgwvFFcL7yj2TXDE0GQ4wVf8CQCyyvW44Z9nQgxRJC2WEv0iEQ3FK7WYcZ0vwAhjzghKjYCFDxj9TrjV54fe0BEAVOmRnWYatw/tJxNg9ZPkktG8y+abTBnGW8NYMbkO5qnYCykSyfCezK6d+DBLerECGSWqcbe3fmAL/veWfUOJ3iA9uZQ6OGxEcSWKAlbcOwOzbspHw3s9sPa4oNIrUL4yC4VzM+K6a2syXCSiQRAEVN2choLFGrTtc8DS7oGoAnLmaFC4JHDBsPTi8FYQyRHUPvG6ZV+djyCjWLIEdBx0YMU/ZaFpuy3wvBcRyKxSTWjIaKxE24060d+TuhwFcuep0XPSFXEPTOXN/osZDraEqCcE30ozR68XaYW8tKU6/oUpIRiLtLjis4lzQp4uweVyaYVKzLwt/Ho6ltbwliEPtoS/VNbjkH2FCkOQ3L7VILPvysCpVwZ9n8gvDzwCoMkQMfvO6C1nTYQAk0zbUsy6MwOO50y+IoYRzH8JONE3zA48gaVXpgUOGxGNMR2Dy4SEeTGKpJCsUiuMG7ryR1QCCpWAgsVaLHkoEzmz1SNDCgqtgLJrdFj61SxoM6N7JYvnEFIyBRcAUOlELHk4E7PuzIChTBlWqBBE3wRcf7Kq1SGvWOoMAdpsppfpgD0vRJdhcAlfRmkYpw8RyKwcP38hEIVKQO58NXqOuwKWZxdEIH/xpQ0jjRUqLKgwQvLI8LplKDVCyM0kJyMePTDJFlyGKVQCipZpUbRMi3P/a0Hrh/agZfdlybeizJ/Sq3XoPRV8lnjpVfqY7BdFiYc9L0RDGFwio81SIGdOiE/DElB8ZWS7BJdfl+brRQlQ5E1QADPWjl8hJSoFqHRiTIPLSDOm8D2SrMFlrMLl2pD7BSm0AnLm+t8vKKtajcr1Q3/3y99zQ3/u3HlqlF49sR2pKfmw54UIDC4TVfupdBx+emhZ85g5J5CB6o+nIa0gstNMerESC79oxInfmuGxyyPDSLIEqPQC5t9jjMok3MkSskuxJd6NSCLphUoUrdCifX/gTTprbkkLWrm2/Po0ZJSp0PK+Df3n3JAl3/uldI0OBUs0UxJcKTHE/wxAFGcMLhOnMSiw9CtZaNphQ/sBB7xO33wFY7kKM9bqkDNnYrvuZtWoser/y0H3USdMjW4Igm94KG+BJqr1UiaL75nI1H4yHUq9gJb37b6J2UMhV6kXUH1LOoqWha7EnD1TjeyZvqXYsizHdUUixQ/DC01rDC6Tp04XUXNrOqo2pMFtlSCqBKj0kx+RVqgEFC7VonBp9PdFCoc8NNM41S6OHqcMr1OCUidO+f48giigekM6ZqzVo/eUCx67BI1RgZzZ6gmF0lT721D4GF5o2mNwiQ5RKUBjTO6VHrIso/uYCy27bRhs9i3xNlaoUHqVDjlz1Ul9sTQ3u9G43eab9Cr75g4VLNFgxnVp0OdO7d9NpRNReEV8QimlBoYXmrZSZSIkRYcsy6j/4yA6Dozes2ngohsD590ouUqLmo+nJ2WA6TnpxIktZt+vNPR7yV6g87AT3cd8ewKlF/NyQMmDq41oWmJwobE6Djp8wQUYXcNmaCJy624Huo8G2FsngXnsEk6+bPat9Bmz2keWAK9LxonfmUeGyYiSAcMLTTsMLjSWLMto3mUPfpAANL8f4pgE1HHECckd5AAZsPd4MXA+2EFEiYX9hDStMLjEl+SR0X3Mic4jDrhtMrRZIoqW6ZA1UxXXZa4emwxbV4h9CWRgsNkDySMn1IqnUMxN7tDl+UXA3Oj2VbEdYu/zoueEE16HDG2OAnnzNVCok+f3ptTG8ELTRqLsCjxdOQa8+Oi5Adh7pJGL6WAr0H3UhayZKsy/xxi3i2MkIyahCq0lmrBeURnA0Fwer8s396erzjf3RxB8v/PZNyyo/ngailewEBzFH8MLTQtcEj11zE1utOy2o++MC7IXSC9RomSVFhe2WmHvG7ryD4eFoW/7z7lR/6dBzP2sIS5tVukFqA0iXObgyUSXq0i63gdjpQqdR0LM1ZF92zjIsozjW0zoP+seuX042HmdMs780QJRIcRt+TrRMM55oZTH4DJ1Wj+04/DTA+g+5oTHLsPrkmG66MbJlwdh75bGTRgdIQNddU44BsLbqTraBFFA6RpdyG6K0jXJ1+uQv1gLhVYI/LuJQFqhAoZyJQYa3Og/4w46xNTwVwskLyf3UnwxvFBKY3CZOqZGN87+2QJgzNBKBNe5npPBN96LpZI1OhgrVAH3VMqepULRyuTrcVBqBMz/BwNEBcaf8UVApRMw724DBEFA+0FHyKuC2yqj/0z8/k5EAIeNKIUxuEyt1g/sEMSJzwkRBIxsLxAL9j4v2vbY0X3cCcklQ1+oRMkqHXLnqiGIAhQqAQsfMKJ5pw2te+xwW31tUWcIKFmjR9k1uqTdsTirRo2lX8lC8y4bOj9yQvYACo2AohValF2tGyku6BwI0jt2GacpySb+UMpheKGUxuAydXrrXZOazCpLgC4nNpVee087cfyloVonQ/nI1eDGwDk3cuaoMe9uA0SlL8AUrdDC65LQccgBtx2ACMiSDK9DhpgWv/DidckwN7kheWSkFSihzYrstUorVGL2ZwyY9WkZkgcQVePL66vShNArkwAox2z/4HX5hgiVOiFpAx4lF4YXIooKWZpcr4lSJyB3rjr0gRFy9Ht91WXHTqcZam7vaRcubLWiekM6Blvd+OhZEzxO+VLIMcm4uNWG9n0OLH4oE7rsqS2lL3lkXHjHirY9dngvG63JnqXCzNsyIg58gihAEeBlLlisRc/x4ENCCjWQM9v3AKZGN5p22NB72rflgKgCCpdpMWOtHtrM5N4qghIb57wQUVSkFyvDXJfrnzZbxKlXBtG23w6vK3rDR237HJCCzQOWgbYP7XDbJBx7wTwquFx+jHNQwoktprAq0XpdMtr323F8iwlHnx9Aw98ssPdGPhlZlmQc32JG867RwQUA+s66cejn/RN63EBy5qqRVqiAEOTKMOM6PRRqAV1HnTjyiwH01rtGXi/J7Xu9D/20H7Yez4TaIMu+Sd4tu21o+dAOa8fEHodSG3teiCgqSlfrcPLiYOADRCCjRAlbl9fv3BZLqxeWNi+6jzlx/m9WLLjPCGO5atLt6jnpDDkM4nUBzbtswZdKS4ClzQtzo8c3sTeAwRY3jj5vGpkzA/iCRvNOOyrX61F+fVrYbe8+5kTf6QA9IRLgccho+KsF8+8xhv2YwYgKAQu/lIljvzHB0urxzWEa/jVkoOxaHWZcp4fLKuHUq2bf6zr2tZUAt13Gqd8PYulXsiJ6fkubBydfMcPW6b0UhGXAWKXC3M9mJP3GnxQ9DC9EFBV5CzTIW+Db6G8cEdBliVh4vxGiWkD/WRf6zrjQtscx+rihC6HHLuPor01YvjFr0sMPkie8XpzBFk/oCcci0H/OFTC8uAYlfPQrEzyOMc859JgX/m6D2qBA0bLwVi217nEEn4Mi+VZouQYlqDP8d5cMXHChZbcd/WfdkCUZGSUqlKzRIW++elRVY49DwmCrr5dj4RcNsHb4gqTHIUOXrUDhcu3IkFnHAfv4Ybgx7Rps8aBxhxVZM9XIKFGG3NDS1u3BkWcGLvW6XfY7my+6ceQXA1j61Syo9BwwIIYXIooSQRQw53MGpJfY0bLbBrfFd/URlUDBUi2q1qdBlea78OTMUePC29bAF2bZN/TSuseO6g3pk2pXepESzv7Qk4kVmtBjXgKCh5u2fXZfcAmSlxrftaLwCk1Y2yFYOz2hl5rLgK3H6ze8NL1nw/m3rL4JAkPtNjW6YbroRv5iDebclQGvG7jwtgXt+x2QhkZoBAWQv1iDmlvSR/5mlzM1Bq8FM+zC2zZceNsGfYECM29LH7X9wFgXt9oguf2/drIEOAYktO21R9RzRamL4YWIokZUCCi/zres2NblhSzJ0OUqoNSMvgDae72wdobeS6jzsHPS4aVklS74JFQByKpRIbNaHXKyqiz5KgYH0lkXeojK0S/B0u5BRknoITGFSoAnjJSg8PNQA+ddvuACjF7+PPRwXXVOpBcr0X3MicGW0SFJ9gKdR5wwN3lwxf/NnHRvh63Li6O/MmHhl4x+A4zHLqH7mDN4wJSBtr0OhhcCwAm7RBQDokJAepESGSWqccEF8A0LhcPjmHw9kcxqFQqXafz/UPQVcZt5WzoKl2ggBvs4JwDqDHFkpY0/Hnt47R03rBRA7nxNyLO0OkNAWtH4hrfstgedeAsAjdttGGwO0Lsj+XabbtppG/ejzMoAxfwCGdpm4MyfBv1OeHZZpLCW2TvNUlgTpin1MbwQ0ZTTGMM79ajTJ3+KEgQBs+7IQNWGNF8dk5EfADm1alzxaCb0eUoodSJmfybDd1Eee2EWAEEE5nwuI2gdE122IqyLerjzeEpWaRFiqgjKrtH7bVPfmdBDZd5QIUoeXq01+rjCZVpfxd5IyIC9R4Lp4vjVQwpteH9nhVoIOXeGpgcOGxHRlNMYFMiqVfn20QnCZZZg6/FCnzu5SbuCKGDGWj1Kr9ZhsMUDySNDn6sYt3olf5EWqjQRF7ZaYb7sIptdq0LFTWkwlAYf6ilaqYO5KciKKwEwVqjCrs2iz1Ni3t0GnPjt6AJ7wxOLi5ZrUHqV//2WorX7tdchwzUojQpcKr2IuZ/3tSvS57L3eHw9N5fRZIgwVChhbgw8x0cQgYIlAXrQaNpheCGiuKjakI7D5/qDXvgkyTfBdU6UdpsWFULI5ddZNWpk1ajhGPDCY5OhzhADruQZq2CxBm177L5VO2MvwkO9N9Ufi2zORu5cDVb8Uzba9tnRe9IFySMjvdi3tUFmtSpgT0R6sXLcXJaJElXjnyN3ngZLHslE804buo+7wn6eQLtyV9yQhqO/Nvm/09BrFyio0fQT02Gjvr4+fOELX4DBYEBmZiYeeOABWCyWoPdZu3YtBEEY9fXQQw/FsplEFAcZxUoodSGGACSg66gTHufU76WjzVQgvVgZdnABAFEpYOGXjMidN35ejDpDwNy7M2CYEVntGo9DguyVUXqVHisez8aV/5KD+f9gRFaNOugQSslqXfBAIQCaLDH4VUDwTVBW+1lxBACGUhXm3W3ENd/PRfWtoUOZoACyZvqfM5Rdq8asOzN883SGf62h/yrUAhbcb4Q+j5+3ySem74QvfOELaG9vx9atW+F2u3H//ffjy1/+Ml5++eWg93vwwQfxne98Z+R7vV4fy2YSURzIsjyqkFvA47yAa1CGMgFGDLwuGb2nXXBbJagNInJmqSEqx+wPpBMx/x+MsPd5faXzTznhGpThMss48dIgcuY4UbU+DWmFwU+/th4PLm61ofvopVU4hnIlKm5IQ/as0NsoFCzWoOeE0/8KKgHQ5ylQdXMajr9kDvwgMjDj2tC9HaJCQNFyHZp22Hx/U39/VgEoXqkNunKpaJkW2bPU6Dhgh7nZV3cnq1qNgis0UIY5L4amh5iFl1OnTuHtt9/GgQMHsGzZMgDAz372M3zsYx/DD3/4QxQXFwe8r16vR2FhYayaRkQJQBB8e+yMLXvvj1Ib30masiyjaacdTduto9qr1AmoujkNxVeOv8D3nnKifb9j9ARe2beXUv85F5Y8lBlwubSl/bKCbZd1OpmbPDj6vAmz7khH0YrgoUIQBcz9vAHNu+xo/cAG1+BQ3R01ULRch4ob9VDpRNR8Ig3n/mIdVaBv+P/Lb9Ajf1F4BfWUGgGLvpSJj54bGB1Kh2rM5MxRo/qW0MveNRkil0NTSDELL3v27EFmZuZIcAGAdevWQRRF7Nu3D5/85CcD3vd3v/sdfvvb36KwsBC33nor/vVf/zVg74vT6YTT6Rz53mwO8imCiBJK/mIN2g86R9chudzQBNdorDqajAvv2NC0ffySYY9dxpnXLZC8MkrXXDpHOfq9OPe/QzVW/JTPl9zAqVcGsXxj1rihH1mWcepV87jgcvljnXndguzZamgMwSf+Xl53x97jHdm5+/J5J6Vr9DBWqND6oR3953zF54wVKpSs1gXdBsGf9CIlVv5zNjqOONH1kQNeh6/OT/FKHbJmBp6fQxSpmIWXjo4O5Ofnj34ypRLZ2dno6OgIeL/Pf/7zKC8vR3FxMY4ePYp/+Zd/QX19Pf70pz/5PX7z5s148skno9p2IpoapVfp0XHICTlIpd3y6+M7bOwY8A3/BHP+LSsKl2pHhjba9jmCHg/ZV7jN3z5Jg80eWNuDF/CTZaB9nwMVN4bXQyEqBKQVBD7dZ5SoMPvOye8jBQBKnYjS1TqUrubkWoqdiD/OPPHEE+Mm1I79On369IQb9OUvfxnr16/HggUL8IUvfAEvvfQSXn/9dTQ0NPg9ftOmTTCZTCNfzc3NE35uIppaaQVKLLjPCHHsdXNodcmsOzOQXRt6fkesyLKMlt32kMdJHqDro0s9wIOt4ZXPt7SNr3libgljF2UZMDcHX2ZOlMoi7nl5/PHHcd999wU9pqqqCoWFhejq6hp1u8fjQV9fX0TzWVauXAkAOHfuHKqrq8f9XKPRQKNJgJl8RDQh2bVqrP5GDjoOOzDQ4IYsARllShSt0EETwUqfaOusc6Bphw3WjhDbGMC3isbRd+m4YIXsxt5v3G1hjqyEszcSUaqKOLzk5eUhLy8v5HGrVq3CwMAADh06hKVLlwIAtm/fDkmSRgJJOOrq6gAARUVFkTaViJKEUieidI0epWvi3RKfi+9acXGrLewS+LI0ukpsdq0avadCz0TOqhnfq5RZHd7wTVZNdIZ5iJJRzD7WzJkzBzfffDMefPBB7N+/Hx988AEeffRRfPaznx1ZadTa2orZs2dj//79AICGhgZ897vfxaFDh3Dx4kX85S9/wT333INrrrkGCxcujFVTiYhGDLa6fcEFCL/AmwzkLbgURAqu0Ph2qQ4UfkQge7bab6XdtHwlMmtUgc/Ogm/FUPZsNbqPOdFZ5/DtPk00jcS0zsvvfvc7PProo7jhhhsgiiLuuOMO/PSnPx35udvtRn19PWw234lCrVbj3XffxY9//GNYrVaUlZXhjjvuwDe/+c1YNpOIaETbXvuoZcMhCUDuPDX0uZdOp0qtiAX3GXD0eRMkL8atGtLnKnz7KAUw564MHPnFABz90qgANVzAzViuwoEfja5ObKhQYtYdGUjLZyE3Sn2CnGJbdJrNZhiNRvQ8+1UY9JwLM51tqaqFkF0a72ZQkjnwX32wdoae54KhFVKZNSrMv8cIpWZ8N4u9z4vWD+3oPOKA1ylDm6VA8ZVaFC7T+T3+cm6bhLa9drTtdcBpkqDQCMhfqIG5xe2bh+Nn+wGlRsAVX8kcFaSIkoXdYsUjyz4Nk8kEgyH4liB8hxMRXS7MwfTMShXKb9AH3V9Il61AzcfTUfPx0MXZxlLpfcXayq9PgyzLEAQBbfvtaD8QYBm2DHhcMs7/zYr5/2CM+PmIkgnrLRMRXSZ7ljrkRF1BAcy92xByf6FoGX6Otj324G2TgJ4TLrgsU78XFNFUYnghIrpM8Upd8OXKAlCwRBNws8JYsvdIoScRy74Kv0SpjOGFiOgyumwF5nw2Y6RQ3ljpxUrUfCLyYaBoEMOs13d5+X+iVMQ5L0REY+Qv0kKXq0DL+3Z0n3BCcvv2BCpZpUXRSh0UqviEg/yFGrTudQTeCwqANluEPj/4nkdEyY7hhYjIj4wSFeZ8VoU58W7IZUpW69C2zxF05Kj8+jRugEgpj8NGRERJQp+nxPx7DBCVGDVxd3h4q/x6PQqXsUQEpT72vBARJZGc2RqsfCIH7fvt6DvtguQFMkqVKL5Sh4xintJpeuA7nYgoyWgyRFTckIaKG9Li3RSiuOCwERERESUVhhciIiJKKgwvRERElFQYXoiIiCipMLwQERFRUmF4ISIioqTC8EJERERJheGFiIiIkgrDCxERESUVhhciIiJKKgwvRERElFQYXoiIiCipMLwQERFRUmF4ISIioqTC8EJERERJheGFiIiIkgrDCxERESUVhhciIiJKKgwvlJK2VNXGuwlERBQjDC+UcoaDi5BdGueWEBFRLDC8UEphcCEiSn0ML5QyGFyIiKYHhhdKCQwuRETTB8MLJT0GFyKi6YXhhZIagwsR0fTD8EJJj8GFiGh6YXihpLWlqpbBhYhoGmJ4oaTE4EJENH0xvFDSYXAhIpreYhZevv/972P16tXQ6/XIzMwM6z6yLONb3/oWioqKoNPpsG7dOpw9ezZWTaQkxOBCREQxCy8ulwt33nknHn744bDv85//+Z/46U9/imeeeQb79u1DWloa1q9fD4fDEatmUhLhfkVERAQAylg98JNPPgkAeOGFF8I6XpZl/PjHP8Y3v/lN3HbbbQCAl156CQUFBXjjjTfw2c9+NlZNpSTAJdFERDQsYea8XLhwAR0dHVi3bt3IbUajEStXrsSePXsC3s/pdMJsNo/6otTC4EJERJdLmPDS0dEBACgoKBh1e0FBwcjP/Nm8eTOMRuPIV1lZWUzbSVOLwYWIiMaKKLw88cQTEAQh6Nfp06dj1Va/Nm3aBJPJNPLV3Nw8pc9PscPgQkRE/kQ05+Xxxx/HfffdF/SYqqqqCTWksLAQANDZ2YmioqKR2zs7O7F48eKA99NoNNBoNBN6TkpcDC5ERBRIROElLy8PeXl5MWlIZWUlCgsLsW3btpGwYjabsW/fvohWLFHyY3AhIqJgYjbnpampCXV1dWhqaoLX60VdXR3q6upgsVhGjpk9ezZef/11AIAgCHjsscfwve99D3/5y19w7Ngx3HPPPSguLsbtt98eq2ZSgmFwISKiUGK2VPpb3/oWXnzxxZHvlyxZAgDYsWMH1q5dCwCor6+HyWQaOebrX/86rFYrvvzlL2NgYABXXXUV3n77bWi12lg1kxIIgwsREYVDkGVZjncjoslsNsNoNKLn2a/CoOdcmGTB4EJENL3ZLVY8suzTMJlMMBgMQY9NmKXSNH0xuBARUSQYXiiuGFyIiChSDC8UNwwuREQ0EQwvFFcMLkREFCmGFyIiIkoqDC9ERESUVBheiIiIKKkwvBAREVFSYXghIiKipMLwQkREREmF4YWIiIiSCsMLERERJRWGFyIiIkoqDC9ERESUVBheiIiIKKkwvBAREVFSYXghIiKipMLwQkREREmF4YWIiIiSCsMLERERJRWGFyIiIkoqDC9ERESUVBheiIiIKKkwvBAREVFSYXghIiKipMLwQkREREmF4YWIiIiSCsMLERERJRWGF4qLLVW18W4CERElKYYXmnLDwUXILo1zS4iIKBkxvNCUYnAhIqLJYnihKcPgQkRE0cDwQlOCwYWIiKKF4YVijsGFiIiiieGFYorBhYiIoo3hhWKGwYWIiGKB4YVigsGFiIhiheGFoo7BhYiIYilm4eX73/8+Vq9eDb1ej8zMzLDuc99990EQhFFfN998c6yaSDHA4EJERLGmjNUDu1wu3HnnnVi1ahV+/etfh32/m2++Gb/5zW9GvtdoNLFoHsUAgwsREU2FmIWXJ598EgDwwgsvRHQ/jUaDwsLCGLSIYonBhYiIpkrCzXnZuXMn8vPzMWvWLDz88MPo7e2Nd5MoBAYXIiKaSjHreZmIm2++GZ/61KdQWVmJhoYGfOMb38CGDRuwZ88eKBQKv/dxOp1wOp0j35vN5qlqLoHBhYiIpl5EPS9PPPHEuAm1Y79Onz494cZ89rOfxSc+8QksWLAAt99+O958800cOHAAO3fuDHifzZs3w2g0jnyVlZVN+PlpYhhciIhoKkXU8/L444/jvvvuC3pMVVXVZNoz7rFyc3Nx7tw53HDDDX6P2bRpEzZu3DjyvdlsZoCZIluqahlciIhoykUUXvLy8pCXlxertozT0tKC3t5eFBUVBTxGo9FwRVIcMLgQEVG8xGzCblNTE+rq6tDU1ASv14u6ujrU1dXBYrGMHDN79my8/vrrAACLxYJ//ud/xt69e3Hx4kVs27YNt912G2pqarB+/fpYNZMmgMGFiIjiKWYTdr/1rW/hxRdfHPl+yZIlAIAdO3Zg7dq1AID6+nqYTCYAgEKhwNGjR/Hiiy9iYGAAxcXFuOmmm/Dd736XPSsJhMGFiIjiTZBlWY53I6LJbDbDaDSi59mvwqBn6IkmriwiIqJYsVuseGTZp2EymWAwGIIem3B1XigxMbgQEVGiYHihkBhciIgokSRUkbpoGB4FG7Q7QxxJ4Xilsgaw2iFkFQMWa7ybQ0REKcpusQG4dB0PJuXmvLS0tLDOCxERUZJqbm5GaWnwnv6UCy+SJKGtrQ0ZGRkQBGFCjzFc6K65uTnkpCGKHr7u8cPXPj74uscPX/v4CPa6y7KMwcFBFBcXQxSDz2pJuWEjURRDJrZwGQwGvqnjgK97/PC1jw++7vHD1z4+Ar3uRqMxrPtzwi4RERElFYYXIiIiSioML35oNBp8+9vfZmXfKcbXPX742scHX/f44WsfH9F63VNuwi4RERGlNva8EBERUVJheCEiIqKkwvBCRERESYXhhYiIiJIKw0sQFy9exAMPPIDKykrodDpUV1fj29/+NlwuV7ybNi18//vfx+rVq6HX65GZmRnv5qSsp59+GhUVFdBqtVi5ciX2798f7yalvF27duHWW29FcXExBEHAG2+8Ee8mTQubN2/G8uXLkZGRgfz8fNx+++2or6+Pd7OmhV/84hdYuHDhSHG6VatW4W9/+9uEH4/hJYjTp09DkiT88pe/xIkTJ/DUU0/hmWeewTe+8Y14N21acLlcuPPOO/Hwww/Huykp69VXX8XGjRvx7W9/G4cPH8aiRYuwfv16dHV1xbtpKc1qtWLRokV4+umn492UaeW9997DI488gr1792Lr1q1wu9246aabYLVy09lYKy0txb//+7/j0KFDOHjwIK6//nrcdtttOHHixIQej0ulI/SDH/wAv/jFL3D+/Pl4N2XaeOGFF/DYY49hYGAg3k1JOStXrsTy5cvx85//HIBvb7CysjJ85StfwRNPPBHn1k0PgiDg9ddfx+233x7vpkw73d3dyM/Px3vvvYdrrrkm3s2ZdrKzs/GDH/wADzzwQMT3Zc9LhEwmE7Kzs+PdDKJJc7lcOHToENatWzdymyiKWLduHfbs2RPHlhFNDZPJBAA8p08xr9eLV155BVarFatWrZrQY6TcxoyxdO7cOfzsZz/DD3/4w3g3hWjSenp64PV6UVBQMOr2goICnD59Ok6tIpoakiThsccew5o1azB//vx4N2daOHbsGFatWgWHw4H09HS8/vrrmDt37oQea1r2vDzxxBMQBCHo19iTd2trK26++WbceeedePDBB+PU8uQ3kdeeiCjaHnnkERw/fhyvvPJKvJsybcyaNQt1dXXYt28fHn74Ydx77704efLkhB5rWva8PP7447jvvvuCHlNVVTXy/21tbbjuuuuwevVqPPvsszFuXWqL9LWn2MnNzYVCoUBnZ+eo2zs7O1FYWBinVhHF3qOPPoo333wTu3btQmlpabybM22o1WrU1NQAAJYuXYoDBw7gJz/5CX75y19G/FjTMrzk5eUhLy8vrGNbW1tx3XXXYenSpfjNb34DUZyWnVVRE8lrT7GlVquxdOlSbNu2bWSyqCRJ2LZtGx599NH4No4oBmRZxle+8hW8/vrr2LlzJyorK+PdpGlNkiQ4nc4J3Xdahpdwtba2Yu3atSgvL8cPf/hDdHd3j/yMn0xjr6mpCX19fWhqaoLX60VdXR0AoKamBunp6fFtXIrYuHEj7r33XixbtgwrVqzAj3/8Y1itVtx///3xblpKs1gsOHfu3Mj3Fy5cQF1dHbKzszFjxow4tiy1PfLII3j55Zfx5z//GRkZGejo6AAAGI1G6HS6OLcutW3atAkbNmzAjBkzMDg4iJdffhk7d+7E3//+94k9oEwB/eY3v5EB+P2i2Lv33nv9vvY7duyId9NSys9+9jN5xowZslqtllesWCHv3bs33k1KeTt27PD73r733nvj3bSUFuh8/pvf/CbeTUt5X/ziF+Xy8nJZrVbLeXl58g033CC/8847E3481nkhIiKipMIJHERERJRUGF6IiIgoqTC8EBERUVJheCEiIqKkwvBCRERESYXhhYiIiJIKwwsRERElFYYXIiIiSioML0RERJRUGF6IiIgoqTC8EBERUVJheCEiIqKk8v8DywlP9AAxR0sAAAAASUVORK5CYII=\n" 1262 | }, 1263 | "metadata": {} 1264 | } 1265 | ] 1266 | } 1267 | ] 1268 | } -------------------------------------------------------------------------------- /readme.md: -------------------------------------------------------------------------------- 1 | ### Курс "Основы глубокого обучения" в ИТМО (осень 2024) 2 | - [1.1 Вводное занятие, знакомство с PyTorch (06.12.2024)](Lecture%201) 3 | - [1.2 Полносвязные сети, метод обратного распространения (09.12.2024)](Lecture%202) 4 | - [1.3 Autograd, Micrograd, SGD (13.12.2024)](Lecture%203) 5 | - [1.4 Оптимизаторы, функции активации, инициализация весов (16.12.2024)](Lecture%204-5) 6 | - [1.5 Dropout & BatchNorm (23.12.2024)](Lecture%206) 7 | 8 | 9 | - [2.1 Функции потерь, сверточные сети (28.12.2024)](Lecture%207) 10 | - [2.2 PyTorch Lightning, Learning Rate Schedulers (30.12.2024)](Lecture%208) 11 | 12 | 13 | - [3.1 Dropout2d, BatchNorm2d, NLP basics (13.01.2025, 17.01.2025)](Lecture%209) 14 | 15 | - [4.1 Huggingface Transformers (20.01.2025)](Lecture%2010) 16 | - [4.2 Аугментация, интерпретация, дистилляция (27.01.2025)](Lecture%2011-12) 17 | ______ 18 | 19 | #### Домашние задания 20 | 21 | | # | Тема | Max Балл | Мягкий Дедлайн | Жесткий Дедлайн | 22 | |------|----------|-----------|----------------|-----------------| 23 | | [ДЗ 1](HW/hw_1/itmo_dl_course_hw_1.md) | Создание и обучение MLP | 10 | 11.01.2025 20:00 МСК| 14.01.2025 20:00 МСК| 24 | | [ДЗ 2](HW/hw_2/itmo_dl_course_hw_2.md) | Создание и обучение AlexNet | 10 | 21.01.2025 20:00 МСК| 27.01.2025 20:00 МСК| 25 | 26 | ______ 27 | 28 | #### Дополнительные задания 29 | 30 | | # | Тема | Max Балл | Жесткий Дедлайн | 31 | |------|----------|-----------|-----------------| 32 | | [Extra Task 1](HW/itmo_dl_course_extra_task_1.md) | Создание и обучение MLP | 5 | 31.12.2024 20:00 МСК| 33 | | [Extra Task 2](HW/itmo_dl_course_extra_task_2.md) | Классификатор FashionMNIST на PyTorch Lightning | 5 | 18.01.2025 20:00 МСК| 34 | | [Extra Task 3](HW/itmo_dl_course_extra_task_3.md) | Классификация текстов с CNN и RNN | 5 | 25.01.2025 20:00 МСК| 35 | | [Extra Task 4](HW/itmo_dl_course_extra_task_4.md) | Классификация текстов с transformers | 5 | 28.01.2025 18:00 МСК| 36 | ______ 37 | 38 | #### Соревнование 39 | 40 | | Ссылка | Max Балл | Жесткий Дедлайн | 41 | |----------|----------|-----------------| 42 | | [AITH DL Competition - Tabular data](https://www.kaggle.com/t/bde680ca0f054b4c85fc5065c9ef6fbf) | 30 | 27.01.2025 19:00 МСК| 43 | | [AITH DL Competition - Text data](https://www.kaggle.com/t/1afde63d76e04a4b91bb3b6bdd5e08e3) | 30 | 27.01.2025 19:00 МСК| 44 | 45 | Необходимо выбрать одно соревнование (можно больше, но баллы в итоге будут выставлены за лучший результат). 46 | 47 | Для решения задач необходимо использовать **только нейросетевые модели**. 48 | 49 | ##### Схема выставления баллов 50 | 51 | - за позицию на private LB - max 15 баллов (по 5 баллов за преодоление каждого бенчмарка) 52 | - оставшиеся 15 баллов - за код решения, посланный на ревью **@pacifikus** через github, как все ДЗ, отдельным PR 53 | 54 | Баллы за позицию на LB выставляются только в случае создания PR с воспроизводимым решением! 55 | 56 | #### Критерии для оценки кода 57 | 58 | - Решение реализовано без концептуальных ошибок в архитектуре / дата-ликов в предобработке данных и т.п. - **5 баллов** 59 | - Принимаемые решения обоснованы (почему выбрана определенная архитектура/гиперпараметр/оптимизатор/преобразование и т.п.) - **3 балла** 60 | - Обеспечена воспроизводимость решения: зафиксированы random_state, ноутбук воспроизводится от начала до конца без ошибок - **5 баллов** 61 | - Ноутбук структурирован, код экспериментов не содержит дублирования, оформление соответствует стандарту pep8 - **2 балла** 62 | --------------------------------------------------------------------------------