├── .gitignore ├── Beginner-Images ├── tf2doc-cnn-cifar10.md ├── tf2doc-cnn-cifar10 │ ├── cifar10-eg.jpg │ └── pooling.jpg ├── tf2doc-tfhub-image-tl.md └── tf2doc-tfhub-image-tl │ └── cifar10-eg.jpg ├── Beginner-ML-basics ├── tf2doc-ml-basic-image.md ├── tf2doc-ml-basic-image │ ├── fashion-mnist-sm.jpg │ └── fashion-mnist.jpg ├── tf2doc-ml-basic-overfit.md ├── tf2doc-ml-basic-overfit │ ├── 3_train_val_loss.jpg │ ├── imdb.jpg │ └── l2_dropout_loss.jpg ├── tf2doc-ml-basic-regression.md ├── tf2doc-ml-basic-regression │ ├── early_mse.jpg │ ├── mse.jpg │ ├── sns.jpg │ └── test_true_pred.jpg ├── tf2doc-ml-basic-save-model.md ├── tf2doc-ml-basic-save-model │ └── hdf5.png ├── tf2doc-ml-basic-structured-data.md ├── tf2doc-ml-basic-structured-data │ └── structured-data.jpg ├── tf2doc-ml-basic-text.md └── tf2doc-ml-basic-text │ ├── imdb-sm.jpg │ └── imdb.jpg ├── Beginner-Text-and-sequences ├── tf2doc-rnn-lstm-text.md └── tf2doc-rnn-lstm-text │ ├── acc1.jpg │ ├── acc2.jpg │ ├── rnn.jpg │ └── rnn_small.jpg ├── README.md ├── code ├── auto_mpg_regression.ipynb ├── cnn-cifar-10.ipynb ├── explore_imdb_overfit.ipynb ├── rnn-text.ipynb ├── save_restore_model.ipynb └── tfhub_image_transfer_learning.ipynb ├── tf2doc.md └── tf2doc └── tf.jpg /.gitignore: -------------------------------------------------------------------------------- 1 | .DS_Store 2 | .vscode 3 | *.mx 4 | # code only include ipynb file 5 | code/* 6 | !code/*.ipynb -------------------------------------------------------------------------------- /Beginner-Images/tf2doc-cnn-cifar10.md: -------------------------------------------------------------------------------- 1 | --- 2 | title: TensorFlow 2 中文文档 - 卷积神经网络分类 CIFAR-10 3 | date: 2019-07-19 20:00:10 4 | description: TensorFlow2文档，TensorFlow2.0文档，TensorFlow2.0 TF2.0 TensorFlow 2 / 2.0 官方文档中文版，卷积神经网络(Convolutional Neural Networks, CNN)分类 CIFAR-10 。 5 | tags: 6 | - TensorFlow 2 7 | - 官方文档 8 | keywords: 9 | - TensorFlow2.0 10 | - TensorFlow2文档 11 | - TensorFlow2.0文档 12 | nav: 简明教程 13 | categories: 14 | - TensorFlow2 文档 15 | image: post/tf2doc-cnn-cifar10/pooling.jpg 16 | github: https://github.com/geektutu/tensorflow2-docs-zh 17 | --- 18 | 19 | **TF2.0 TensorFlow 2 / 2.0 中文文档：卷积神经网络分类 CIFAR-10 Convolutional Neural Networks** 20 | 21 | 主要内容：使用卷积神经网络(Convolutional Neural Network, CNN)分类[CIFAT-10数据集](https://www.cs.toronto.edu/~kriz/cifar.html) 22 | 23 | 官方文档使用的是MNIST数据集，之前在[mnist手写数字识别(CNN卷积神经网络)](https://geektutu.com/post/tensorflow2-mnist-cnn.html)这篇文章中已经有详细的介绍了，包括训练模型、使用真实图片预测等。这篇文章选用 CIFAR-10 数据集来验证简单的卷积神经网络在图像分类问题上的表现。 24 | 25 | ## CIFAR-10 数据集简介 26 | 27 | 与 MNIST 手写数字一样，CIFAR-10 包含了60,000张图片，共10类。训练集50,000张，测试集10,000张。但与MNIST不同的是，CIFAR-10 数据集中的图片是彩色的，每张图片的大小是 _32x32x3_ ，3代表 _R/G/B_ 三个通道，每个像素点的颜色由 _R/G/B_ 三个值决定，_R/G/B_ 的取值范围为0-255。熟悉计算机视觉的童鞋应该了解，图片像素点的值还可以由 _R/G/B/A_ 四个值决定，A 代表透明度，取值范围为0-1。比如下面2个颜色，同样是黑色，透明度不同，感官上会有很大差别： 28 | 29 |

rgba(0, 0, 0, 1)

30 |

rgba(0, 0, 0, 0.5)

31 | 32 | 下载 CIFAR-10 数据集 33 | 34 | ```python 35 | # geektutu.com 36 | import matplotlib.pyplot as plt 37 | import tensorflow as tf 38 | from tensorflow.keras import layers, datasets, models 39 | 40 | (train_x, train_y), (test_x, test_y) = datasets.cifar10.load_data() 41 | ``` 42 | 43 | 看一看前15张图片长啥样吧。 44 | 45 | ```python 46 | # geektutu.com 47 | plt.figure(figsize=(5, 3)) 48 | plt.subplots_adjust(hspace=0.1) 49 | for n in range(15): 50 | plt.subplot(3, 5, n+1) 51 | plt.imshow(train_x[n]) 52 | plt.axis('off') 53 | _ = plt.suptitle("geektutu.com CIFAR-10 Example") 54 | ``` 55 | 56 | ![cifar-10 first 15 images](tf2doc-cnn-cifar10/cifar10-eg.jpg) 57 | 58 | 将0-255的像素值转换到0-1 59 | 60 | ```python 61 | # geektutu.com 62 | train_x, test_x = train_x / 255.0, test_x / 255.0 63 | print('train_x shape:', train_x.shape, 'test_x shape:', test_x.shape) 64 | # (50000, 32, 32, 3), (10000, 32, 32, 3) 65 | ``` 66 | 67 | ## 卷积层 68 | 69 | ```python 70 | # geektutu.com 71 | model = models.Sequential() 72 | model.add(layers.Conv2D(32, (3, 3), activation='relu', input_shape=(32, 32, 3))) 73 | model.add(layers.MaxPooling2D((2, 2))) 74 | model.add(layers.Conv2D(64, (3, 3), activation='relu')) 75 | model.add(layers.MaxPooling2D((2, 2))) 76 | model.add(layers.Conv2D(64, (3, 3), activation='relu')) 77 | model.summary() 78 | ``` 79 | 80 | ```bash 81 | Model: "sequential_1" 82 | _________________________________________________________________ 83 | Layer (type) Output Shape Param # 84 | ================================================================= 85 | conv2d_3 (Conv2D) (None, 30, 30, 32) 896 86 | _________________________________________________________________ 87 | max_pooling2d_2 (MaxPooling2 (None, 15, 15, 32) 0 88 | _________________________________________________________________ 89 | conv2d_4 (Conv2D) (None, 13, 13, 64) 18496 90 | _________________________________________________________________ 91 | max_pooling2d_3 (MaxPooling2 (None, 6, 6, 64) 0 92 | _________________________________________________________________ 93 | conv2d_5 (Conv2D) (None, 4, 4, 64) 36928 94 | ================================================================= 95 | Total params: 56,320 96 | Trainable params: 56,320 97 | Non-trainable params: 0 98 | _________________________________________________________________ 99 | ``` 100 | 101 | CNN 的输入是三维张量 (image_height, image_width, color_channels)，即 input_shape。每一层卷积层使用`tf.keras.layers.Conv2D`来搭建。Conv2D 共接收2个参数，第2个参数是卷积核大小，第1个参数是卷积核的个数。 102 | 103 | 关于 CNN 更详细的内容，可以参考[mnist手写数字识别(CNN卷积神经网络)](https://geektutu.com/post/tensorflow2-mnist-cnn.html)，这里有卷积的动态效果图和推荐的视频。 104 | 105 | 第1、2卷积层后紧跟了最大池化层(MaxPooling2D)，最大池化即选择图像区域的最大值作为该区域池化后的值，另一个常见的池化操作是平均池化，即计算图像区域的平均值作为该区域池化后的值。 106 | 107 | ![Max & Avg Pooling](tf2doc-cnn-cifar10/pooling.jpg) 108 | 109 | 每一轮卷积或池化后，图像的宽和高的值都会减小，假设图像的高度为h，卷积核大小为 m，那么很容易得出卷积后的高度 h1 = h - m + 1。池化前的高度为 h1，池化滤波器大小为 s，那么池化后的高度为 h1 / s。对应到`model.summary()`的输出，输入大小为 (32, 32)，经过32个3x3的卷积核卷积后，大小为 (30, 30)，紧接着池化后，大小变为(15, 15)。 110 | 111 | ## 全连接层 112 | 113 | 我们的目的是对图像进行分类，即期望输出一个长度为10的一维向量，第k个值代表输入图片分类为k的概率。因此需要通过 Dense 层，即全连接层，将3维的卷积层输出，转换为一维。这里可以使用`tf.keras.layers.Flatten()`。 114 | 115 | ```python 116 | # geektutu.com 117 | model.add(layers.Flatten()) 118 | model.add(layers.Dense(64, activation='relu')) 119 | model.add(layers.Dense(10, activation='softmax')) 120 | model.summary() 121 | ``` 122 | 123 | 看一下最终的模型。 124 | 125 | ```bash 126 | Model: "sequential_1" 127 | _________________________________________________________________ 128 | Layer (type) Output Shape Param # 129 | ================================================================= 130 | conv2d_3 (Conv2D) (None, 30, 30, 32) 896 131 | _________________________________________________________________ 132 | max_pooling2d_2 (MaxPooling2 (None, 15, 15, 32) 0 133 | _________________________________________________________________ 134 | conv2d_4 (Conv2D) (None, 13, 13, 64) 18496 135 | _________________________________________________________________ 136 | max_pooling2d_3 (MaxPooling2 (None, 6, 6, 64) 0 137 | _________________________________________________________________ 138 | conv2d_5 (Conv2D) (None, 4, 4, 64) 36928 139 | _________________________________________________________________ 140 | flatten_1 (Flatten) (None, 1024) 0 141 | _________________________________________________________________ 142 | dense_2 (Dense) (None, 64) 65600 143 | _________________________________________________________________ 144 | dense_3 (Dense) (None, 10) 650 145 | ================================================================= 146 | Total params: 122,570 147 | Trainable params: 122,570 148 | Non-trainable params: 0 149 | _________________________________________________________________ 150 | ``` 151 | 152 | ## 编译训练模型 153 | 154 | ```python 155 | # geektutu.com 156 | model.compile(optimizer='adam', 157 | loss='sparse_categorical_crossentropy', 158 | metrics=['accuracy']) 159 | 160 | model.fit(train_x, train_y, epochs=5) 161 | ``` 162 | 163 | ## 评估模型 164 | 165 | ```python 166 | # geektutu.com 167 | test_loss, test_acc = model.evaluate(test_x, test_y) 168 | test_acc # 0.683 169 | ``` 170 | 171 | 卷积神经网络非常适合用来处理图像，这个模型如果用来训练 MNIST 手写数字数据集，可以达到 99% 的正确率，但是在 CIFAR10 数据集上，只有 68.3% 的正确率，我们将在后面的文章中，使用复杂网络模型或者迁移学习来提高准确率。 172 | 173 | 返回[文档首页](https://geektutu.com/post/tf2doc.html) 174 | 175 | > 完整代码：[Github - cnn-cifar-10.ipynb](https://github.com/geektutu/tensorflow2-docs-zh/tree/master/code) 176 | > 参考文档：[Convolutional Neural Networks](https://www.tensorflow.org/beta/tutorials/images/intro_to_cnns) 177 | 178 | ## 附 推荐 179 | 180 | - [一篇文章入门 Python](https://geektutu.com/post/quick-python.html) -------------------------------------------------------------------------------- /Beginner-Images/tf2doc-cnn-cifar10/cifar10-eg.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/geektutu/tensorflow2-docs-zh/3418ce1c0e721f549d0cff7eeff65ca030c55f82/Beginner-Images/tf2doc-cnn-cifar10/cifar10-eg.jpg -------------------------------------------------------------------------------- /Beginner-Images/tf2doc-cnn-cifar10/pooling.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/geektutu/tensorflow2-docs-zh/3418ce1c0e721f549d0cff7eeff65ca030c55f82/Beginner-Images/tf2doc-cnn-cifar10/pooling.jpg -------------------------------------------------------------------------------- /Beginner-Images/tf2doc-tfhub-image-tl.md: -------------------------------------------------------------------------------- 1 | --- 2 | title: TensorFlow 2 中文文档 - TFHub 迁移学习 3 | date: 2019-07-19 22:00:10 4 | description: TensorFlow2文档，TensorFlow2.0文档，TensorFlow2.0 TF2.0 TensorFlow 2 / 2.0 官方文档中文版，迁移学习(transfer learning)分类 CIFAR-10 。 5 | tags: 6 | - TensorFlow 2 7 | - 官方文档 8 | keywords: 9 | - TensorFlow2.0 10 | - TensorFlow2文档 11 | - TensorFlow2.0文档 12 | nav: 简明教程 13 | categories: 14 | - TensorFlow2 文档 15 | image: post/tf2doc-cnn-cifar10/cifar10-eg.jpg 16 | github: https://github.com/geektutu/tensorflow2-docs-zh 17 | --- 18 | 19 | **TF2.0 TensorFlow 2 / 2.0 中文文档：TFHub 迁移学习 transfer learning** 20 | 21 | 主要内容：使用 [TFHub](https://www.tensorflow.org/hub) 中的预训练模型 _ImageNet_ 进行迁移学习，实现图像分类，数据集使用 CIFAR-10。 22 | 23 | ## ImageNet 模型简介 24 | 25 | TFHub 上有很多预训练好的模型(pretrained model)，这次我们选择`ImageNet`。ImageNet 数据集大约有1500万张图片，2.2万类，可以说你能想到，想象不到的图片都能在里面找到。想下载感受一下的话可以到官网下载[ImageNet](http://www.image-net.org/)。 26 | 27 | 当然每次训练不太可能使用所有的图片，一般使用子集，比如2012年ILSVRC分类数据集使用了大概1/10的图片。我们今天用于迁移学习的预训练模型就只有1001个分类，想知道这1001类分别有哪些可以看[这里](https://storage.googleapis.com/download.tensorflow.org/data/ImageNetLabels.txt)。 28 | 29 | ### 下载 ImageNet Classifier 30 | 31 | ```python 32 | # geektutu.com 33 | import numpy as np 34 | from PIL import Image 35 | import matplotlib.pylab as plt 36 | import tensorflow as tf 37 | import tensorflow_hub as hub 38 | from tensorflow.keras import layers, datasets 39 | 40 | url ="https://tfhub.dev/google/tf2-preview/mobilenet_v2/classification/4" 41 | model = tf.keras.Sequential([ 42 | hub.KerasLayer(url, input_shape=(224, 224, 3)) 43 | ]) 44 | ``` 45 | 46 | ImageNet 数据集中的图片大小固定为 (224, 224, 3)，因此模型的输入也是 (224, 224, 3)。 47 | 48 | ### 测试任意图片 49 | 50 | 在这里选取一张兔子的图片，也就是本站的 logo 来测试这个预训练好的模型。 51 | 52 | ```python 53 | # geektutu.com 54 | tutu = tf.keras.utils.get_file('tutu.png','https://geektutu.com/img/icon.png') 55 | tutu = Image.open(tutu).resize((224, 224)) 56 | tutu 57 | ``` 58 | ![geektutu](https://geektutu.com/img/icon.png) 59 | 60 | ```python 61 | # geektutu.com 62 | result = model.predict(np.array(tutu).reshape(1, 224, 224, 3)/255.0) 63 | ans = np.argmax(result[0], axis=-1) 64 | print('result.shape:', result.shape, 'ans:', ans) 65 | # result.shape: (1, 1001) ans: 332 66 | ``` 67 | 68 | 模型的输出有1001个分类，测试的结果是332，接下来我们将下载 _ImageNetLabels.txt_ ，就可以知道332代表的分类的名称，可以看到结果是 hare，即`兔`。 69 | 70 | ```python 71 | # geektutu.com 72 | labels_url = 'https://storage.googleapis.com/download.tensorflow.org/data/ImageNetLabels.txt' 73 | labels_path = tf.keras.utils.get_file('ImageNetLabels.txt', labels_url) 74 | imagenet_labels = np.array(open(labels_path).read().splitlines()) 75 | print(imagenet_labels[ans]) 76 | # hare 77 | ``` 78 | 79 | ## 迁移学习 80 | 81 | 在实际的应用中，预训练好的模型的输入输出可能并不能满足我们的需求，另外，训练上百万甚至上千万张图片，可能需要花费好几天的时间，那有没有办法只使用训练好的模型的一部分呢？训练好的模型的前几层对特征提取有非常好的效果，如果可以直接使用，那就事半功倍了。这种方法被称之为迁移学习(transfer learning)。 82 | 83 | 在接下来的例子中，我们复用了 _ImageNet Classifier_ 的特征提取的部分，并定义了自己的输出层。因为原来的模型输出是1001个分类，而我们希望识别的 CIFAR-10 数据集只有10个分类。 84 | 85 | ### resize 数据集 86 | 87 | 这次demo使用的是 CIFAR-10 数据集，这个数据集在上篇文档 [卷积神经网络分类 CIFAR-10](https://geektutu.com/post/tf2doc-cnn-cifar10.html)有比较详细的介绍，这里就不重复介绍了。再简单看一看这个数据集中的15张样例图片。 88 | 89 | ![CIFAR-10 examples](tf2doc-tfhub-image-tl/cifar10-eg.jpg) 90 | 91 | _ImageNet Classifier_ 的输入固定为(224, 224, 3)，但 CIFAR-10 数据集中的图片大小是 32 * 32，简单起见，我们将每一张图片大小从 32x32 转换为 224x224，使用`pillow`库提供的 resize 方法。因为读取全部的数据，内存会被撑爆，所以训练集只截取了 30,000 张图片。 92 | 93 | ```python 94 | def resize(d, size=(224, 224)): 95 | return np.array([np.array(Image.fromarray(v).resize(size, Image.ANTIALIAS)) 96 | for i, v in enumerate(d)]) 97 | 98 | (train_x, train_y), (test_x, test_y) = datasets.cifar10.load_data() 99 | train_x, test_x = resize(train_x[:30000])/255.0, resize(test_x)/255.0 100 | train_y = train_y[:30000] 101 | ``` 102 | 103 | ### 下载特征提取层 104 | 105 | TFHub 提供了 _ImageNet Classifier_ 去掉了最后的分类层的版本，可以直接下载使用。 106 | 107 | ```python 108 | feature_extractor_url = 'https://tfhub.dev/google/tf2-preview/mobilenet_v2/feature_vector/4' 109 | feature_extractor_layer = hub.KerasLayer(feature_extractor_url, 110 | input_shape=(224,224,3)) 111 | # 这一层的训练值保持不变 112 | feature_extractor_layer.trainable = False 113 | ``` 114 | 115 | ### 添加分类层 116 | 117 | ```python 118 | model = tf.keras.Sequential([ 119 | feature_extractor_layer, 120 | layers.Dense(10, activation='softmax') 121 | ]) 122 | model.compile(optimizer=tf.keras.optimizers.Adam(), 123 | loss='sparse_categorical_crossentropy', 124 | metrics=['acc']) 125 | model.summary() 126 | ``` 127 | 128 | 这一步，我们在特征提取层后面，添加了输出为10的全连接层，用于最后的分类。从`model.summary()`中我们可以看到特征提取层的输出是1280。 129 | 130 | ``` 131 | Model: "sequential_1" 132 | _________________________________________________________________ 133 | Layer (type) Output Shape Param # 134 | ================================================================= 135 | keras_layer_1 (KerasLayer) (None, 1280) 2257984 136 | _________________________________________________________________ 137 | dense (Dense) (None, 10) 12810 138 | ================================================================= 139 | Total params: 2,270,794 140 | Trainable params: 12,810 141 | Non-trainable params: 2,257,984 142 | _________________________________________________________________ 143 | ``` 144 | 145 | ### 训练并评估模型 146 | 147 | ```python 148 | history = model.fit(train_x, train_y, epochs=1) 149 | loss, acc = model.evaluate(test_x, test_y) 150 | print(acc) 151 | ``` 152 | 153 | ```bash 154 | 10000/10000 [=====] - 256s 26ms/sample - loss: 0.7636 - acc: 0.7657 155 | ``` 156 | 157 | 本文的示例模型非常简单，在`feature_extractor_layer`直接添加了输出层，可训练参数很少。而且只使用大约一半的训练集，正确率仍然达到了 76% 。 158 | 159 | 类似于 ImageNet 的预训练模型还有很多，比如非常出名的 VGG 模型，有兴趣都可以尝试。 160 | 161 | 返回[文档首页](https://geektutu.com/post/tf2doc.html) 162 | 163 | > 完整代码：[Github - tfhub_image_transfer_learning.ipynb](https://github.com/geektutu/tensorflow2-docs-zh/tree/master/code) 164 | > 参考文档：[TensorFlow Hub with Keras](https://www.tensorflow.org/beta/tutorials/images/hub_with_keras) 165 | 166 | ## 附 推荐 167 | 168 | - [一篇文章入门 Python](https://geektutu.com/post/quick-python.html) -------------------------------------------------------------------------------- /Beginner-Images/tf2doc-tfhub-image-tl/cifar10-eg.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/geektutu/tensorflow2-docs-zh/3418ce1c0e721f549d0cff7eeff65ca030c55f82/Beginner-Images/tf2doc-tfhub-image-tl/cifar10-eg.jpg -------------------------------------------------------------------------------- /Beginner-ML-basics/tf2doc-ml-basic-image.md: -------------------------------------------------------------------------------- 1 | --- 2 | title: TensorFlow 2 中文文档 - MNIST 图像分类 3 | date: 2019-07-09 00:30:10 4 | description: TensorFlow2文档，TensorFlow2.0文档，TensorFlow2.0 TF2.0 TensorFlow 2 / 2.0 官方文档中文版，图像分类 Classify images，示例使用 Fashion MNIST 数据集。 5 | tags: 6 | - TensorFlow 2 7 | - 官方文档 8 | keywords: 9 | - TensorFlow2.0 10 | - TensorFlow2文档 11 | - TensorFlow2.0文档 12 | - Fashion MNIST 13 | - 图像分类 14 | - Classify images 15 | nav: 简明教程 16 | categories: 17 | - TensorFlow2 文档 18 | image: post/tf2doc-ml-basic-image/fashion-mnist-sm.jpg 19 | github: https://github.com/geektutu/tensorflow2-docs-zh 20 | --- 21 | 22 | **TF2.0 TensorFlow 2 / 2.0 中文文档 - 图像分类 Classify images** 23 | 24 | 主要内容：使用神经网络对服饰图片进行分类。 25 | 26 | 这篇文档使用高级API`tf.keras`在TensorFlow中搭建和训练模型。 27 | 28 | ```python 29 | # TensorFlow and tf.keras 30 | import tensorflow as tf 31 | from tensorflow import keras 32 | 33 | # Helper libraries 34 | import numpy as np 35 | ``` 36 | 37 | ## 使用 Fashion MNIST 数据集 38 | 39 | ![tf2doc-ml-basic-image](tf2doc-ml-basic-image/fashion-mnist.jpg) 40 | 41 | Fashion Mnist数据集由70,000张黑白图片构成，每张图片大小为 28x28，由十类服饰图片构成。另一个MNIST数据集是手写数字，Fashion MNIST 与之相比更有挑战性，适合用来验证算法。 42 | 43 | 我们使用60,000张图片作为训练集，10,000张图片作为测试集。这个数据集可以从 TensorFlow 中直接获取，返回值为numpy数组。 44 | 45 | ```python 46 | fashion_mnist = keras.datasets.fashion_mnist 47 | 48 | (train_images, train_labels), (test_images, test_labels) = fashion_mnist.load_data() 49 | ``` 50 | 51 | 图片大小28x28，每个像素值取值范围0-255。标签是整数，取值范围0-9，与实际的服饰类别对应关系如下。 52 | 53 | | 标签 | 类别 | 标签 | 类别 | 标签 | 类别 | 标签 | 类别 | 54 | | ---- | ----------- | ---- | ------ | ---- | ------- | ---- | ---------- | 55 | | 0 | T-shirt/top | 3 | Dress | 6 | Shirt | 9 | Ankle boot | 56 | | 1 | Trouser | 4 | Coat | 7 | Sneaker | | | 57 | | 2 | Pullover | 5 | Sandal | 8 | Bag | | | 58 | 59 | ## 数据格式 60 | 61 | ```python 62 | train_images.shape # (60000, 28, 28) 63 | len(train_labels) # 60000 64 | train_labels # ([9, 0, 0, ..., 3, 0, 5], dtype=uint8) 65 | test_images.shape # (10000, 28, 28) 66 | len(test_labels) # 10000 67 | ``` 68 | 69 | ## 预处理 70 | 71 | 训练之前，我们需要对数据进行预处理。图片的每个像素值在0-255之间，需要转为0-1。训练集和测试集都需要经过相同的处理。 72 | 73 | ```python 74 | train_images = train_images / 255.0 75 | test_images = test_images / 255.0 76 | ``` 77 | 78 | ## 搭建模型 79 | 80 | 神经网络的基本构成是网络层(layer)，大部分深度学习网络都由多个简单的 layers 构成。 81 | 82 | ```python 83 | model = keras.Sequential([ 84 | keras.layers.Flatten(input_shape=(28, 28)), 85 | keras.layers.Dense(128, activation='relu'), 86 | keras.layers.Dense(10, activation='softmax') 87 | ]) 88 | ``` 89 | 90 | 网络的第一层，`Flatten`将输入从28x28 的二维数组转为784的一维数组，这一层的作用仅仅是将每一行值平铺在一行。 91 | 92 | 接下来是2层`Dense`，即全连接层(fully connected, FC)，第一层`Dense`有128个神经元。第二层有10个神经元，经过 _softmax_ 后，返回了和为1长度为10的概率数组，每一个数分别代表当前图片属于分类0-9的概率。 93 | 94 | ## 编译模型 95 | 96 | 模型准备训练前，在模型编译(Compile)时还需要设置一些参数。 97 | 98 | - _Loss function_ - 损失函数，训练时评估模型的正确率，希望最小化这个函数，往正确的方向训练模型。 99 | - _Optimizer_ - 优化器算法，更新模型参数的算法。 100 | - _Metrics_ - 指标，用来监视训练和测试步数，下面的例子中使用`accuracy`，即图片被正确分类的比例。 101 | 102 | ```python 103 | model.compile(optimizer='adam', 104 | loss='sparse_categorical_crossentropy', 105 | metrics=['accuracy']) 106 | ``` 107 | 108 | ## 训练模型 109 | 110 | 训练神经网络，通常有以下几个步骤。 111 | 112 | - 传入训练数据，`train_images`和`train_labels`。 113 | - 训练模型去关联图片和标签。 114 | - 模型对测试集`test_images`作预测，并用`test_labels`验证预测结果。 115 | 116 | 使用`model.fit`函数开始训练。 117 | 118 | ```python 119 | model.fit(train_images, train_labels, epochs=10) 120 | ``` 121 | 122 | ```bash 123 | Train on 60000 samples 124 | Epoch 1/10 125 | 60000/60000 [========] - 4s 70us/sample - loss: 0.5032 - accuracy: 0.8234 126 | Epoch 2/10 127 | 60000/60000 [========] - 4s 64us/sample - loss: 0.3793 - accuracy: 0.8618 128 | ... 129 | Epoch 10/10 130 | 60000/60000 [========] - 4s 66us/sample - loss: 0.2389 - accuracy: 0.9115 131 | ``` 132 | 133 | 最终达到了88%左右的准确率。 134 | 135 | 136 | 137 | ## 评估准确率 138 | 139 | 接下来，看看在测试集中表现如何？ 140 | 141 | ```python 142 | test_loss, test_acc = model.evaluate(test_images, test_labels) 143 | print('\nTest accuracy:', test_acc) 144 | # 10000/10000 [========] - 0s 37us/sample - loss: 0.3610 - accuracy: 0.8777 145 | # Test accuracy: 0.8777 146 | ``` 147 | 148 | 测试集的准确率低于训练集，训练集和测试集准确率之间的差距代表模型_过拟合_(_overfitting_)。即对于训练中没有见过的新数据，模型表现差。 149 | 150 | ## 预测 151 | 152 | 使用`predict`函数进行预测。 153 | 154 | ```python 155 | predictions = model.predict(test_images) 156 | predictions[0] 157 | ``` 158 | 159 | 看下第一张图片的预测结果。 160 | 161 | ```bash 162 | array([1.06-05, 5.06-12, 8.44-08, 4.09-09, 2.87-07, 2.28-04, 163 | 6.18-06, 2.48-02, 3.81-06, 9.74-01], dtype=float32) 164 | ``` 165 | 166 | 每次预测返回长度为10的数组，代表属于每一种分类的可能性，最大可能性的label是9。测试集中的数据也是9，预测正确。 167 | 168 | ```python 169 | np.argmax(predictions[0]) # 9 170 | test_labels[0] # 9 171 | ``` 172 | 173 | 返回[文档首页](https://geektutu.com/post/tf2doc.html) 174 | 175 | > 参考地址：[Train your first neural network: basic classification](https://www.tensorflow.org/beta/tutorials/keras/basic_classification) 176 | 177 | ## 附 推荐 178 | 179 | - [一篇文章入门 Python](https://geektutu.com/post/quick-python.html) -------------------------------------------------------------------------------- /Beginner-ML-basics/tf2doc-ml-basic-image/fashion-mnist-sm.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/geektutu/tensorflow2-docs-zh/3418ce1c0e721f549d0cff7eeff65ca030c55f82/Beginner-ML-basics/tf2doc-ml-basic-image/fashion-mnist-sm.jpg -------------------------------------------------------------------------------- /Beginner-ML-basics/tf2doc-ml-basic-image/fashion-mnist.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/geektutu/tensorflow2-docs-zh/3418ce1c0e721f549d0cff7eeff65ca030c55f82/Beginner-ML-basics/tf2doc-ml-basic-image/fashion-mnist.jpg -------------------------------------------------------------------------------- /Beginner-ML-basics/tf2doc-ml-basic-overfit.md: -------------------------------------------------------------------------------- 1 | --- 2 | title: TensorFlow 2 中文文档 - 过拟合与欠拟合 3 | date: 2019-07-12 01:10:10 4 | description: TensorFlow2文档，TensorFlow2.0文档，TensorFlow2.0 TF2.0 TensorFlow 2 / 2.0 官方文档中文版，过拟合与欠拟合 Explore overfitting and underfitting，示例使用 IMDB 影评数据集。 5 | tags: 6 | - TensorFlow 2 7 | - 官方文档 8 | keywords: 9 | - TensorFlow2.0 10 | - TensorFlow2文档 11 | - TensorFlow2.0文档 12 | - overfitting 13 | - 过拟合 14 | nav: 简明教程 15 | categories: 16 | - TensorFlow2 文档 17 | image: post/tf2doc-ml-basic-overfit/imdb.jpg 18 | github: https://github.com/geektutu/tensorflow2-docs-zh 19 | --- 20 | 21 | **TF2.0 TensorFlow 2 / 2.0 中文文档：过拟合与欠拟合 Explore overfitting and underfitting** 22 | 23 | 24 | 主要内容：探索正则化(weight regularization)和 dropout 两种避免过拟合的方式改善 IMDB 影评分类效果。 25 | 26 | 之前不管是对影评数据分类，还是预测燃油效率，都可以看到模型在验证集的准确率会在训练一段时间后达到顶峰，之后开始下降。 27 | 28 | 换句话说，过拟合了。在训练集上达到较高的准确率是容易的，但我们的目的是在测试集，即模型没有见过的数据集上表现良好。 29 | 30 | 过拟合的反面是欠拟合。即对于测试数据还存在改进的空间。通常由如下原因导致： 31 | 32 | - 模型不够好。 33 | - 过度正则化(over-regularized) 34 | - 训练时间过短。 35 | 36 | 这些都意味着模型没有学习到训练数据的特征。 37 | 38 | 如果训练太久，模型可能开始过拟合，导致在测试集上表现不佳，因此我们需要取得一个平衡。 39 | 40 | 41 | ## IMDB 数据集 42 | 43 | 我们这次采用 `multi-hot encoding`的方式处理 IMDB 数据集，这样能快速达到过拟合的效果。multi-hot 的方式很简单，给每一个单词一个编号，每句话用长度为10,000的一维向量表示，将出现单词的位置置为1即可。 44 | 45 | ```python 46 | import tensorflow as tf 47 | from tensorflow import keras 48 | from tensorflow.keras.datasets import imdb 49 | 50 | import numpy as np 51 | import matplotlib.pyplot as plt 52 | 53 | # 解决中文乱码问题 54 | plt.rcParams['font.sans-serif'] = ['SimHei'] 55 | plt.rcParams['axes.unicode_minus'] = False 56 | plt.rcParams['font.size'] = 20 57 | 58 | N = 10000 59 | 60 | def multi_hot_encoding(sentences, dim=10000): 61 | results = np.zeros((len(sentences), dim)) 62 | for i, word_indices in enumerate(sentences): 63 | results[i, word_indices] = 1.0 64 | return results 65 | 66 | 67 | (train_x, train_y), (test_x, test_y) = imdb.load_data(num_words=N) 68 | train_x = multi_hot_encoding(train_x) 69 | test_x = multi_hot_encoding(test_x) 70 | 71 | plt.plot(train_x[0]) 72 | plt.show() 73 | ``` 74 | ![IMDB words distribution](tf2doc-ml-basic-overfit/imdb.jpg) 75 | 76 | 如果出现这样的错误，[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1045)，解决办法可以参考 [Github - google-images-download](https://github.com/hardikvasa/google-images-download/issues/140) 77 | 78 | ## 过拟合 79 | 80 | 防止过拟合最简单的方式是降低模型复杂度，比如减少模型的学习参数(learnable parameters)。减少神经网络的层数，减少每一层网络的节点数都能达到目的。在深度学习中，学习参数的数量往往代表了模型“容量”(capacity)。容量越大，记忆能力越强，更容易学习到训练集的特征和标签的映射关系，类似于字典。如果这种映射关系缺乏泛化能力，在测试集上不可能有好的表现。 81 | 82 | 训练的目的不是为了让模型完全拟合训练数据，而是为了训练出具有泛化能力的模型。 83 | 84 | 反过来，如果模型较小，记忆能力较弱，学习映射关系比较困难。为了最小化损失(loss)，模型需要压缩学习到的东西，因而具备更好的预测能力。但是，如果模型过小，没有足够的神经元记住有用的信息，训练会非常困难。因此需要取得权衡。 85 | 86 | 如何选择合适的网络层数和神经元数量呢？没有公式，只能不断尝试。 87 | 88 | 为了确定合适的模型大小，建议一开始采用一个参数少的模型，然后逐渐增大，直到验证集的损失(loss)基本不再变化。我们用之前的 IMDB 影评分类模型来试一试。 89 | 90 | 先搭一个基线模型，再创建大、小2个模型比较。 91 | 92 | ```python 93 | def build_and_train(hidden_dim, regularizer=None, dropout=0): 94 | model = keras.Sequential([ 95 | keras.layers.Dense(hidden_dim, activation='relu', 96 | input_shape=(N,), 97 | kernel_regularizer=regularizer), 98 | keras.layers.Dropout(dropout), 99 | keras.layers.Dense(hidden_dim, activation='relu', 100 | kernel_regularizer=regularizer), 101 | keras.layers.Dropout(dropout), 102 | keras.layers.Dense(1, activation='sigmoid') 103 | ]) 104 | 105 | model.compile(optimizer='adam', loss='binary_crossentropy', 106 | metrics=['accuracy', 'binary_crossentropy']) 107 | history = model.fit(train_x, train_y, epochs=10, batch_size=512, 108 | validation_data=(test_x, test_y), verbose=0) 109 | 110 | return history 111 | 112 | 113 | baseline_history = build_and_train(16) 114 | smaller_history = build_and_train(4) 115 | larger_history = build_and_train(512) 116 | ``` 117 | 118 | 我们画图看一看这三个模型在训练集和验证集上的表现。 119 | 120 | ```python 121 | def plot_history(histories, key='binary_crossentropy'): 122 | plt.figure(figsize=(10, 6)) 123 | 124 | for name, history in histories: 125 | val = plt.plot(history.epoch, history.history['val_'+key], 126 | '--', label=name + ' 验证集') 127 | plt.plot(history.epoch, history.history[key], 128 | color=val[0].get_color(), label=name + ' 训练集') 129 | 130 | plt.xlabel('Epochs') 131 | plt.ylabel('Loss - ' + key) 132 | plt.legend() 133 | 134 | plt.xlim([0, max(history.epoch)]) 135 | 136 | plot_history([('基线', baseline_history), 137 | ('较小', smaller_history), 138 | ('较大', larger_history)]) 139 | ``` 140 | 141 | ![3_train_val_loss](tf2doc-ml-basic-overfit/3_train_val_loss.jpg) 142 | 143 | 大的模型在第一波(epoch)训练后，就已经过拟合了。模型容量越大，就会越快地拟合训练数据，虽然训练集损失(loss)很低，但事实上已经过拟合了，验证集的loss明显大于训练集。 144 | 145 | ## 如何防止过拟合 146 | 147 | ### 权重正则化 148 | 149 | > 如果关于同一个问题有许多种理论，每一种都能作出同样准确的预言，那么应该挑选其中使用假定最少的。尽管越复杂的方法通常能做出越好的预言，但是在结果大致相同的情况下，假设越少越好。 ---- 维基百科《奥卡剃须刀》 150 | 151 | 你可能听说过奥卡剃须刀原则——如无必要，勿增实体。将这个原则应用到神经网络模型，相同的预测能力下，优选简单的模型，比起复杂的模型，简单的模型不容易过拟合。 152 | 153 | 将模型变简单，除了将模型变小（减少网络层数和每层神经元个数）以外，还有另一种方式，减小模型权重(w)的熵(entropy)。即限制权重值在一个较小的范围内，这样模型中权重分布看起来更“regular”，这被称为“权重正则化”(weight regularization)。常用**L1正则化**和**L2正则化**2种方式，L2 正则化更通用。 154 | 155 | > 正则化建议参考：[机器学习中常常提到的正则化到底是什么意思？](https://www.zhihu.com/question/20924039) 156 | 157 | ### Dropout 158 | 159 | 在神经网络中，Dropout是最有效的以及使用最广泛的正则化方式。Dropout作用在网络层，训练过程中随机丢弃(dropping out)一部分输出值（例如置为0），Dropout的比例一般置为0.2到0.5之间。例如： 160 | 161 | ```python 162 | [0.2, 0.3, 0.5, 0.7, 0.9] 163 | # after 40% dropout 164 | [0.2, 0, 0.5, 0.7, 0] 165 | ``` 166 | 167 | 看看效果吧。 168 | 169 | ```python 170 | l2_model_history = build_and_train(16, keras.regularizers.l2(0.001)) 171 | dpt_model_history = build_and_train(16, dropout=0.2) 172 | plot_history([('基线', baseline_history), 173 | ('L2正则', l2_model_history), 174 | ('Dropout', dpt_model_history)]) 175 | ``` 176 | 177 | ![l2_dropout_loss](tf2doc-ml-basic-overfit/l2_dropout_loss.jpg)) 178 | 179 | 返回[文档首页](https://geektutu.com/post/tf2doc.html) 180 | 181 | > 完整代码：[Github - explore_imdb_overfit.ipynb](https://github.com/geektutu/tensorflow2-docs-zh/tree/master/code) 182 | > 参考文档：[Explore overfitting and underfitting](https://www.tensorflow.org/beta/tutorials/keras/overfit_and_underfit) 183 | 184 | 185 | ## 附 推荐 186 | 187 | - [一篇文章入门 Python](https://geektutu.com/post/quick-python.html) -------------------------------------------------------------------------------- /Beginner-ML-basics/tf2doc-ml-basic-overfit/3_train_val_loss.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/geektutu/tensorflow2-docs-zh/3418ce1c0e721f549d0cff7eeff65ca030c55f82/Beginner-ML-basics/tf2doc-ml-basic-overfit/3_train_val_loss.jpg -------------------------------------------------------------------------------- /Beginner-ML-basics/tf2doc-ml-basic-overfit/imdb.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/geektutu/tensorflow2-docs-zh/3418ce1c0e721f549d0cff7eeff65ca030c55f82/Beginner-ML-basics/tf2doc-ml-basic-overfit/imdb.jpg -------------------------------------------------------------------------------- /Beginner-ML-basics/tf2doc-ml-basic-overfit/l2_dropout_loss.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/geektutu/tensorflow2-docs-zh/3418ce1c0e721f549d0cff7eeff65ca030c55f82/Beginner-ML-basics/tf2doc-ml-basic-overfit/l2_dropout_loss.jpg -------------------------------------------------------------------------------- /Beginner-ML-basics/tf2doc-ml-basic-regression.md: -------------------------------------------------------------------------------- 1 | --- 2 | title: TensorFlow 2 中文文档 - 回归预测燃油效率 3 | date: 2019-07-11 01:00:10 4 | description: TensorFlow2文档，TensorFlow2.0文档，TensorFlow2.0 TF2.0 TensorFlow 2 / 2.0 官方文档中文版，Regression 回归，示例使用 Auto MPG 燃油效率数据集。 5 | tags: 6 | - TensorFlow 2 7 | - 官方文档 8 | keywords: 9 | - TensorFlow2.0 10 | - TensorFlow2文档 11 | - TensorFlow2.0文档 12 | - Regression 13 | - 回归 14 | nav: 简明教程 15 | categories: 16 | - TensorFlow2 文档 17 | image: post/tf2doc-ml-basic-regression/sns.jpg 18 | github: https://github.com/geektutu/tensorflow2-docs-zh 19 | --- 20 | 21 | **TF2.0 TensorFlow 2 / 2.0 文档：Regression 回归** 22 | 23 | 主要内容：使用回归预测烟油效率。 24 | 25 | 回归通常用来预测连续值，比如价格和概率。分类问题不一样，类别是固定的，目的是判断属于哪一类。比如给你一堆猫和狗的图片，判断一张图片是猫还是狗就是一个典型的分类问题。 26 | 27 | 接下来使用的是经典的 [Auto MPG](https://archive.ics.uci.edu/ml/datasets/auto+mpg) 数据集，这个数据集包括气缸(cylinders)，排量(displayment)，马力(horsepower) 和重量(weight)等属性。我们需要利用这些属性搭建模型，预测汽车的燃油效率(fuel efficiency)。 28 | 29 | 模型搭建使用`tf.keras` API。 30 | 31 | ```python 32 | import pathlib 33 | 34 | import matplotlib.pyplot as plt 35 | import pandas as pd 36 | import seaborn as sns 37 | import tensorflow as tf 38 | from tensorflow import keras 39 | from tensorflow.keras import layers 40 | ``` 41 | 42 | 43 | 44 | ## Auto MPG 数据集 45 | 46 | ### 获取数据 47 | 48 | ```python 49 | # 下载数据集到本地 50 | url = "http://archive.ics.uci.edu/ml/machine-learning-databases/auto-mpg/auto-mpg.data" 51 | dataset_path = keras.utils.get_file("auto-mpg.data", url) 52 | 53 | # 使用Pandas读取数据 54 | column_names = ['MPG','气缸','排量','马力','重量','加速度', '年份', '产地'] 55 | raw_dataset = pd.read_csv(dataset_path, names=column_names, 56 | na_values = "?", comment='\t', 57 | sep=" ", skipinitialspace=True) 58 | 59 | dataset = raw_dataset.copy() 60 | # 查看前3条数据 61 | dataset.head(3) 62 | ``` 63 | 64 | | | MPG | 气缸 | 排量 | 马力 | 重量 | 加速度 | 年份 | 产地 | 65 | | ---: | ---: | ---: | ----: | ----: | -----: | -----: | ---: | ---: | 66 | | 0 | 18.0 | 8 | 307.0 | 130.0 | 3504.0 | 12.0 | 70 | 1 | 67 | | 1 | 15.0 | 8 | 350.0 | 165.0 | 3693.0 | 11.5 | 70 | 1 | 68 | | 2 | 18.0 | 8 | 318.0 | 150.0 | 3436.0 | 11.0 | 70 | 1 | 69 | 70 | ### 清洗数据 71 | 72 | 检查是否有 NA 值。 73 | 74 | ```python 75 | dataset.isna().sum() 76 | ``` 77 | 78 | ```bash 79 | MPG 0 80 | 气缸 0 81 | 排量 0 82 | 马力 6 83 | 重量 0 84 | 加速度 0 85 | 年份 0 86 | 产地 0 87 | dtype: int64 88 | ``` 89 | 90 | 直接去除含有NA值的行（马力） 91 | 92 | ```python 93 | dataset = dataset.dropna() 94 | ``` 95 | 96 | 在获取的数据集中，`Origin`(产地)不是数值类型，需转为独热编码。 97 | 98 | ```python 99 | origin = dataset.pop('产地') 100 | dataset['美国'] = (origin == 1)*1.0 101 | dataset['欧洲'] = (origin == 2)*1.0 102 | dataset['日本'] = (origin == 3)*1.0 103 | # 看一看转换后的结果 104 | dataset.head(3) 105 | ``` 106 | 107 | | | MPG | 气缸 | 排量 | 马力 | 重量 | 加速度 | 年份 | 美国 | 欧洲 | 日本 | 108 | | ---: | ---: | ---: | ----: | ----: | -----: | -----: | ---: | ---: | ---: | ---: | 109 | | 0 | 18.0 | 8 | 307.0 | 130.0 | 3504.0 | 12.0 | 70 | 1.0 | 0.0 | 0.0 | 110 | | 1 | 15.0 | 8 | 350.0 | 165.0 | 3693.0 | 11.5 | 70 | 1.0 | 0.0 | 0.0 | 111 | | 2 | 18.0 | 8 | 318.0 | 150.0 | 3436.0 | 11.0 | 70 | 1.0 | 0.0 | 0.0 | 112 | 113 | ### 划分训练集与测试集 114 | 115 | ```python 116 | # 训练集 80%， 测试集 20% 117 | train_dataset = dataset.sample(frac=0.8, random_state=0) 118 | test_dataset = dataset.drop(train_dataset.index) 119 | ``` 120 | 121 | ### 检查数据 122 | 123 | 快速看一看训练集中属性两两之间的关系吧。 124 | 125 | ```python 126 | # 解决中文乱码问题 127 | plt.rcParams['font.sans-serif']=['SimHei'] 128 | plt.rcParams['axes.unicode_minus']=False 129 | 130 | sns.pairplot(train_dataset[["MPG", "气缸", "排量", "重量"]], diag_kind="kde") 131 | ``` 132 | 133 | > matplotlib 中文乱码看这里：[matplotlib图例中文乱码?](https://www.zhihu.com/question/25404709) 134 | 135 | ![Seaborn four feature relation](tf2doc-ml-basic-regression/sns.jpg) 136 | 137 | 你还可以使用`train_dataset.describle()`快速浏览每一属性的平均值、标准差、最小值、最大值等信息，能够帮助你快速地识别出不合理的数据。 138 | 139 | ```python 140 | train_stats = train_dataset.describe() 141 | train_stats.pop("MPG") 142 | train_stats = train_stats.transpose() 143 | train_stats 144 | ``` 145 | 146 | | | count | mean | std | min | 25% | 50% | 75% | max | 147 | | ---: | ----: | ---------: | ---------: | ---: | -----: | ----: | -----: | ----: | 148 | | 气缸 | 314.0 | 5.477707 | 1.699788 | 3.0 | 4.00 | 4.0 | 8.00 | 8.0 | 149 | | 排量 | 314.0 | 195.318471 | 104.331589 | 68.0 | 105.50 | 151.0 | 265.75 | 455.0 | 150 | | ... | ... | ... | ... | ... | ... | .... | ... | ... | 151 | 152 | ### 分离 label 153 | 154 | ```python 155 | # 分离 label 156 | train_labels = train_dataset.pop('MPG') 157 | test_labels = test_dataset.pop('MPG') 158 | ``` 159 | 160 | ### 归一化数据 161 | 162 | 通常训练前需要归一化数据，不同属性使用的计量单位不一样，值的范围不一样，训练就会很困难。比如其中一个属性的范围是[0.1, 0.5]，而另一个属性的范围是[1000, 5000]，那数值大的属性就容易对训练产生干扰，很可能导致训练不能收敛，或者是数值小的属性在模型中几乎没有发挥作用。归一化将不同范围的数据映射到[0,1]的空间内，可以有效地避免这个问题。 163 | 164 | ```python 165 | def norm(x): 166 | return (x - train_stats['mean']) / train_stats['std'] 167 | normed_train_data = norm(train_dataset) 168 | normed_test_data = norm(test_dataset) 169 | ``` 170 | 171 | ## 模型 172 | 173 | ### 搭建模型 174 | 175 | 我们的模型包含2个全连接的隐藏层构成，输出层返回一个连续值。 176 | 177 | ```python 178 | def build_model(): 179 | input_dim = len(train_dataset.keys()) 180 | model = keras.Sequential([ 181 | layers.Dense(64, activation='relu', input_shape=[input_dim,]), 182 | layers.Dense(64, activation='relu'), 183 | layers.Dense(1) 184 | ]) 185 | 186 | model.compile(loss='mse', metrics=['mae', 'mse'], 187 | optimizer=tf.keras.optimizers.RMSprop(0.001)) 188 | return model 189 | 190 | model = build_model() 191 | # 打印模型的描述信息，每一层的大小、参数个数等 192 | model.summary() 193 | ``` 194 | 195 | ```python 196 | Model: "sequential_1" 197 | _________________________________________________________________ 198 | Layer (type) Output Shape Param # 199 | ================================================================= 200 | dense_4 (Dense) (None, 64) 640 201 | _________________________________________________________________ 202 | dense_5 (Dense) (None, 32) 2080 203 | _________________________________________________________________ 204 | dense_6 (Dense) (None, 1) 33 205 | ================================================================= 206 | Total params: 2,753 207 | Trainable params: 2,753 208 | Non-trainable params: 0 209 | _________________________________________________________________ 210 | ``` 211 | 212 | ### 训练模型 213 | 214 | 在之前的案例，比如[结构化数据分类](https://geektutu.com/post/tf2doc-ml-basic-structured-data.html)，我们调用`model.fit`会打印出训练的进度。我们可以禁用默认的行为，并自定义训练进度条。 215 | 216 | ```python 217 | import sys 218 | 219 | 220 | EPOCHS = 1000 221 | 222 | class ProgressBar(keras.callbacks.Callback): 223 | def on_epoch_end(self, epoch, logs): 224 | # 显示进度条 225 | self.draw_progress_bar(epoch + 1, EPOCHS) 226 | 227 | def draw_progress_bar(self, cur, total, bar_len=50): 228 | cur_len = int(cur / total * bar_len) 229 | sys.stdout.write("\r") 230 | sys.stdout.write("[{:<{}}] {}/{}".format("=" * cur_len, bar_len, cur, total)) 231 | sys.stdout.flush() 232 | 233 | history = model.fit( 234 | normed_train_data, train_labels, 235 | epochs=EPOCHS, validation_split = 0.2, verbose=0, 236 | callbacks=[ProgressBar()]) 237 | ``` 238 | 239 | ```bash 240 | [==================================================] 1000/1000 241 | ``` 242 | 243 | 训练过程都存储在了`history`对象中，我们可以借助 matplotlib 将训练过程可视化。 244 | 245 | ```python 246 | hist = pd.DataFrame(history.history) 247 | hist['epoch'] = history.epoch 248 | hist.tail(3) 249 | ``` 250 | 251 | | loss | mae | mse | val_loss | val_mae | val_mse | epoch | | 252 | | ---: | -------: | -------: | -------: | -------: | -------: | -------: | ---- | 253 | | 997 | 3.132053 | 1.142280 | 3.132053 | 9.711935 | 2.361466 | 9.711935 | 997 | 254 | | 998 | 3.021109 | 1.093424 | 3.021109 | 9.488593 | 2.298264 | 9.488593 | 998 | 255 | | 999 | 3.028849 | 1.132241 | 3.028849 | 9.453931 | 2.275017 | 9.453931 | 999 | 256 | 257 | ```python 258 | def plot_history(history): 259 | hist = pd.DataFrame(history.history) 260 | hist['epoch'] = history.epoch 261 | plt.figure() 262 | plt.xlabel('epoch') 263 | plt.ylabel('metric - MSE') 264 | plt.plot(hist['epoch'], hist['mse'], label='训练集') 265 | plt.plot(hist['epoch'], hist['val_mse'], label = '验证集') 266 | plt.ylim([0, 20]) 267 | plt.legend() 268 | 269 | plt.figure() 270 | plt.xlabel('epoch') 271 | plt.ylabel('metric - MAE') 272 | plt.plot(hist['epoch'], hist['mae'], label='训练集') 273 | plt.plot(hist['epoch'], hist['val_mae'], label = '验证集') 274 | plt.ylim([0, 5]) 275 | plt.legend() 276 | 277 | plot_history(history) 278 | ``` 279 | 280 | ![MAE](tf2doc-ml-basic-regression/mse.jpg) 281 | 282 | 从图中，我们可以看到，从100 epoch开始，训练集的loss仍旧继续降低，但验证集的loss却在升高，说明过拟合了，训练应该早一点结束。接下来，我们使用 `keras.callbacks.EarlyStopping`，每一波(epoch)训练结束时，测试训练情况，如果训练不再有效果（验证集的loss，即val_loss 不再下降），则自动地停止训练。 283 | 284 | ```python 285 | model = build_model() 286 | early_stop = keras.callbacks.EarlyStopping(monitor='val_loss', patience=10) 287 | history = model.fit(normed_train_data, train_labels, epochs=EPOCHS, 288 | validation_split = 0.2, verbose=0, 289 | callbacks=[early_stop, ProgressBar()]) 290 | plot_history(history) 291 | ``` 292 | 293 | ```bash 294 | [=== ] 70/1000 295 | ``` 296 | 297 | 在第 70 epoch 时，停止了训练。 298 | 299 | ![MAE](tf2doc-ml-basic-regression/early_mse.jpg) 300 | 301 | 接下来使用测试集来评估训练效果。 302 | 303 | ```python 304 | loss, mae, mse = model.evaluate(normed_test_data, test_labels, verbose=0) 305 | print("测试集平均绝对误差(MAE): {:5.2f} MPG".format(mae)) 306 | # 测试集平均绝对误差(MAE): 1.90 MPG 307 | ``` 308 | 309 | 从图中我们也可以看出，1.9比验证集还略低一点。 310 | 311 | ## 预测 312 | 313 | 最后，我们使用测试集中的数据来预测 MPG 值。 314 | 315 | ```python 316 | test_pred = model.predict(normed_test_data).flatten() 317 | 318 | plt.scatter(test_labels, test_pred) 319 | plt.xlabel('真实值') 320 | plt.ylabel('预测值') 321 | plt.axis('equal') 322 | plt.axis('square') 323 | plt.xlim([0,plt.xlim()[1]]) 324 | plt.ylim([0,plt.ylim()[1]]) 325 | plt.plot([-100, 100], [-100, 100]) 326 | ``` 327 | 328 | 看起来，模型训练得还不错。 329 | 330 | ![Test True Pred](tf2doc-ml-basic-regression/test_true_pred.jpg) 331 | 332 | 333 | ## 结论 334 | 335 | - 均方误差(Mean Squared Error, MSE) 常作为回归问题的损失函数(loss function)，与分类问题不太一样。 336 | - 同样，评价指标(evaluation metrics)也不一样，分类问题常用准确率(accuracy)，回归问题常用平均绝对误差 (Mean Absolute Error, MAE) 337 | - 每一列数据都有不同的范围，每一列，即每一个feature的数据需要分别缩放到相同的范围。常用归一化的方式，缩放到[0, 1]。 338 | - 如果训练数据过少，最好搭建一个隐藏层少的小的神经网络，避免过拟合。 339 | - 早停法(Early Stoping)也是防止过拟合的一种方式。 340 | 341 | 返回[文档首页](https://geektutu.com/post/tf2doc.html) 342 | 343 | > 完整代码：[Github - auto_mpg_regression.ipynb](https://github.com/geektutu/tensorflow2-docs-zh/tree/master/code) 344 | > 参考文档：[Regression: Predict fuel efficiency](https://www.tensorflow.org/beta/tutorials/keras/basic_regression) 345 | 346 | ## 附 推荐 347 | 348 | - [一篇文章入门 Python](https://geektutu.com/post/quick-python.html) -------------------------------------------------------------------------------- /Beginner-ML-basics/tf2doc-ml-basic-regression/early_mse.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/geektutu/tensorflow2-docs-zh/3418ce1c0e721f549d0cff7eeff65ca030c55f82/Beginner-ML-basics/tf2doc-ml-basic-regression/early_mse.jpg -------------------------------------------------------------------------------- /Beginner-ML-basics/tf2doc-ml-basic-regression/mse.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/geektutu/tensorflow2-docs-zh/3418ce1c0e721f549d0cff7eeff65ca030c55f82/Beginner-ML-basics/tf2doc-ml-basic-regression/mse.jpg -------------------------------------------------------------------------------- /Beginner-ML-basics/tf2doc-ml-basic-regression/sns.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/geektutu/tensorflow2-docs-zh/3418ce1c0e721f549d0cff7eeff65ca030c55f82/Beginner-ML-basics/tf2doc-ml-basic-regression/sns.jpg -------------------------------------------------------------------------------- /Beginner-ML-basics/tf2doc-ml-basic-regression/test_true_pred.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/geektutu/tensorflow2-docs-zh/3418ce1c0e721f549d0cff7eeff65ca030c55f82/Beginner-ML-basics/tf2doc-ml-basic-regression/test_true_pred.jpg -------------------------------------------------------------------------------- /Beginner-ML-basics/tf2doc-ml-basic-save-model.md: -------------------------------------------------------------------------------- 1 | --- 2 | title: TensorFlow 2 中文文档 - 保存与加载模型 3 | date: 2019-07-13 00:05:10 4 | description: TensorFlow2文档，TensorFlow2.0文档，TensorFlow2.0 TF2.0 TensorFlow 2 / 2.0 官方文档中文版，保存与加载模型 Save and Restore model。 5 | tags: 6 | - TensorFlow 2 7 | - 官方文档 8 | keywords: 9 | - TensorFlow2.0 10 | - TensorFlow2文档 11 | - TensorFlow2.0文档 12 | - overfitting 13 | - 过拟合 14 | nav: 简明教程 15 | categories: 16 | - TensorFlow2 文档 17 | image: post/tf2doc-ml-basic-save-model/hdf5.png 18 | github: https://github.com/geektutu/tensorflow2-docs-zh 19 | --- 20 | 21 | ![TensorFlow HDF5](tf2doc-ml-basic-save-model/hdf5.png) 22 | 23 | 27 | 28 | **TF2.0 TensorFlow 2 / 2.0 中文文档：保存与加载模型 Save and Restore model** 29 | 30 | 主要内容：使用 `tf.keras`接口训练、保存、加载模型，数据集选用 MNIST 。 31 | 32 | ```bash 33 | $ pip install -q tensorflow==2.0.0-beta1 34 | $ pip install -q h5py pyyaml 35 | ``` 36 | 37 | ## 准备训练数据 38 | 39 | ```python 40 | import tensorflow as tf 41 | from tensorflow import keras 42 | from tensorflow.keras import datasets, layers, models, callbacks 43 | from tensorflow.keras.datasets import mnist 44 | 45 | import os 46 | file_path = os.path.abspath('./mnist.npz') 47 | 48 | (train_x, train_y), (test_x, test_y) = datasets.mnist.load_data(path=file_path) 49 | train_y, test_y = train_y[:1000], test_y[:1000] 50 | train_x = train_x[:1000].reshape(-1, 28 * 28) / 255.0 51 | test_x = test_x[:1000].reshape(-1, 28 * 28) / 255.0 52 | ``` 53 | 54 | ## 搭建模型 55 | 56 | ```python 57 | def create_model(): 58 | model = models.Sequential([ 59 | layers.Dense(512, activation='relu', input_shape=(784,)), 60 | layers.Dropout(0.2), 61 | layers.Dense(10, activation='softmax') 62 | ]) 63 | 64 | model.compile(optimizer='adam', metrics=['accuracy'], 65 | loss='sparse_categorical_crossentropy') 66 | 67 | return model 68 | 69 | def evaluate(target_model): 70 | _, acc = target_model.evaluate(test_x, test_y) 71 | print("Restore model, accuracy: {:5.2f}%".format(100*acc)) 72 | ``` 73 | 74 | ## 自动保存 checkpoints 75 | 76 | 这样做，一是训练结束后得到了训练好的模型，使用得不必再重新训练，二是训练过程被中断，可以从断点处继续训练。 77 | 78 | 设置`tf.keras.callbacks.ModelCheckpoint`回调可以实现这一点。 79 | 80 | ```python 81 | # 存储模型的文件名，语法与 str.format 一致 82 | # period=10：每 10 epochs 保存一次 83 | checkpoint_path = "training_2/cp-{epoch:04d}.ckpt" 84 | checkpoint_dir = os.path.dirname(checkpoint_path) 85 | cp_callback = callbacks.ModelCheckpoint( 86 | checkpoint_path, verbose=1, save_weights_only=True, period=10) 87 | 88 | model = create_model() 89 | model.save_weights(checkpoint_path.format(epoch=0)) 90 | model.fit(train_x, train_y, epochs=50, callbacks=[cp_callback], 91 | validation_data=(test_x, test_y), verbose=0) 92 | ``` 93 | 94 | ```python 95 | Epoch 00010: saving model to training_2/cp-0010.ckpt 96 | Epoch 00020: saving model to training_2/cp-0020.ckpt 97 | Epoch 00030: saving model to training_2/cp-0030.ckpt 98 | Epoch 00040: saving model to training_2/cp-0040.ckpt 99 | Epoch 00050: saving model to training_2/cp-0050.ckpt 100 | ``` 101 | 102 | 加载权重： 103 | 104 | ```python 105 | latest = tf.train.latest_checkpoint(checkpoint_dir) 106 | # 'training_2/cp-0050.ckpt' 107 | model = create_model() 108 | model.load_weights(latest) 109 | evaluate(model) 110 | ``` 111 | 112 | ```bash 113 | 1000/1000 [===] - 0s 90us/sample - loss: 0.4703 - accuracy: 0.8780 114 | Restore model, accuracy: 87.80% 115 | ``` 116 | 117 | ## 手动保存权重 118 | 119 | ```python 120 | # 手动保存权重 121 | model.save_weights('./checkpoints/mannul_checkpoint') 122 | model = create_model() 123 | model.load_weights('./checkpoints/mannul_checkpoint') 124 | evaluate(model) 125 | ``` 126 | 127 | ```python 128 | 1000/1000 [===] - 0s 90us/sample - loss: 0.4703 - accuracy: 0.8780 129 | Restore model, accuracy: 87.80% 130 | ``` 131 | 132 | ## 保存整个模型 133 | 134 | 上面的示例仅仅保存了模型中的权重(weights)，模型和优化器都可以一起保存，包括权重(weights)、模型配置(architecture)和优化器配置(optimizer configuration)。这样做的好处是，当你恢复模型时，完全不依赖于原来搭建模型的代码。 135 | 136 | 保存完整的模型有很多应用场景，比如在浏览器中使用 TensorFlow.js 加载运行，比如在移动设备上使用 TensorFlow Lite 加载运行。 137 | 138 | ### HDF5 139 | 140 | 直接调用`model.save`即可保存为 HDF5 格式的文件。 141 | 142 | ```python 143 | model.save('my_model.h5') 144 | ``` 145 | 146 | 从 HDF5 中恢复完整的模型。 147 | 148 | ```python 149 | new_model = models.load_model('my_model.h5') 150 | evaluate(new_model) 151 | ``` 152 | 153 | ```bash 154 | 1000/1000 [===] - 0s 90us/sample - loss: 0.4703 - accuracy: 0.8780 155 | Restore model, accuracy: 87.80% 156 | ``` 157 | 158 | ### saved_model 159 | 160 | 保存为`saved_model`格式。 161 | 162 | ```python 163 | import time 164 | saved_model_path = "./saved_models/{}".format(int(time.time())) 165 | tf.keras.experimental.export_saved_model(model, saved_model_path) 166 | ``` 167 | 168 | 恢复模型并预测 169 | 170 | ```python 171 | new_model = tf.keras.experimental.load_from_saved_model(saved_model_path) 172 | model.predict(test_x).shape 173 | ``` 174 | 175 | ```bash 176 | (1000, 10) 177 | ``` 178 | 179 | `saved_model`格式的模型可以直接用来预测(predict)，但是 saved_model 没有保存优化器配置，如果要使用`evaluate`方法，则需要先 compile。 180 | 181 | ```python 182 | new_model.compile(optimizer=model.optimizer, 183 | loss='sparse_categorical_crossentropy', 184 | metrics=['accuracy']) 185 | evaluate(new_model) 186 | ``` 187 | 188 | ```bash 189 | 1000/1000 [===] - 0s 90us/sample - loss: 0.4703 - accuracy: 0.8780 190 | Restore model, accuracy: 87.80% 191 | ``` 192 | 193 | ## 最后 194 | 195 | TensorFlow 中还有其他的方式可以保存模型。 196 | 197 | - [Saving in eager](https://www.tensorflow.org/guide/eager#object_based_saving) eager 模型保存模型 198 | - [Save and Restore](https://www.tensorflow.org/guide/saved_model) -- low-level 的接口。 199 | 200 | 返回[文档首页](https://geektutu.com/post/tf2doc.html) 201 | 202 | > 完整代码：[Github - save_restore_model.ipynb](https://github.com/geektutu/tensorflow2-docs-zh/tree/master/code) 203 | > 参考文档：[Save and restore models](https://www.tensorflow.org/beta/tutorials/keras/save_and_restore_models) 204 | 205 | ## 附 推荐 206 | 207 | - [一篇文章入门 Python](https://geektutu.com/post/quick-python.html) -------------------------------------------------------------------------------- /Beginner-ML-basics/tf2doc-ml-basic-save-model/hdf5.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/geektutu/tensorflow2-docs-zh/3418ce1c0e721f549d0cff7eeff65ca030c55f82/Beginner-ML-basics/tf2doc-ml-basic-save-model/hdf5.png -------------------------------------------------------------------------------- /Beginner-ML-basics/tf2doc-ml-basic-structured-data.md: -------------------------------------------------------------------------------- 1 | --- 2 | title: TensorFlow 2 中文文档 - 特征工程结构化数据分类 3 | date: 2019-07-09 23:45:10 4 | description: TensorFlow2文档，TensorFlow2.0文档，TensorFlow2.0 TF2.0 TensorFlow 2 / 2.0 官方文档中文版，结构化数据分类 Classify structured data，示例使用心脏病数据集。 5 | tags: 6 | - TensorFlow 2 7 | - 官方文档 8 | keywords: 9 | - TensorFlow2.0 10 | - TensorFlow2文档 11 | - TensorFlow2.0文档 12 | - Feature Column 13 | - structured data 14 | nav: 简明教程 15 | categories: 16 | - TensorFlow2 文档 17 | image: post/tf2doc-ml-basic-structured-data/structured-data.jpg 18 | github: https://github.com/geektutu/tensorflow2-docs-zh 19 | --- 20 | 21 | ![structured-data](tf2doc-ml-basic-structured-data/structured-data.jpg) 22 | 23 | 27 | 28 | **TF2.0 TensorFlow 2 / 2.0 中文文档 - 结构化数据分类 Classify structured data** 29 | 30 | 主要内容：介绍如何对结构化数据（例如 CSV 中的表格数据）分类。 31 | 32 | 这个教程包含完整的代码： 33 | 34 | - 使用 Pandas 加载 CSV 文件。 35 | - 使用 tf.data 打乱数据并获取batch。 36 | - 使用特征工程(feature columns)将 CSV 中的列映射为特征值(features) 37 | - 使用 Keras 搭建、训练和评估模型。 38 | 39 | ## 数据集 40 | 41 | 数据集由 Cleveland Clinic Foundation 提供的几百行心脏病数据构成。每一行代表一个病人，每一列描述一个属性，数据共14列，最后一列为是否患病。我们将使用这些信息预测一个病人是否有心脏病。这是一个典型的二分分类问题。 42 | 43 | | 列 | 描述 | 特征类型 | 数据类型 | 44 | | ------ | --------------------------------------------------- | -------- | -------- | 45 | | Age | 年龄 | 数值 | integer | 46 | | Sex | 性别(1男性； 0女性) | 类别 | integer | 47 | | ... | ... | ... | ... | 48 | | Thal | 3 = normal; 6 = fixed defect; 7 = reversable defect | 类别 | string | 49 | | Target | 是否感染，(1是；0否) | 分类 | integer | 50 | 51 | ## 导入库 52 | 53 | ```bash 54 | pip install -q sklearn 55 | pip install -q tensorflow==2.0.0-beta1 56 | ``` 57 | 58 | ```python 59 | import numpy as np 60 | import pandas as pd 61 | import tensorflow as tf 62 | 63 | from tensorflow import feature_column 64 | from tensorflow.keras import layers 65 | from sklearn.model_selection import train_test_split 66 | ``` 67 | 68 | ## 使用 Pandas 读取数据 69 | 70 | ```python 71 | URL = 'https://storage.googleapis.com/applied-dl/heart.csv' 72 | dataframe = pd.read_csv(URL) 73 | dataframe.head() 74 | ``` 75 | 76 | | - | age | sex | cp | … | slope | ca | thal | target | 77 | | ---- | ---- | ---- | ---- | ---- | ----- | ---- | ---------- | ------ | 78 | | 0 | 63 | 1 | 1 | | 3 | 0 | fixed | 0 | 79 | | 1 | 67 | 1 | 4 | | 2 | 3 | normal | 1 | 80 | | 2 | 67 | 1 | 4 | | 2 | 2 | reversible | 0 | 81 | | 3 | 37 | 1 | 3 | | 3 | 0 | normal | 0 | 82 | | 4 | 41 | 0 | 2 | | 1 | 0 | normal | 0 | 83 | 84 | ## 分割训练集、验证集和测试集 85 | 86 | ```python 87 | train, test = train_test_split(dataframe, test_size=0.2) 88 | train, val = train_test_split(train, test_size=0.2) 89 | print(len(train), 'train examples') # 193 90 | print(len(val), 'validation examples') # 49 91 | print(len(test), 'test examples') # 61 92 | ``` 93 | 94 | ## 创建 input pipeline 95 | 96 | 使用 tf.data ，我们可以使用特征工程(feature columns)将 Pandas DataFrame 中的列映射为特征值(features)。如果是一个非常大的 CSV 文件，不能直接放在内存中，就必须直接使用 tf.data 从磁盘中直接读取数据了。 97 | 98 | ```python 99 | # 帮助函数，返回 tf.data 数据集。 100 | def df_to_dataset(dataframe, shuffle=True, batch_size=32): 101 | dataframe = dataframe.copy() 102 | labels = dataframe.pop('target') 103 | ds = tf.data.Dataset.from_tensor_slices((dict(dataframe), labels)) 104 | if shuffle: 105 | ds = ds.shuffle(buffer_size=len(dataframe)) 106 | ds = ds.batch(batch_size) 107 | return ds 108 | ``` 109 | 110 | ```python 111 | batch_size = 5 # 为了方面展示下面的示例，batch暂时设置为5。 112 | train_ds = df_to_dataset(train, batch_size=batch_size) 113 | val_ds = df_to_dataset(val, shuffle=False, batch_size=batch_size) 114 | test_ds = df_to_dataset(test, shuffle=False, batch_size=batch_size) 115 | ``` 116 | 117 | ## 理解 input pipeline 118 | 119 | ```python 120 | for feature_batch, label_batch in train_ds.take(1): 121 | print('Every feature:', list(feature_batch.keys())) 122 | print('A batch of ages:', feature_batch['age']) 123 | print('A batch of targets:', label_batch ) 124 | ``` 125 | 126 | ```python 127 | Every feature: ['cp', 'age', 'sex', ... , 'slope', 'ca'] 128 | A batch of ages: tf.Tensor([50 62 37 69 58], shape=(5,), dtype=int32) 129 | A batch of targets: tf.Tensor([0 0 0 0 0], shape=(5,), dtype=int32) 130 | ``` 131 | 132 | 可以看到数据集返回了一个键为列名的字典。 133 | 134 | ## 特征列示例 135 | 136 | TensorFlow 提供了很多种类型的特征列(feature column)，接下来给几个例子看一看每一列的值是怎么被转换的。 137 | 138 | ```python 139 | example_batch = next(iter(train_ds))[0] 140 | 141 | # 帮助函数：创建一个特征列，并转换。 142 | def demo(feature_column): 143 | feature_layer = layers.DenseFeatures(feature_column) 144 | print(feature_layer(example_batch).numpy()) 145 | ``` 146 | 147 | ### 1) Numeric column 148 | 149 | 特征列的输出是模型的输入，Numeric columns 是最简单的类型，数值本身代表某个特征真实的值，因此转换后，值不发生改变。 150 | 151 | ```python 152 | age = feature_column.numeric_column("age") 153 | demo(age) 154 | ``` 155 | 156 | ```python 157 | [[50.] 158 | [62.] 159 | [37.] 160 | [69.] 161 | [58.]] 162 | ``` 163 | 164 | 在这个数据集中，大部分列都是数值类型。 165 | 166 | ### 2) Bucketized columns 167 | 168 | 有时候，并不想直接将数值传给模型，而是希望基于数值的范围离散成几个种类。比如人的年龄，0-10归为一类，用0表示；11-20归为一类，用1表示。我们可以用 bucketized column 将年龄划分到不同的 bucket 中。用中文比喻，就好像提供了不同的桶，在某一范围内的扔进A桶，另一范围的数据扔进B桶，以此类推。下面的例子使用独热编码来表示不同的 bucket。 169 | 170 | ```python 171 | age_buckets = feature_column.bucketized_column(age, boundaries=[18, 25, 30, 35, 40, 45, 50, 55, 60, 65]) 172 | demo(age_buckets) 173 | ``` 174 | 175 | ```python 176 | [[0. 0. 0. 0. 0. 0. 0. 1. 0. 0. 0.] 177 | [0. 0. 0. 0. 0. 0. 0. 0. 0. 1. 0.] 178 | [0. 0. 0. 0. 1. 0. 0. 0. 0. 0. 0.] 179 | [0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 1.] 180 | [0. 0. 0. 0. 0. 0. 0. 0. 1. 0. 0.]] 181 | ``` 182 | 183 | ### 3) Categorical columns 184 | 185 | 在这个数据集中，`thal`列使用字符串表示(e.g. 'fixed', 'normal', 'reversible')。字符串不能直接传给模型。所以我们要先将字符串映射为数值。可以使用categorical_column_with_vocabulary_list 和 categorical_column_with_vocabulary_file 来转换，前者接受一个列表作为输入，后者可以传入一个文件。 186 | 187 | ```python 188 | thal = feature_column.categorical_column_with_vocabulary_list( 189 | 'thal', ['fixed', 'normal', 'reversible']) 190 | 191 | thal_one_hot = feature_column.indicator_column(thal) 192 | demo(thal_one_hot) 193 | ``` 194 | 195 | ```python 196 | [[0. 1. 0.] 197 | [0. 1. 0.] 198 | [0. 1. 0.] 199 | [0. 1. 0.] 200 | [0. 0. 1.]] 201 | ``` 202 | 203 | 可以看到，最终的输出向量也是独热编码，和 Bucketized column 相似。 204 | 205 | ### 4) Embedding column 206 | 207 | 假设某一列有上千种类别，用独热编码来表示就不太合适了。这时候，可以使用 embedding column。embedding column 可以压缩维度，因此向量中的值不再只由0或1组成，可以包含任何数字。 208 | 209 | 在有很多种类别时使用 embedding column 是最合适的。接下来只是一个示例，不管输入有多少种可能性，最终的输出向量定长为8。 210 | 211 | ```python 212 | # embedding column的输入是categorical column 213 | thal_embedding = feature_column.embedding_column(thal, dimension=8) 214 | demo(thal_embedding) 215 | ``` 216 | 217 | ```python 218 | [[-0.42 -0.42 0.34 0.47 0.21 0.33 0.34 0.65] 219 | [-0.42 -0.42 0.34 0.47 0.21 0.33 0.34 0.65] 220 | [-0.42 -0.42 0.34 0.47 0.21 0.33 0.34 0.65] 221 | [-0.42 -0.42 0.34 0.47 0.21 0.33 0.34 0.65] 222 | [ 0.20 0.07 0.06 0.01 -0.47 -0.10 -0.70 0.00]] 223 | ``` 224 | 225 | ### 5) Hashed feature columns 226 | 227 | 另一种表示类别很多的 categorical column 的方式是使用 categorical_column_with_hash_bucket。这个特征列会计算输入的哈希值，然后根据哈希值对字符串进行编码。哈希桶(bucket)个数即参数`hash_bucket_size`。哈希桶(hash_buckets)的个数应明显小于实际的类别个数，以节省空间。 228 | 229 | 注意：哈希的一大副作用是可能存在冲突，不同的字符串可能映射到相同的哈希桶中。不过，在某些数据集，这个方式还是非常有效的。 230 | 231 | ```python 232 | thal_hashed = feature_column.categorical_column_with_hash_bucket( 233 | 'thal', hash_bucket_size=1000) 234 | demo(feature_column.indicator_column(thal_hashed)) 235 | ``` 236 | 237 | ```python 238 | [[0. 0. 0. ... 0. 0. 0.] 239 | [0. 0. 0. ... 0. 0. 0.] 240 | [0. 0. 0. ... 0. 0. 0.] 241 | [0. 0. 0. ... 0. 0. 0.] 242 | [0. 0. 0. ... 0. 0. 0.]] 243 | ``` 244 | 245 | ### 6) Crossed feature columns 246 | 247 | 将几个特征组合成一个特征，即 feature crosses，模型可以对每一个特征组合学习独立的权重。接下来，我们将组合 `age` 和 `thal` 列创建一个新的特征。注意：`crossed_column`不会创建所有可能的组合，因为组合可能性会非常多。背后是通过`hashed_column`处理的，可以设置哈希桶的大小。 248 | 249 | ```python 250 | crossed_feature = feature_column.crossed_column([age_buckets, thal], hash_bucket_size=1000) 251 | demo(feature_column.indicator_column(crossed_feature)) 252 | ``` 253 | 254 | ```python 255 | [[0. 0. 0. ... 0. 0. 0.] 256 | [0. 0. 0. ... 0. 0. 0.] 257 | [0. 0. 0. ... 0. 0. 0.] 258 | [0. 0. 0. ... 0. 0. 0.] 259 | [0. 0. 0. ... 0. 0. 0.]] 260 | ``` 261 | 262 | ## 选择需要使用的列 263 | 264 | 为了训练出准确率高的模型，大数据集、选取有意义的列、数据的展示方式都是非常重要的。 265 | 266 | 接下来的示例，我们随机选取一些列来训练。 267 | 268 | ```python 269 | feature_columns = [] 270 | 271 | # numeric cols 272 | for header in ['age', 'trestbps', 'chol', 'thalach', 'oldpeak', 'slope', 'ca']: 273 | feature_columns.append(feature_column.numeric_column(header)) 274 | 275 | # bucketized cols 276 | age_buckets = feature_column.bucketized_column(age, boundaries=[18, 25, 30, 35, 40, 45, 50, 55, 60, 65]) 277 | feature_columns.append(age_buckets) 278 | 279 | # indicator cols 280 | thal = feature_column.categorical_column_with_vocabulary_list( 281 | 'thal', ['fixed', 'normal', 'reversible']) 282 | thal_one_hot = feature_column.indicator_column(thal) 283 | feature_columns.append(thal_one_hot) 284 | 285 | # embedding cols 286 | thal_embedding = feature_column.embedding_column(thal, dimension=8) 287 | feature_columns.append(thal_embedding) 288 | 289 | # crossed cols 290 | crossed_feature = feature_column.crossed_column([age_buckets, thal], hash_bucket_size=1000) 291 | crossed_feature = feature_column.indicator_column(crossed_feature) 292 | feature_columns.append(crossed_feature) 293 | ``` 294 | 295 | ### 创建特征层 296 | 297 | 我已经定义好了特征列，接下来使用 DenseFeatures 层将特征列传入到模型中。 298 | 299 | ```python 300 | feature_layer = tf.keras.layers.DenseFeatures(feature_columns) 301 | ``` 302 | 303 | 之前 batch 大小设置为5，是为了方便示例。接下来batch设置为32，创建新的 input pipeline。 304 | 305 | ## 创建、编译和训练模型 306 | 307 | ```python 308 | model = tf.keras.Sequential([ 309 | feature_layer, 310 | layers.Dense(128, activation='relu'), 311 | layers.Dense(128, activation='relu'), 312 | layers.Dense(1, activation='sigmoid') 313 | ]) 314 | 315 | model.compile(optimizer='adam', 316 | loss='binary_crossentropy', 317 | metrics=['accuracy'], 318 | run_eagerly=True) 319 | 320 | model.fit(train_ds, 321 | validation_data=val_ds, 322 | epochs=5) 323 | ``` 324 | 325 | ```bash 326 | Epoch 1/5 327 | 7/7 [========] - 1s 142ms/step - loss: 1.3386 - accuracy: 0.6090 - val_loss: 1.0882 - val_accuracy: 0.2857 328 | Epoch 2/5 329 | 7/7 [========] - 0s 31ms/step - loss: 1.4225 - accuracy: 0.3849 - val_loss: 0.9518 - val_accuracy: 0.7347 330 | Epoch 3/5 331 | 7/7 [========] - 0s 32ms/step - loss: 0.6602 - accuracy: 0.7165 - val_loss: 0.7390 - val_accuracy: 0.6327 332 | Epoch 4/5 333 | 7/7 [========] - 0s 30ms/step - loss: 0.7332 - accuracy: 0.6310 - val_loss: 0.6794 - val_accuracy: 0.7143 334 | Epoch 5/5 335 | 7/7 [========] - 0s 31ms/step - loss: 0.5617 - accuracy: 0.7003 - val_loss: 0.5326 - val_accuracy: 0.7143 336 | 337 | 338 | ``` 339 | 340 | ```python 341 | loss, accuracy = model.evaluate(test_ds) 342 | print("Accuracy", accuracy) 343 | ``` 344 | 345 | ```bash 346 | 2/2 [========] - 0s 15ms/step - loss: 0.3907 - accuracy: 0.7869 347 | Accuracy 0.78688526 348 | ``` 349 | 350 | 如果你使用深度学习模型，以及更大、更复杂的数据集，准确率会更高。一般来说，像这样的小数据集，建议使用决策树或者随机森林。这个教程的主要目的不是训练一个准确率高的模型，而是作一个示例：TensorFlow 如何处理结构化的数据。 351 | 352 | 试一试吧。 353 | 354 | 返回[文档首页](https://geektutu.com/post/tf2doc.html) 355 | 356 | > 参考文档：[Classify structured data](https://www.tensorflow.org/beta/tutorials/keras/feature_columns) 357 | 358 | ## 附 推荐 359 | 360 | - [一篇文章入门 Python](https://geektutu.com/post/quick-python.html) -------------------------------------------------------------------------------- /Beginner-ML-basics/tf2doc-ml-basic-structured-data/structured-data.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/geektutu/tensorflow2-docs-zh/3418ce1c0e721f549d0cff7eeff65ca030c55f82/Beginner-ML-basics/tf2doc-ml-basic-structured-data/structured-data.jpg -------------------------------------------------------------------------------- /Beginner-ML-basics/tf2doc-ml-basic-text.md: -------------------------------------------------------------------------------- 1 | --- 2 | title: TensorFlow 2 中文文档 - IMDB 文本分类 3 | date: 2019-07-09 00:40:10 4 | description: TensorFlow2文档，TensorFlow2.0文档，TensorFlow2.0 TF2.0 TensorFlow 2 / 2.0 官方文档中文版，文本分类 Classify text，示例使用 IMDB 数据集。 5 | tags: 6 | - TensorFlow 2 7 | - 官方文档 8 | keywords: 9 | - Classify text 10 | - TensorFlow2.0 11 | - TensorFlow2文档 12 | - TensorFlow2.0文档 13 | - IMDB datasets 14 | - 文本分类 15 | nav: 简明教程 16 | categories: 17 | - TensorFlow2 文档 18 | image: post/tf2doc-ml-basic-text/imdb-sm.jpg 19 | github: https://github.com/geektutu/tensorflow2-docs-zh 20 | --- 21 | 22 | **TF2.0 TensorFlow 2 / 2.0 中文文档 - 文本分类 Classify text** 23 | 24 | 主要内容：使用迁移学习算法解决一个典型的二分分类(Binary Classification)问题——电影正向评论和负向评论分类。 25 | 26 | 这篇文档使用包含有50,000条电影评论的 IMDB 数据集，25,000用于训练，25,000用于测试。而且训练集和测试集是均衡的，即其中包含同等数量的正向评论和负向评论。 27 | 28 | 代码使用`tf.keras`和`TensorFlow Hub`，TensorFlow Hub 是一个用于迁移学习的平台/库。 29 | 30 | ```python 31 | import numpy as np 32 | import tensorflow as tf # 2.0.0-beta1 33 | import tensorflow_hub as hub # 0.5.0 34 | import tensorflow_datasets as tfds 35 | ``` 36 | 37 | ## 下载 IMDB 数据集 38 | 39 | ![IMDB datasets](tf2doc-ml-basic-text/imdb.jpg) 40 | 41 | IMDB 数据集在`tfds`中是可以直接获取的，调用时会自动下载到你的机器上。 42 | 43 | ```python 44 | # 进一步划分训练集。 45 | # 60%(15,000)用于训练，40%(10,000)用于验证(validation)。 46 | train_validation_split = tfds.Split.TRAIN.subsplit([6, 4]) 47 | 48 | (train_data, validation_data), test_data = tfds.load( 49 | name="imdb_reviews", 50 | split=(train_validation_split, tfds.Split.TEST), 51 | as_supervised=True) 52 | ``` 53 | 54 | ## 数据格式 55 | 56 | 每个例子包含一句电影评论和对应的标签，0或1。0代表负向评论，1代表正向评论。 57 | 58 | 看一下前十条数据。 59 | 60 | ```python 61 | train_examples_batch, train_labels_batch = next(iter(train_data.batch(10))) 62 | train_examples_batch 63 | ``` 64 | 65 | ```python 66 | 77 | ``` 78 | 79 | 前十个标签。 80 | 81 | ```python 82 | train_labels_batch 83 | # 84 | ``` 85 | 86 | ## 搭建模型 87 | 88 | 神经网络需要堆叠多层，架构上需要考虑三点。 89 | 90 | - 文本怎么表示？ 91 | - 模型需要多少层？ 92 | - 每一层多少个_隐藏节点_ 93 | 94 | 一种表示文本的方式是将句子映射为向量(embeddings vectors)，或者称为文本嵌入(text embedding)。嵌入方法很多，比如我们可以采用最简单的独热编码，假设常用单词总共1000个，给每一个单词一个独热编码。假设每句话由10个单词构成，那么每句话均可以映射到10x1000的二维空间中。那么某句话就可以表示为： 95 | 96 | ```python 97 | # 10x1000的二维向量表示一句话 98 | [[0, 0, 0, 1, 0, ... 0], 99 | [0, 0, ..., 0, 1,... 0], 100 | [0, 0, 0, 0, 0, ... 1], 101 | ... 102 | [0, 0, 0, 0, 0, ... 1]] 103 | ``` 104 | 105 | 文本嵌入的方法很多，要考虑的因素也很多，比如同义词如何处理，维度过高怎么办？知乎上有比较详细的回答：[word embedding的解释](https://www.zhihu.com/question/32275069)。 106 | 107 | 我们可以使用一个预训练(pre-trained)好的文本嵌入模型作为第一层，有3个好处。 108 | 109 | - 不用担心文本处理。 110 | - 能从迁移学习中受益。 111 | - 嵌入后size固定，处理起来简单。 112 | 113 | 接下来从 TensorFlow Hub 中选用的**pre-trained 文本嵌入模型**称为[google/tf2-preview/gnews-swivel-20dim/1](https://tfhub.dev/google/tf2-preview/gnews-swivel-20dim/1)。 114 | 115 | 接下来创建一个 Keras Layer 使用这个模型将句子转为向量。取前三条评论试一试。注意无论句子的长度如何，最终的嵌入结果均为长度20的一维向量。 116 | 117 | ```python 118 | embedding = "https://tfhub.dev/google/tf2-preview/gnews-swivel-20dim/1" 119 | hub_layer = hub.KerasLayer(embedding, input_shape=[], 120 | dtype=tf.string, trainable=True) 121 | hub_layer(train_examples_batch[:3]) 122 | ``` 123 | 124 | ```bash 125 | 130 | ``` 131 | 132 | 接下来，搭建完整的神经网络模型。 133 | 134 | ```python 135 | model = tf.keras.Sequential() 136 | model.add(hub_layer) 137 | model.add(tf.keras.layers.Dense(16, activation='relu')) 138 | model.add(tf.keras.layers.Dense(1, activation='sigmoid')) 139 | 140 | model.summary() 141 | ``` 142 | 143 | ```bash 144 | Model: "sequential" 145 | _________________________________________________________________ 146 | Layer (type) Output Shape Param 147 | ================================================================= 148 | keras_layer (KerasLayer) (None, 20) 400020 149 | _________________________________________________________________ 150 | dense (Dense) (None, 16) 336 151 | _________________________________________________________________ 152 | dense_1 (Dense) (None, 1) 17 153 | ================================================================= 154 | Total params: 400,373 155 | Trainable params: 400,373 156 | Non-trainable params: 0 157 | _________________________________________________________________ 158 | ``` 159 | 160 | 1. 第一层是 TensorFlow Hub 层，将句子转换为 tokens，然后映射每个 token，并组合成最终的向量。输出的维度是：句子个数 * 嵌入维度(20)。 161 | 2. 接下来是全连接层(Full-connected, FC)，即`Dense`层，16个节点。 162 | 3. 最后一层，也是全连接层，只有一个节点。使用`sigmoid`激活函数，输出值是float，范围0-1，代表可能性/置信度。 163 | 164 | ## 损失函数和优化器 165 | 166 | `binary_crossentropy`更适合处理概率问题，`mean_squared_error`适合处理回归(Regression)问题。 167 | 168 | ```python 169 | model.compile(optimizer='adam', 170 | loss='binary_crossentropy', 171 | metrics=['accuracy']) 172 | ``` 173 | 174 | ## 训练模型 175 | 176 | 共 20 epochs，每个batch 512个数据。即对所有训练数据进行20轮迭代。在训练过程中，将监视模型在包含10,000条数据的验证集上的损失(loss)和正确率(accuracy)。 177 | 178 | ```python 179 | history = model.fit(train_data.shuffle(10000).batch(512), 180 | epochs=20, 181 | validation_data=validation_data.batch(512), 182 | verbose=1) 183 | ``` 184 | 185 | ```bash 186 | Epoch 1/20 187 | 30/30 [========] - 6s 190ms/step - loss: 1.0201 - accuracy: 0.4331 - val_loss: 0.0000e+00 - val_accuracy: 0.0000e+00 188 | Epoch 2/20 189 | 30/30 [========] - 5s 159ms/step - loss: 0.7801 - accuracy: 0.4677 - val_loss: 0.7407 - val_accuracy: 0.5009 190 | ...... 191 | Epoch 20/20 192 | 30/30 [========] - 5s 152ms/step - loss: 0.1917 - accuracy: 0.9348 - val_loss: 0.2930 - val_accuracy: 0.8784 193 | ``` 194 | 195 | ## 评估模型 196 | 197 | `evaluate`返回2个值，Loss(误差，越小越好) 和 accuracy。 198 | 199 | ```python 200 | results = model.evaluate(test_data.batch(512), verbose=0) 201 | for name, value in zip(model.metrics_names, results): 202 | print("%s: %.3f" % (name, value)) 203 | # loss: 0.314 204 | # accuracy: 0.866 205 | ``` 206 | 207 | 这个非常基础的模型达到了87%的正确率，复杂一点的模型可以达到95%。 208 | 209 | 返回[文档首页](https://geektutu.com/post/tf2doc.html) 210 | 211 | > 参考地址：[Text classification of movie reviews with Keras and TensorFlow Hub](https://www.tensorflow.org/beta/tutorials/keras/basic_text_classification_with_tfhub) 212 | 213 | ## 附 推荐 214 | 215 | - [一篇文章入门 Python](https://geektutu.com/post/quick-python.html) -------------------------------------------------------------------------------- /Beginner-ML-basics/tf2doc-ml-basic-text/imdb-sm.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/geektutu/tensorflow2-docs-zh/3418ce1c0e721f549d0cff7eeff65ca030c55f82/Beginner-ML-basics/tf2doc-ml-basic-text/imdb-sm.jpg -------------------------------------------------------------------------------- /Beginner-ML-basics/tf2doc-ml-basic-text/imdb.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/geektutu/tensorflow2-docs-zh/3418ce1c0e721f549d0cff7eeff65ca030c55f82/Beginner-ML-basics/tf2doc-ml-basic-text/imdb.jpg -------------------------------------------------------------------------------- /Beginner-Text-and-sequences/tf2doc-rnn-lstm-text.md: -------------------------------------------------------------------------------- 1 | --- 2 | title: TensorFlow 2 中文文档 - RNN LSTM 文本分类 3 | date: 2019-07-22 23:18:10 4 | description: TensorFlow2文档，TensorFlow2.0文档，TensorFlow2.0 TF2.0 TensorFlow 2 / 2.0 官方文档中文版，循环神经网络(Recurrent Neural Network, RNN) 和 长短期记忆模型(Long Short-Term Memory，LSTM) 分类 IMDB 。 5 | tags: 6 | - TensorFlow 2 7 | - 官方文档 8 | keywords: 9 | - TensorFlow2.0 10 | - TensorFlow2文档 11 | - TensorFlow2.0文档 12 | nav: 简明教程 13 | categories: 14 | - TensorFlow2 文档 15 | image: post/tf2doc-rnn-lstm-text/rnn_small.jpg 16 | github: https://github.com/geektutu/tensorflow2-docs-zh 17 | --- 18 | 19 | **TF2.0 TensorFlow 2 / 2.0 中文文档：RNN LSTM 文本分类 Text classification with an RNN** 20 | 21 | 主要内容：使用循环神经网络(Recurrent Neural Network, RNN) 分类 影评数据 IMDB 22 | 23 | 循环神经网络(Recurrent Neural Network, RNN)广泛适用于自然语言处理领域(Natural Language Processing, NLP)，RNN有什么显著的特点呢？普通的神经网络，每一层的输出是下一层的输入，每一层之间是相互独立的，没有关系。但是对于语言来说，一句话中的单词顺序不同，整个语义就完全变了。因此自然语言处理往往需要能够更好地处理序列信息的神经网络，RNN 能够满足这个需求。 24 | 25 | RNN 中，隐藏层的状态，不仅取决于当前输入层的输出，还和上一步隐藏层的状态有关。 26 | 27 | ![RNN](tf2doc-rnn-lstm-text/rnn.jpg) 28 | 29 | 长短期记忆模型(Long short-term memory, LSTM)是一种特殊的RNN，主要是为了解决长序列训练过程中的梯度消失和梯度爆炸问题。简单来说，就是相比普通的RNN，LSTM能够在更长的序列中有更好的表现。 30 | 31 | 接下来我们使用`tf.keras`提供的 LSTM 网络层搭建 RNN 网络模型，对 IMDB 影评数据进行分类。 32 | 33 | ## 下载 IMDB 34 | 35 | ```python 36 | # geektutu.com 37 | import matplotlib.pyplot as plt 38 | import tensorflow_datasets as tfds 39 | import tensorflow as tf 40 | from tensorflow.keras import Sequential, layers 41 | 42 | ds, info = tfds.load('imdb_reviews/subwords8k', with_info=True, 43 | as_supervised=True) 44 | train_ds, test_ds = ds['train'], ds['test'] 45 | 46 | BUFFER_SIZE, BATCH_SIZE = 10000, 64 47 | train_ds = train_ds.shuffle(BUFFER_SIZE) 48 | train_ds = train_ds.padded_batch(BATCH_SIZE, train_ds.output_shapes) 49 | test_ds = test_ds.padded_batch(BATCH_SIZE, test_ds.output_shapes) 50 | ``` 51 | 52 | ## 文本预处理 53 | 54 | 通过 tfds 获取到的数据已经经过了文本预处理，即 Tokenizer，向量化文本(将文本转为数字序列)。 55 | 56 | 接下来我们看一看是如何转换的。 57 | 58 | ```python 59 | # geektutu.com 60 | tokenizer = info.features['text'].encoder 61 | print ('词汇个数:', tokenizer.vocab_size) 62 | 63 | sample_str = 'welcome to geektutu.com' 64 | tokenized_str = tokenizer.encode(sample_str) 65 | print ('向量化文本:', tokenized_str) 66 | 67 | for ts in tokenized_str: 68 | print (ts, '-->', tokenizer.decode([ts])) 69 | ``` 70 | 71 | 可以看到，有些单词被拆分了。因为 tokenizer 中不可能包含所有可能出现的单词，如果在 tokenizer 中没有的单词，就会被拆分。文本预处理有很多种方式，比如我们在[TensorFlow 2 中文文档 - IMDB 文本分类](https://geektutu.com/post/tf2doc-ml-basic-text.html)中使用了预训练好的字词嵌入模型 _google/tf2-preview/gnews-swivel-20dim/1_，来直接将影评文本转换为向量；还有非常出名的自然语言处理工具包 ntlk 在文本预处理环节提供了非常强大的功能。 72 | 73 | ```bash 74 | 词汇个数: 8185 75 | 向量化文本: [6351, 7961, 7, 703, 3108, 999, 999, 7975, 2449] 76 | 6351 --> welcome 77 | 7961 --> 78 | 7 --> to 79 | 703 --> ge 80 | 3108 --> ek 81 | 999 --> tu 82 | 999 --> tu 83 | 7975 --> . 84 | 2449 --> com 85 | ``` 86 | 87 | ## 搭建 RNN 模型 88 | 89 | 借助`tf.keras`，我们可以非常方便地搭建出 LSTM 网络层，在这里我们只使用一层来试一试。 90 | 91 | 如果你比较细心地话，会发现在第一层使用了`tf.keras.layers.Embedding`，那为什么要使用这一层呢？从我们刚才的预处理实验你会发现，IMDB 数据集的预处理是按照单词在 tokenizer 中的下标来处理的，维度(tokenizer.vocab_size)很高也很稀疏，经过 Embedding 层的转换，将产生大小固定为64的向量。而且这个转换是可训练的，经过足够的训练之后，相似语义的句子将产生相似的向量。 92 | 93 | 我们在 LSTM 层外面套了一个壳(层封装器, layer wrappers): `tf.keras.layers.Bidirectional`，这是 RNN 的双向封装器，用于对序列进行前向和后向计算。 94 | 95 | ```python 96 | # geektutu.com 97 | model = Sequential([ 98 | layers.Embedding(tokenizer.vocab_size, 64), 99 | layers.Bidirectional(layers.LSTM(64)), 100 | layers.Dense(64, activation='relu'), 101 | layers.Dense(1, activation='sigmoid') 102 | ]) 103 | model.compile(loss='binary_crossentropy', optimizer='adam', 104 | metrics=['accuracy']) 105 | history1 = model.fit(train_ds, epochs=3, validation_data=test_ds) 106 | loss, acc = model.evaluate(test_ds) 107 | print('准确率:', acc) # 0.81039 108 | ``` 109 | 110 | 最终达到了81%的准确率，我们使用 _matplotlib_ 把训练过程可视化吧。 111 | 112 | ```python 113 | # geektutu.com 114 | # 解决中文乱码问题 115 | plt.rcParams['font.sans-serif'] = ['SimHei'] 116 | plt.rcParams['axes.unicode_minus'] = False 117 | plt.rcParams['font.size'] = 20 118 | 119 | def plot_graphs(history, name): 120 | plt.plot(history.history[name]) 121 | plt.plot(history.history['验证集 - '+ name]) 122 | plt.xlabel("Epochs") 123 | plt.ylabel(name) 124 | plt.legend([name, '验证集 - ' + name]) 125 | plt.show() 126 | 127 | plot_graphs(history1, 'accuracy') 128 | ``` 129 | 130 | ![rnn acc1](tf2doc-rnn-lstm-text/acc1.jpg) 131 | 132 | ## 添加更多 LSTM 层 133 | 134 | ```python 135 | # geektutu.com 136 | model = Sequential([ 137 | layers.Embedding(tokenizer.vocab_size, 64), 138 | layers.Bidirectional(layers.LSTM(64, return_sequences=True)), 139 | layers.Bidirectional(layers.LSTM(32)), 140 | layers.Dense(64, activation='relu'), 141 | layers.Dense(1, activation='sigmoid') 142 | ]) 143 | model.compile(loss='binary_crossentropy', optimizer='adam', 144 | metrics=['accuracy']) 145 | history = model.fit(train_ds, epochs=3, validation_data=test_ds) 146 | loss, acc = model.evaluate(test_ds) 147 | print('准确率:', acc) # 0.83096% 148 | ``` 149 | 150 | 这一次，我们使用了2层 LSTM，正确率达到了83%。 151 | 152 | ```python 153 | # geektutu.com 154 | plot_graphs(history, 'accuracy') 155 | ``` 156 | 157 | ![rnn acc2](tf2doc-rnn-lstm-text/acc2.jpg) 158 | 159 | 返回[文档首页](https://geektutu.com/post/tf2doc.html) 160 | 161 | > 完整代码：[Github - rnn-text.ipynb](https://github.com/geektutu/tensorflow2-docs-zh/tree/master/code) 162 | > 参考文档：[Text classification with an RNN](https://www.tensorflow.org/beta/tutorials/text/text_classification_rnn) 163 | 164 | ## 附 推荐 165 | 166 | - [一篇文章入门 Python](https://geektutu.com/post/quick-python.html) -------------------------------------------------------------------------------- /Beginner-Text-and-sequences/tf2doc-rnn-lstm-text/acc1.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/geektutu/tensorflow2-docs-zh/3418ce1c0e721f549d0cff7eeff65ca030c55f82/Beginner-Text-and-sequences/tf2doc-rnn-lstm-text/acc1.jpg -------------------------------------------------------------------------------- /Beginner-Text-and-sequences/tf2doc-rnn-lstm-text/acc2.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/geektutu/tensorflow2-docs-zh/3418ce1c0e721f549d0cff7eeff65ca030c55f82/Beginner-Text-and-sequences/tf2doc-rnn-lstm-text/acc2.jpg -------------------------------------------------------------------------------- /Beginner-Text-and-sequences/tf2doc-rnn-lstm-text/rnn.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/geektutu/tensorflow2-docs-zh/3418ce1c0e721f549d0cff7eeff65ca030c55f82/Beginner-Text-and-sequences/tf2doc-rnn-lstm-text/rnn.jpg -------------------------------------------------------------------------------- /Beginner-Text-and-sequences/tf2doc-rnn-lstm-text/rnn_small.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/geektutu/tensorflow2-docs-zh/3418ce1c0e721f549d0cff7eeff65ca030c55f82/Beginner-Text-and-sequences/tf2doc-rnn-lstm-text/rnn_small.jpg -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # TensorFlow 2 / 2.0 官方文档中文版 2 | 3 | ![TensorFlow 2.0](tf2doc/tf.jpg) 4 | 5 | ## 相关链接 6 | 7 | - [一篇文章入门 Python](https://geektutu.com/post/quick-python.html) 8 | - [机器学习笔试面试题](https://geektutu.com/post/qa-ml.html)，[Github](https://github.com/geektutu/interview-questions) 9 | - [TensorFlow 2.0 中文文档](https://geektutu.com/post/tf2doc.html)，[Github](https://github.com/geektutu/tensorflow2-docs-zh) 10 | - [TensorFlow 2.0 图像识别&强化学习实战](https://geektutu.com/post/tensorflow2-mnist-cnn.html)，[Github](https://github.com/geektutu/tensorflow-tutorial-samples) 11 | ## 目录(持续更新) 12 | 13 | ### 基础 - 机器学习基础 ML basics 14 | 15 | 1. [图像分类 Classify images](https://geektutu.com/post/tf2doc-ml-basic-image.html) 16 | 2. [文本分类 Classify text](https://geektutu.com/post/tf2doc-ml-basic-text.html) 17 | 3. [结构化数据分类 Classify structured data](https://geektutu.com/post/tf2doc-ml-basic-structured-data.html) 18 | 4. [回归 Regression](https://geektutu.com/post/tf2doc-ml-basic-regression.html) 19 | 5. [过拟合与欠拟合 Overfitting and underfitting](https://geektutu.com/post/tf2doc-ml-basic-overfit.html) 20 | 6. [保存和恢复模型 Save and restore models](https://geektutu.com/post/tf2doc-ml-basic-save-model.html) 21 | 22 | ### 基础 - 图像分类 23 | 24 | 1. [卷积神经网络 Convolutional Neural Networks](https://geektutu.com/post/tf2doc-cnn-cifar10.html) 25 | 2. [使用TFHub进行迁移学习 TensorFlow Hub with Keras](https://geektutu.com/post/tf2doc-tfhub-image-tl.html) 26 | 3. 使用预训练CNN进行迁移学习 Transfer Learning Using Pretrained ConvNets 27 | 28 | ### 基础 - 文本分类 29 | 30 | 1. [使用RNN对文本分类进行分类 Text classification with an RNN](https://geektutu.com/post/tf2doc-rnn-lstm-text.html) 31 | 32 | ### 进阶 - 自定义 33 | 34 | 1. 张量和操作 Tensors and operations 35 | 2. 自定义层 Custom layers 36 | 3. 自动微分 Automatic differentiation 37 | 4. 自定义训练：攻略 Custom training：walkthrough 38 | 5. 动态图机制 TF function and AutoGraph -------------------------------------------------------------------------------- /code/cnn-cifar-10.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "## 下载数据集" 8 | ] 9 | }, 10 | { 11 | "cell_type": "code", 12 | "execution_count": 46, 13 | "metadata": {}, 14 | "outputs": [], 15 | "source": [ 16 | "# geektutu.com\n", 17 | "import matplotlib.pyplot as plt\n", 18 | "import tensorflow as tf\n", 19 | "from tensorflow.keras import layers, datasets, models\n", 20 | "\n", 21 | "(train_x, train_y), (test_x, test_y) = datasets.cifar10.load_data()" 22 | ] 23 | }, 24 | { 25 | "cell_type": "code", 26 | "execution_count": 47, 27 | "metadata": {}, 28 | "outputs": [ 29 | { 30 | "data": { 31 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAASUAAADFCAYAAAASVORBAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nOy9eZRk+VXf+fm9LfaI3DMrs7LWri5Vt3rRLnULLTQgwGIfPNjGBuOxxzYeD8OZ4+HYHi8H+9gHz2AbvDCYzcbYeDAghIRAC2hBLSG66b27uqq6tqyq3DMjY3/rb/64N15mN1VZWSBG1Zy453RXZLwX7/3W+7vr9xprLSMa0YhGdLeQ85VuwIhGNKIR7aURUxrRiEZ0V9GIKY1oRCO6q2jElEY0ohHdVTRiSiMa0YjuKhoxpRGNaER3FY2Y0h4yxvycMeYnvtLtGNH+ZIyZM8b8eWPMg1/ptvxJkDHmmDHGGmPmvtJt+UrQiCkdkIwxx40xj97B/XPGmMf+JNv05SZjzIQx5t8aY64ZYzrGmHVjzGeNMW/Q69YY8079/I+MMbExprnnv2/f8yzHGHPVGPORm7znsjGma4zpG2OWjTH/4A7a+M+BzwM/Ddx3k+v/mzFmxRizYYz5x/s8533an73t/9GDtmNEf3LkfaUb8Dqi7wHeCXz9Ae//euCHgDf8ibXoy0jGmBLwGWAJeI+19qIxZgp4PzC4xc8+Za291Xh8HbK+PmCMmbPWrrzm+l+x1v6iMu6PG2N+01r7pQM09YettT9kjLl8kz58HfC/Am8H+sAnjTHnrLW/cItnhdbasQO8c0T/P9LrVlIyxrzJGPOMnoj/1hjzijHme/dc/9vGmEt6Wn/3nu8dY8w/NcZcN8acN8Z8zS2e/wY9xR80xnwAYTCP6Yn6F/WeXHLQvwd6Ap8B/h1wSu//e3r9sjHmu/bcf3Zvm1/z/rox5v/Rdq4bY35Wvw+MMf/CGLNkjLlhjPkJY0xFrw3F/m81xjxrjNkxxvxNY8y36PisGWO+4RZD+j3ADPCd1tqLANbaDWvtL1lrL+8/Gzel7wP+D+BZ4LtvdZO19lNAC5g6yEOttd19Lv8vwL+31l611q4DP4owqQOTjvsVY8zf2vPdh4wxP6Cf32CM+bQxZtMYs2qM+Ut77vu0MeaHjTG/qGP/Wzonn9R18B/23PtzxpifNcb8vF57YiiR3qRNRV0LqzqvD99Jn153ZK193f0HuMDLwP+JnMbfhpyM36vXPwicRxb6/UAHmNVrfwsR/8vIab4KFPTazwE/AdSBs8Bf2PPOfwT85mvaYYF37vl7ALxPP38vcPY1918GvmvP32eHbb5JH38O+D1gDjDAgn7/D4EvABNAFfgN4N/otWPapo9p3/8q0AP+o977T4DP3eJ9/w34j7cZ97y/NxuPPfdNANeAAvCXgeduNg46zn9f/y7f4Rp41Vjqd0vAN+75+y06J+Ymv38fMLjFs9+va+a0rq0nAVevnQLeqnPyg8DSnt99GlgBvgZoAF3geeAh4B4dv5N75ncZeAwRDv71cG72zOOc/v1/Ab+ErPW/Bjz9ld6Df5L/vV4lpXcAi8C/sNYm1tpfBa7suf49wH+1ctK/gDCwx/Zc+2lrbc9a+3Fkot+y57cG+Hngt+ytxf4/UTLGuMCfB/6utXbFCl3Xy38R+FfW2i1rbQf4EaRPe+n7rbUbCGP2gP9Z773ArSWSSWST3AkNJcemMeZje77/buDnrbUh8F+BQ8aYt77mtz8DbGrbH7HW9u7w3TejGUTqGlILYYyNW9xf2GtTGn5prf0d4CeB/wz8K2T8Ur123lr7hBVu8Qf84fH8SWvtJ621O8A68KPW2mestRf0+t77f81a+ylrbYbYyB41xhRv0s7vQSTABBnPh4wx87cbjNcrvV6Z0iKwYq29la1jAfjf9yy2M8D0nms/tudaGVnMQ3oL8E3I5H+laBLwgVducm2BVzOPZaBqjKnv+W7vBs9eM07mFu/cAu50oX/KWjum/+1VC78P+EFjzABoAmOI5Mhr7nkzcAQ4kTfOmB96jfG5aYz5xQO2Zw2RvobUAEJg5xb3h3va/1rb0r9G1sLz1ton9rRvUlWpi8CH+cPj+Vrm+tq/bzX+Tb22t/0YYwKEkX1I1+uS9mn2Fs953dPrlSlt8ZrJQ/oyhDy4hJxQwwVXttb+6z3XfmDPtZK19kN7nvME8I+B/26MObTne8vNF5QD4m3j1Y6Dm91v99xfRtScm9E6spjvucm1a8Deds0DbWtt6yb33gl9BvgGY8ytpIoDkTHmLchmL1hri9baIvAo8Od0g+VkrX0O+DHgp4YSgrX2n+9lFPrfd/2hF92cngIe2PP3g8AzKtXcKf1L4D8B7zfGvGPP9z+JjPmjwDf/EZ57K1pATBDre7+01kbADeDb94xH0Vr71Jfx3XcVvV6Z0hcA1xjz7cYY1xjzd5ATN9Lr/xb4G0Zd+MaYmjFmQa/9OPB3jTFv1GtTxpi9khLW2n8M/C7wa3vE6XXgDUOjstIq8F4j8TL/8TVtXAcOm1fHmqwiIvq9wC8AuahujPk2NXwv6Cb6SeCfGWPm1Th/v976n4AfMOK+rwF/5ybv/qPQzyHq1IeMMWeM0IQx5ntvZYC9BX0f8Kuv+e5LyOl+s038j4ASYiv749KPAX/dGLOoc/qDiMRzR2SM+QvAYeCvAD8M/LweIgAngd8HNhCTQPDHYORfbYx5SJn1DyAmh5sx0B9H1sJRbd/8ayTjP1X0umRKah/584gBcBkxfN9AJCistY8jBtZ/pyLvU4gdCmvtLwL/DJGEtoHPcnOJ5C/rvz+r//4XxG51zRjzLfrdDyFM4ZcQI3Ky5/cfBz4BvGyM+X797h8C3wF8CrFX3NhzfwMxrPp7nv1F4Gnt4w/q9/8cYZjPIrayK3rvH4useLXeg9idfhuxxzyNGH1vpf68ipSB/zlgr+SJbrQP84dVuOF7/zaibr/pAO/4MZ3TI8BPq3o3oc/6JMKEngBeBH7ZWvtf9nncq2xKxpgv6SHyI8D3qB3pR5AD5v/W3/wD4G8C5/S/zwC/crt234Je3PP8McRbeTP6EYTR/44xZgv4KK82OfypIvNHk2y/8mSMWQSuW2szY8yfQWxAR62121/hpo1oRLclY8zPId6/v/6VbsvdRq/n4Ml/AnzQGGMQg/C3jhjSiEb0+qfXraQ0ohGN6E8nvS5tSiMa0Yj+9NKIKY1oRCO6q2jElEY0ohHdVTRiSiMa0YjuKhoxpRGNaER3FY2Y0ohGNKK7ikZMaUQjGtFdRSOmNKIRjeiuohFTGtGIRnRX0YgpjWhEI7qraN/ct3e/930WoNncouBkAEwEliOTguIwPVFhaqwKQOD6eIWS/ND12NoWIL8osYyPCbKDk8aEYQjAYDCgWBLkjpQUgF6/Q2NMERlsShQKEomLj+u6ANSqVSoVQQ/x/SJ9vccaBxzpThRGJFagjL7/h3/iVqBaOf2HD3/SAlw7+yTrl16SNqUes0cEsePIyTOMzx0BoFjyOPfC4wBcufAscbsjbUw96uPST69Y5u2PvgeAe+59A4OdLQBeeF4gcLIsIooFd+3FF56j1dwAIIxC4kj6ubXZo9OTe5I0ZHpaoJfGJ6qkti3fxzDoS5rQh37lt27bTxRvKsuyA9x6QNIsJWMM/a7gmW1uSX8mJsZJI+lDqVzGDQryE+OQKdSUe4evcxxn334uzlcsQKlUQtIiwXNcHEfO3yRLQb9v7rQoOgLxVHE82mFf3lEuUCro95UKjYbgv21vbxF1w73dJo7iHDXL9VwCX97TqBQ5ND0OwPXVVbqRrPF6fZwkll93uzscXpD17vsenifr9//99advO5e/9NEv5HNZKsi4BsUimSufE+vg6ei6KfjDKbcW60kbY2PzfjipBSsAFUlsSR1p715EsGFKmrU2/z7LLKn+Yffck2UZaZru/lb/TaxFgDbh+775/pv2c1+m9MKLLwDQ3NhgQpF/zGSRqbQmn0szdDPZcJ3UYhXDqzeI6PVl8uI0Y8OVdxc9S5JIg1zHo6CD2RsIFnySRZjBpAySC7EysJJXpKPMZytNKJeFKRnHx7iK9OE49AaxPCeOcb3Cfl17FbW2pQ+TYxPYaQH0s16dQ0cEEDHNYpxMNlzWSxhsb8o9/QELU4IgcWTxHhbvOQrA/MJhZmbkOb5fIBkTJr54WKCVkiRiMJAN0NzusLEh7/eCIhhZSOOTBYoVuWentU2hKFOV2QRf+9baaRKFd567ONygX24Ke4JwsnXtIgBLL+2w05K5ffSrH6NeGsJHORhdyF/ulvh6eKVJTJbKWjNBQJgIqozruTlTGquVqesBF7W7ZH1ZY2W/RKMsB2y5VKQayBrb6IdkVtZksShzMD09xfa25IEXS0XmD8l6cLHMzMhB4peKXFoSlJrAN4yNyTurFZhsyEFmMHR7+9VEeDVlup29gk+Uyebv7rTxK8rs/RLowZxhSIysk3QQM9iRdRUUC6TIGHX6HRxT0HY1sPp9pozFGJMzFmst+jiyzOYHjLW7B561NmdKxhgy/XVm7W0PxX2ZUsnTnhfg6KQsqGOzDWb01C6VK/lp1A8HDGI9RYwhKKnUlFhsJt83Jsr5KRH4JYaMdHiChtGAOJHnlYMCXkWeUQwKJEYmzLEZyfCUNVCtyIbvdHvEiTAlx0C7dSAIIKFYfheFMb2eLMxj9y7Q6co7o3jAxJRKQb7DqVP3AvDIO9/Kwuxh6VtjmtiTDpWLBbyhBJEk9LsiTYX6nnKpzPiYLN6TJ+7jpZde1ptjwlCYX6M+jq84jTutVazi12WZZXtb2tXvhfxR8qm/nEnYw2c5xrKydAmAZ7/wWQDifg+/KtJCv7VDfULWTYYRyZbdE/SgNFxvt6JApQBjHMan5IDr9nv4qTCrJEkw2uZDczPMTcs9ly68wpQnczw3P4eTONovkzPTyUYN68qabCgzKVfKuI4wvOnZKYrKwNqtHRIr890Ya7CQyDtdDzxfPhfcAtlQgqrVsfHBJdiWrqk4jtlYl0Py2vU13KIyvNo4BWcomUKkeyOLE3oq3Zf8AqgG1I7aRJGM7Ynjp7jnpBywpaL0PcuyXWZiwOoezIzNJ3EI/P9aMsbgDO/n9n0c2ZRGNKIR3VW0r6RUNHIC1Goe9y7IiTdZcvEzsRN0tiLSTPhav5eg6jn1sSqeSj/NnTaqKjNRK9NWcT4adOmrujXkutVKhThSvT718FW9S9MYT1XAMIwJVIRwsoSwoxBKqaWgBooky9hR3f8glKgqZZKUQiAn4c7GBpNzIgUduf8eZhYFU9/3AzHmAHEy4OyynFK9i+vEjkgzLz/3DG87I8Vb3/P2t+WnR0ult6tXbhD4cgIFQZ2paUHqvbp0nqCokl+/S6slthnPN9Tr8n2/3yNVfMskySgUXgV7fSC6nbRxJzQU8+Owx40lKShTH6o+YzXWtsX+tbl8ndlFscvhuLmEZPY3Ed0xNepiWiiWiszMiDS6trlJUdfSznaT2SmpIVEouJRKItksLM7ltso4SgiQcS0EBXp9WR+L8zNYNc4EOu5RFDE1qVK0kxGGsr5r9TJ9tVG1d7YJQ5GIJqdqlCqyITyT4kXynEG3TxLGB+7n41/8AgCdbgdHwUr7oWWQynr0g01c3ZupgYFN9LOlEsjaKxmPom6a1InoduX9Tzz7FGsbom6eOH4cgKmpKUplWYM221XNMpth9D23EtttlmHNUMX7Y6pv4wW5XCoUaKgqNV33SVWHTVEdHcBxCDPplOd5eGrMSsM+1pVGr601SWP5bbvXo5fKJq6W1Lgdpri6yB1jcQsyeP3ugLIv93jWMhjI7/pxkuuqzc6AZk/e3+klDOKDC4Gh6vLVUpH6hCzYNz/0MIsnTklbk4SXLy4B0Or16DTFiL/Z3GR5RZhivTENjjDCj/y3X8b/s/L+977r3fi+tGtuTouF2A2auln/4Kln8XzZMJVanSSV/kSdJjpsTE9PkOpYbW5t4FDOx3ls7I+F8//HImstjhoX1rc2uXz5KgDhlmyMWjGg15F6BmefeYq5YycBGJtbyBewtV9eJjmlKluWZUQDOTxn52YoF2X9FlyXQ9Myx3HcY3NjTdpar+GpkTqLMnw1XTiOpd/TmgwGnKKs91APzzAKc9top9WmUpW5SdOUzS1ZGwW/MjRjEUUh7Y6oTw6GqJXq9zHVyl749/2p2ZH3W2sYWnu8wKdsZM+6jpcz1gEpiSpF7V6XvpolCsalaqXtrge+OqoGnQGvLElFryvLUth4rN5g8bAc0tNTk4yNi5DiOS6u3bUjDSm17LE17Rq3xaa0v9I+Ut9GNKIR3VW0r6Q0PSaSSs13KeoJ4biWkhqx4yTdww0jIjXmpVFMpkY+m0ZYTzh2O+qSqsGxl2Yk6h1pq9h4fauLr4a3escQr4j60t/pcWRKsP1nZg5jaqIGhdubdDrC9XfaAzbUq3B5aYfUPTjSb6Eg4m/s1uiXJMThUqvP078rpe23Njtcv7EKgO+avI1hEuVS26Fpj7UVVV8KAe2mnK7nLl3i0CGpP+j70qZDi3PML4on7urKEi8/J1LYzKFpLl+VPhNnZJFKm15KUdXhgufTHwzdy3W8O/AyfvnJojUauX7tGpeuXgNg6YJ436ZqVQ5Pyem/fPUKzz3x+wC89X1jlOsq4X15tTcclbSjcEAaieSaOBnhQBwInuvQam7pq1OsqiHXl5dpVEX1K3sBrVDWmLWWQD2fcRIT6zONejCzJCVz5RmFwM+Nvr1+SFAQqSnwC5SL0tFCIWBHJe2d5g7VonrfXHd3TA5AfV0bsqZ0D6Yxllifl+YesigeEOt2qJWrtFsyFq2oT6iqVBAE1AI1xrsB3UT6OVQBw40dmk2R8CrVEocOidR/8vgJqsO1GQTE6syJM7AakpDZbE84gUhR+9G+O3d+WhZUPUioloWxGBszHHljM8K+dNDBMFmTQa1UirR2ZHM16nXaaju6cn2DTigNDTJYKKtu7Ssz2WwSWrnuG5vbBx657620lmXibc/SmBImEvY8Oh0ZtILvszgn98/MzLLaulWdyj9M5bK479eaCReWhEG8+MLzOMpE0jCm3xbm5zoZ/VAYTrPdoq1ekMvXXqJSkvefPnkaEmFWn//cpzmqevm9p8VrNznZyF38jXoBJ5EN0A0d+j1ZDP1mmzSVPhRLPp2WqHv1Wp2CHhBRFNPr/VEKy2a8mhvchDNYyKNYbJa70c2rhGtDlomtIk5i2hpXdW1VNv3q6hZpKnadwzMOZ39fmPzM3CHufdvb9RkejrquTV4VTzxGxt7E9mD2F+6HqkwQePlGSNKYUO2G46UKvjOMX/IZaFxYUCjmcXFRq0tQLelzAow/DDMIKakaGEeypmv1MYrqoTImzVWzOEoxqpYXi8Xcwxv2QtJI+hB41dwjGccJre7B57IfyliHsZOrv8Vicddtb9Qzhvzb1XVaLBkKw/7EhoHavRKTYfX+wHH36FCqGnpufr3d67BzXuL5NjY3qCljPbxwmHFV64JCieG6ypIEjQQiwSG1u/FLN6OR+jaiEY3orqJ9JaWJmpwKXtSkoFJDuVAm7A9FtISxMeGM1lqiVCNF4wHlqqhBN9ZDXrkiksB6O6GnnqOjJZdv/aqHATh8SO79709e5AsXxLCWZBGeo5y5uU6vIxJEreaLOwEoFn0ClRrKxidRt9SRxXlqW+0DD8LYhKhXF5bOsXxZYm3KfshOVwyVndYaRsXcZrtDsy+nlFfwmZoVSaBUa7Bw7CEAFosul54R74hrImJVEdY3xAD8wANnuOeUBGYuHpqm+k4pd/bs2auEAzl1Qz8j0yLAmU1YWdHgu0KBxviw5FeXvnqG7ozsa/4yf/jSnpgTS7IrIRmTBz7u/f+RY8co16S9ra62yTg8vySG5JJXwFNV94XHP8Pkgkin44dPYJKh5G12418ci3MTMf92NvFhYKjNLCV1zgxMRqBG5LQbghqD52ZnSTb1JUlEZRgv1+7QmBMJZq8kOjU7TdjRLAMj0rrvFygODcT9Tu69dYJq7gGO4xRX1+ZgEEMma7ZULOIFaoyOI9Y3XlUcd1+KVIo06W78ULbXk1kwuYMpc5LcAx5HfQJP1li1FNDTiPuEhGEcbphYCpod4aoKZnGIVSpOSPNxXtla40Yo6/rClatMT8temp9fpKrqcLFQxDrynNg6r4r0vhnty5RmJsST0d8a4OhEdnox/Uga5xmXnnrTHKAfy4SNjdeJVHG8eO0GW+phsF6AqwNVL6bMeMI4ilsyeafqcyxPyPXV5hqhBjI+de4cjsp/caUODS2j7ng0GqK31zLLQEVqG7U4Nn1wT8Yrr4hacfaVC9xYfgWAtN2l1pBnnD51jDeeeSMAy+t9rqyLKjc9N8vRk6Ka1SZnWNWgRrtxiatXxBO13txEowP42nvPANDt9FEHJjaKeEHdu6dOP8zsgqQ0fPFLn2VlVdTEOE4YaLTx9nabUlXuyWx2R1HAu/RqAdns2fy5ByWzZCpmx0lEoJvHmN1q5BZ2I9DHp3j3e94HwHNPnwXg8qUrpIk844K7QvGY2CHSl8/z3Gc+D8A7vmmaUlkOpdTsMh0DJHuY55AR3s5SeH191xZUCWXNVBsVBhqkWHWLLBySg7RQNrgaUTJeDhgrqw11bopQOeK5lRuMaepT2N1moKeq78q9cSthoJkHmXFxVTXqdNokypuj1DKtUf0T9XHOt8XmNjk+Phw+6pUSWVy7Te92Kdmj2qbKLAaddp6qkhrwnGEKFvj+cPw8yIMgbR6tnjgw9OzHWUai3l5nGOSaZHk6WOpaGK5fC8YM01MyWjdkQK8sX6agoQflcjlXcQtBgO8P660+eNO+jdS3EY1oRHcV7R+npEFm49USjiPcrdnaJlajmZOmedi49T2qVT09KPLSxXMAdMNunidUDDxKmhYy7iY8eUE8WkkkzQgbc0yPq9GQOnEiomUv6tPtafxOkmBUIsOQGy2t4+LrKZGEIfZ2Jv499MXPfkIGY/Y0J888AEApyjhzn8Qpnb73MOlAxVinT5dhUGMR1xWpJU4KdNti4G1ESR5vdHVtm2JVYj4adTmhT5w8htXzoN/scfb3npZn9zPe+IGvB+CBB0/Qf0IkpVcuXKas0kRjbJLhMdVqbedpKXdE1rzatp2rabuKXGITzl84L23sd3nDGZHyCgUXZ48OlaljIsPjkUe/Svp8Sfr7Uz/xUyQq4V1db1Ioyzo4NeHw8ueeAGD68Ane8KgYvXsk+HpcB8awpbl0YRTmEtfx2eP7di1UiXpra4uyGt4n4ghfl3qxWmGgcUedXrKbTJskhG2ReKZrVV4+L2p8tVimqt7mMOwzfkjUOpOqdNALUZ8F7UFKQWPrVlZvQCa/qzbGGKhDKIljSmpyqFUCtjTlYxAOqKnJ4yAU6h4wxuRxP9ZaEjVc98MevkpBrnEoePLZmgwznLMsw2bDIEjoqYoZkeGouhXpXPvWYNXrHDtpHifpuC4YGWfH2dX+s8wh6kvfWt0UVPIi7OyJS/uLN+3b/tKwMiKTi1tQKPqUqeiPnVy3jMkolMQKv7HSprchYtyJiSLqKKBYKXP6pEQvO+GARJNpWy2513N3qAXy7Mnxk5w8JRHAl67+PmfPyUIPvBBrpbNJ4uFouIEf+Lu6NQZzGy/NXlpbEibzpof+DIWCMOIJFw7Ni9i+1WyzdEEYTpQVcIxMpOtlpJqgSeKR6oKwaUa1Ibr1ZqeLo33K8uAyyzAFqFqsc2x+UcbHtThI3x5443HGxoThfbj/cVaWZYwWZuZJdRH4vker1TpwP4eU7UmotNZidTEah1x/Wrp+lV//jY8AEon+iAYZvv+9X50HC2bW5plMSZpRrYn68cFv+SAAF14+xyc/Jgy/FSecvS72wnFTojiQ+fnib34cb1I2ozM7RrcpjMjPUpZbEmKw095hoIGQx7/xr+3bt5kJaUMy6FCrau5XEuFqTlypFOQbqtePiDTHrVD0OHNawk5WVlYJ1cAyNT1NkmreGD7lqibw9jSxvGRwNaO+u7XDjjLSRr1ORw/SNIsp6B6Kk4SFI4v6PMN2S+Y7yzLGNHD3INTT8fAcBzLdxllGvysHfRBYJjQvs5SCo3PslgKsI/3Z2d6kr8GtR4+fph1L37a3dyhoOEM8ZH6ku+s32V3LqYVAwxAcNyGJjfbZyT2lNuySNcWrvXn9Itj99+ZIfRvRiEZ0V9G+ktIwN83EfUA4bbfbItIUjsQp0umJsbrVa7OwKI+zSZujU8IxT8779AbyeeHehwiscPjtnZjSmBjS2RRRcXHuEE0NgT/xhlPUx4Vb18fPsL3e1t/t4Kvk4dgC8VD8zCCNpY2OubNM+HJVISYsNJsiERQmxuipKjAYQGlcTuBCZmAwNNzDIBaxvFjycIxm8jse1Ukx6gZ2C7ekHspAxWbTw6TaB9fDr4i0V6oGJKHmil1fZbIiJ+e3fOMHeOKZywB0+hGDULw0Yb/PWG3swP3cpTQ/xba3N9lR6BbjGlbWpf9feOJLPPnCMwC0tpq5unD/A29kRj0sruvRakv/m80mxzQNYf6weAe/969+N0vXxXHwe888S9iV/p+/tkJ5Tj5vPv88vV+RVp189M1sd6T/vV6L0CgmVxzeNjVhSFXN5Tpz8kieq+W4HitLywAkSUilKu1rdga4CrdjMLR35N3raxvDsCLAp6OxR5mN6aljoaNxcPVyjUglBWsSXNUc6rUapWEcnudSq4la5zpuLtFfurqEUUk/cN08zusglCoUCxbG1ftXr5Tp6zsxEb6mohQTJ88DHJSKOWJAqVjGHeYp1uuMVQ4BMDcV5m0c6D7qZRkr6yKFxd0mvgZHe8kAN5O1EcdtPFfGPKNIph48+m1aNy4DEG6v0unsn5e6L1NKVU2xaZJv8lKxRLUmL76x3ufSNdkgnm8JVsVtPVhd59SMiKuPve8Ur1yXRV9bmGZqUiKZ19ZXc1wZJ5N7A8dlbV3UNK/YZL0pC+n6cgffl3eO1TP6CmxmPSdP6MyyNLd1GMe5bdToXjp05Hj+u8FAxNnVlkcwJpsvToJche13OmbT66MAACAASURBVMQqfnpegURBtcr1OjOTsonsVp9IGaTJnDwCXtV0MpvkblHHd3PXbafbzkMPCo5DSxdBqTzBe94lnoqXX7nC8y+KGtRpdfPE3oOReomyNLel7LQ2+NzjvwvAlRvX2GhJH7a7bRxllsWwwtqmqLife/xzHDsm6kehUOC6zn8cRfR78ttOW/71PTjzNgl9ePrCc0RtmZRrzRZldb8fbhS59MQfAOAWHJx5OSB2kt4uAJwNcnDA21FVGX+lXMltKo2xCUra3+3NTV54SeydSeZQCER1nKiMc+O6rL3NjQ0GiYxra6e9q4Zk0Gxua3/leVEYUS7LOycmG7nZIExSrDLS/qCP1bFPkiTvS5qllMq7XmLPv4Pkag3ObZRrjCkjur58lf4wrCFNMJphcHxyhplFMZucvXEDq2BM5W6fRkX6+dzSM1TnNAe04HPp3IvSxoocqGOnHqQ6L+pt98pLuKr21W2HXkfmu9deI/BlPFsDl9KYHKqTJUNHGTdmNxr+VjRS30Y0ohHdVbSvpDSmULeJl9DpiGhp45SdthjzrlxdzUXbUtFh+ZJwz9liwMKCgESNzR/Hb6s5tOhz+CHxtBRXrlNK5JRNkWd3uwMOlYW7RmmGqcj7D1fmqY2JhNXeXGFtVYK1YuMz0FwkHEtFPR9Rv5Ofkgchq8EicZzQa4sIXyiVaLfUuD0I6Wmah2+gVlHUwfEJ6hNy0k2PlUgVJKxfSNg6KupbmC6DqnhpMgRqM6TqyTC+y9iEnEZZ2stV0EajRKDW6Ga7iY1lnB8+M8dYTd7/kY98nPXVjQP384WXRB3zPJ840rinZpNmRyFVlq/TmBGVeqJRYlK9r+uvLPPS888B8IlPfoJGXSQ/13MJI/WKhgN+87fUAK9H3fzhGcqaEvTQw2/gqd8VMLseGec2VQpMK4wnohpf+OKTNKdlDrecDD+Sz0mc7AYx/o39+3h4TtSUNEsZ18Be17j4U/J5bnqST/3OZwDIMpexmkgNK8sDZtXzO9ao0lwT1WdjbYWxcXF4VCoBDf1cq4hEV2s0qFTVE9fvc/HCFR2bgN4wbSWKiBS6xHUdjLoHSsUCqcb4xHFMHB5cfXPU+D5XrbK6LSp3XDN46mxwjEsSi1R39M33sz3MCRwv42rMoVMv0tR13R70yVTSDQcJjbqMxZLu7+76JkfV8TJ/+kGaL+qevX6F7VXpc6u7SaqOg52+oTQu66e2OE2iHs9BP8w9e7eifZlSuymb34va+ENvlgueQo72OjuM12RTjlWK9LflxTPzkyw8+F4Anr8Wce6CTM4jhyZoNuXz7MmHcJCFFqmNZMxmtNbknaUo5pDmBTXTAv6DimDYXObzv/FhAK4trePmzMegWh0xDs6uUeD2pMzCyyIaqg0tNgxvOCGTUC2WcLX/3VaTgXpYSpWY06ekjYtHD+P4wog7zSaLh0Q/P31pjbpiCU/ogva8gKGJxLrilQRIBkkexew7DgMV+SenqnR0U3abKywo9Ma3ftPX8aGPfvLA3Xz8S4It3m91qShC4Qc/+C0kCl/x5HNnadR0nLMB8wrpG6/22dG8rN75lxkvyFhUGhWquvCKlYzGmKyLRl36Wa9XKSmUx/u++h3sbMi4Pf/8RVL10lxtDvJgOm8lob2tUcO1Ek5J1OfrS8u0WgcLEh1CZBQCPw/UjbtdCorHZX2TY4A5jr+rKmQxR48qdtD0NIeXZTMWCj51DaJ1XcPamqh4j7xDDte5+XkStZO2NtfZVq/zZrOL58pkTk81cptYlqY01PW/vdPGqvkh6g/yA+kgNKF5oVPVGs0tYfATRZ+CBkkmccLMydMAnDi0yAtXJWBzrBCQqO45MzeGMyVt6XoOTk3Ux+31FY7OiH2wF+jhlXbZ2pZ96hw6wuH73gnA9Wtn83AH3zV5KI6bxYRqn12nTaLr13EdbhPQPVLfRjSiEd1dtK+kpIcLab+Th9U5JKSq7mzH0Gqp0TmMOKQnytve/34OnxZO+is/+zPMqRrmRn2uXxRvzNyJ+yhOiuGsotU5eltrlDI5qaN+jw317IxNH2dy7hgA/U4dRzHh0mCQG7rjOMJogJ2xKUlycOiS977rLQCcuO+h3Ni5MD/BvacElGxuegZXM9nb7SahqmPGMTkwV7VaxNW8Jz+L6HflVHnzG49y7F5pe5wNkTYdEk0NsK7B1bzCeGDJhh5Ez8EUh+jwTo7v7bk+aSRi9vRUlXd/1dsO3M+Ll+W03Fnb5tRxCQwtlSrcuCEn2pVLV6lqvlgY9zAtDcRrJuLSBO45eYKT06Km1sbrrK2J9DM+4XBoUcZiCI0RZFDUPK/6dIOv/fr3A7C13WL1mrxzI8wo76iEXa/jqcq6UJugMisq+/XLl4l6B8tlvLoksU3VSoW2IjuMFYLcQ5Z6PmVVcaJ+woxWHCk4fU6eEGNwoRDg+IoSUPBzdErHMdi+tCPU+KK40WfykFbrSfocXRQJo1Bs0erKPAWBh6cqkxS10Cz9MMoxtW0SUlWV8CB0VHPzvv0bvporF48B0B50CDW/MAkTjs1LnJ/NLHZKxnInjuj2pO2Hp2bydJVOd4DVIOeqHcdVr/ZsQ8ahu7ZO57rMaxxmVDQGav7+ryKLZQ2s3XiFnnpPyVLqFemnRx87zL3rpa/OtbwJ7btzhwF2aRznFnPPAasJuSaDCS23NFdOePNbBZrjzCPvZHtNxd9khxPqKs5MxtyMiPvJIKGnqlyk7s2475EiDOyV69d47nmJ+n3knRGTc2LraLXXUEccU8cqZNquNEpJVIffWW8Stsv7dnwvveVBKaV0/5seov9GYUSVRj0PDLTG4Gig50RlLo/9ctit3pDEyS48Rdjn5D2yIEpBhX5X87GGLlLj5TAQmbWkZuhBtESaYJtmFZwh+iEO7U1ZEFcuLfHouyWBtxe3c5yeg1B3R9rRG/QpaJ7XTnuHK0uXARhr1Em7ooqYQcjyygUAlm9sYBRV889+x7eTdcTW9tu/+2muPCtMfLIRsHJeQz90M+zEq+AL85mYnOWB05I/GH2rx8/89M8D0G8PuKE4PXgBoeIEdTY2mW/I6ROUfKZmDhb6MKyik2GI9JCamJ7IIVYGg5jFRfEevvj8yznC5KG5aaanhzaojGG8cFDwKOtYua6Bvmzuvgatbq2vYR0tI1U0+b31mqXVk3GyaZxDnhgvyAMS66Uyqb6/Xg7w76DeVN2Vd77rzUd4+/3CTNu9MPcMx4kl6emhMgg5Hsk9vTClownTvu+xrf0oHg/oq1fQjk1xfUU83+cvSQ7nfeMzXF2X/pC5pEVh7NWjb+arTh6TsVh6hZf/4EkA1lZepmI0sTDsMlAcNZNleLfp6Eh9G9GIRnRX0b6SUqYnTT/MCFQF8zwfV7OP75kbp1gSvnbs6CIPvVvE80OnH+TpL/wsAEcWx5m7X/LJgumTeGURdXuDDn21/K/ekBD07dVrpKoalWpFptRzs3TjKWYPCadPeh2snoamu01qNbXDWEqKIBnM+bQKB5cgSkMVrFigMgw+89zcGG2MyWOgMpuRxbt4w0MJMiEbajhY41AdE/E6STPSbBigNMyuT8lrKqaGdJiXhM2N7iZLKejv/NShMsy9W+2zflEMm4dPH2bD6Ry4n5GmwfTCLhcuiRT0qx/6ZX73M+KNMtawqmrJ+pWlvIBhnKUEczJvn//s5wi1oMGL58/RXRUJpLmeMqZluNZX5LvWTpfxMZEQovQcn/60xCOV6pOMa728jXiTXij3X28PsDpv5Z0urgZyjk02cA+IJDqUaMNBREEDE8MopFBU43ackSq+dnu7mWOIHz9ykpK+u1qu0RgfoqvGOT666zpMTYmEsKbeueX1LZ58/lkA7rnnCGvr8rwby+sk6qgYq9fwVe4uFIokqr6Fg0Fev608MUarc/C57Cj+97VLz3N4QQz0C4dm8crSvsx4tDZknprNbSYnhuWmYnqaj9jtdGl3ZF5PnzxBVwOXB/0+0yVR5XwtZvCWdzzClmLgX17ZIXK0kGx/AOrsmH/wONMPfi0AyfYqWy/9HgCXnv99Nl6R2DAn6OJ4+xcOGElKIxrRiO4q2vf48fV02m73SDVVpFQu4arfemayzNKyGPNOvvnrOfzA1+svx4nVyNioNZi+V8Dcut4ELzwlOM1hv0tLo4c3rove6qYRRU25Xji+wIP3iiE8cSv4mo3vBzGeJiP2rlzPpbnEgY6GKpQnK8zOTx54EGoNkWqs6+exJTYMCcPdEyVSO0AYxnmV3ziOc/tAr9ej19Vy2llGbUKNwY0xxmri2i4qJlGaRaDlqxySPAVhcy1i0B8maI5jtBpFlobUNTbp6JFZ+prqYLOERu3guFENbVPsQEslhBeffprVS5e0LR5lbxhdH2A1lsnBcFgl1YnaONtqqzhx7DRXUjmxm1ubpAWZo1W1S/V6ae6uNq7LQG0Mzd4rOOoUyNwgT7/pkZHq2FaCEtWG2nhcJ8d2uh3NqUG34DuUtQxSqWxyfCA/s9SLMvYnF2YZ0zSL+ZmxPEWlXikycNTQnQW0duT+YqWEX5bxWVmXeVra6vGyol2srA1o7QyLRHa474yEhVSLPqnCHJO5eXZEMfBz9APjejlI4UForKROhc0VltWuOTVnaOierdTGoCFSk2tiFK+RRrWG1VpoSRzx0ouCfTU9PU25LLbAXqfLQ8dkvt/71jcD0E9sDtB4ajFldVPWwI2VLVYuiaZzNbUMVFIrjR1m7I3CDx4+/S4WLok0+ezjv8H6yqV9+7YvUwoVYbFc8DAKt+A7SZ5VXqq6fPP/+M0APPINj1GfkriW1Ysv5VVDm+0d1i9L0NyNdsqnP/QhAKoln0EoEzg3K5ulXqtw6Zp0MHISJuaPAXDvA2+BVDblVvNanku33U8watYf9DM6QwiOzoAzd5AS9qEPfwyA1P8c29uywDo7G3nMUBhGrK7K92lmmZgW1WN8apKCLoLuVpNzilvc6nRYPC4xS67vU68Jgzx+XCb98OIcx9XTM1Ew1Iqy0LNGHZSxxmmSZ7a7BcPsMWVs9QKxblA3gImJ+oH7WVWm5NUqRJvC2DbOLbFYVfB6J6Ctcz5wEoxWhi0Yl3XF3X7y955hVr1Xm9tNdtQw38mgvzFELFBAMTegpNVgB1HEugLmp45L2VPDr+PkZYsgA82p6nb7tNT7Nz45tlun+jZkVZ0ulsr4On5+wWHQ3kWBbChC5sMPT+Xt8/0AzxseGhmo8boQeFQ1ODIoGKxm5Pv6nhfPvkxX1RrSLqGqO4Hr4+QVag2ZIgm0+v08x81zAyIFTEzCAdEBU2kADulcmihma1XU3GeevcBTz8tem11Y5Kve+x4AFqYbDLbFLOJ6JYYFGj3P48i8MP5S0acQaN5eUAaNWYpTud7ux/QV8fWl85fZ1tjCN5+YpjMjY3JpeYWXrgiTe+biBdp6SE3Vy9w3K+v9re/5Wp76wif27dtIfRvRiEZ0V9H+hm6rWYdZilGxOrGxQqJCsVDn4bdIjE/B93nx6acA2L7xCqGGzLe3t1i6IMl9HVvC1wodVc+lrjEa0+PC9ZdXV0jUrd5rd1hSdyS8QEfjH4qeJSmIpLKZ1CnpaV6ulShpuaF2r5XHAR2EPvE7Euk8dvg0NhXp7anHf4ejGsowNTnJ9WtD7PCU8oScAJGTsaqS3WNvfxcPP3i/tD0c5JVQLl29wrnzEpv13PMyPmONKt/xP3wbAI/efy+BunEPH1okUknJOCbHrIlJcTwt4zNWpDQs7+NGHDyZBjI9CW1qCDTa2Y9TjtTVKO+4tFXycetVHIUz7a/uEDblpG1vttnQiOhm2OPYmyVReGV9k+a2hBxUFXNo0OsSa8LwIEzoq4PAcQxFfbY1MakagV3Pw1G87izLWFsXySpJwQsOJikNE6Hb3R6OJo73m21izYwvl2q4Kik0N3cIVVLa6fRzqcCGSR4q4DsuvVQlmBQidbKUtVDrysoyoVVcdTcmUGnLLbr0empaiCIKqrrvDPqsbIoaa3EFcA+phFIqHDy27lk1g9jNKzQmxdD85AtnOXv+MgCPvv8x/vMvSNjFNz32bsaLumdLNTyNqekPekxPyl7KChW290hqZhgNr3KL8YtcuCIxYP/yR/8lG2siOb/jne/mg98pYG0zc9NUFAN4PjG80BxihyesXZVUlFNHZjlx+r59+3abUdCHJlHekTRJiRTGZLYxzm99WIDAJmZfYOaQxH9EvR18LS9TrdTxNNel4vvMaW5Vv71NSTPsN9fFSxBHKTWN54g6Hc4/JXFKy2fPEQ4Bj32XdPi8wxWoqN2jMKCojGicEmfu3x+hcC9955/7SwAUZk7RawvzOf/cMxyak/44jkOpKCJ/lPW5940SeDh+aIae5lR98Bu+hrIq7t1w16uS2IyBImiu6UReuXSDclmet3Jtk8svCMKjMxhwcUVE8bd/3Vs5qpjWcZrgFDWD3E8xQ4ZrUgKzvydjLzWbGvjXi6hoaaHpuXk2r8g7L1y+wnqsaI0TEzg6F91sO08LSXohA83jSoxlXWvzdTs9bCwLv6wAYVF/gFFAuGQQ5uD9Ns0YhEOYF0ukHseCHxAMA/jK1Ry7O45tDiZ4O9rYFkY2PzNJW1NjkmzAxKQw3narR5LI92EU5R7Wsxcu4ehYBq7DER17p1pg0FVs6igiUc9dQTdtc3uHc9c1G3/6EBNaZsybqOdlsLeTHbxAtlq7P2BbVeTMOhjdgr5J6PYOrr6t6yFx1l/H1dSsq8vLvOex9wHwd//+3+PH/82/A+Cjv/5h3rAg+84PXCqqvqZpyoTaU6cnZnN87yAIdjH5h2iUnsO//wnxqL949jkKimjwqx/+JQ6fFu/6A6fupaT5p3WbMK9Amonn0FXVz0YhRxeO7Nu3kfo2ohGN6K6i/dU3Pe4Dz6U4jC1wDNZVeNcoZmNDsX3WVyjFYujMcJkYF848Nj9NouLv9RsreYFDx/HySO5huZpKsZwXrXOTLA8pT6MdHG1Lq7dNVJDTqjYf0i3JydjOIgZd4bGT9RNMzRzc+zY08J07+zytHemPtTbPpO90ursF/wo+saY87KxbVq+K+vax3/oY24owsNPZoaZJqY3xCSp1Of2vXRO8qZmpBYp1EZs/99GPsXX+We1nzIUVMahf67Y5dUYkska9TENV3FK5SKOi5X2KLuXyHVTI7auyF0Ki4GZdF5Y1bWg5yehoRDWbO7i+JuFmWY7B00+SvCpu4AdcVyk3SbO84sj6tkbyGpNXoPVLJepD72OS5h4o13MoqRLquA6+nsAmCLDqVTKuk5/ct6OlGzLGvu/mUs3i4lwuhbQ6PRJVEV3HpadS2ksXLuYS/Y2lZaYUuaHRGOP8eYnpsli++c+8C4CClfkdH6tRaolEtNls5lWNfd+l1RGJsRt26WlbnKDAIB72y8szArY7O0wNXWQHoIVj4plOaROrdBtUqhxS3CRrLIvzYn745K/9Mu0V6U+5VKBQGr7H5Njd1XKVcmlY0TegqN7RYerJer/NCy+JGeZrvuYxHnpYyon9h5/6Wb7wWXEUnZgbI1BsqY2VFZ45L7FJfqXEbF1MHmk/pRTsLwvtO9OOUcD/QgmrKlulVKaiLu5ePGBSrfQeCdGObKjMCehp5N3s7HEy3dynHzzM47/zKQAi28PXjd7vyOKv1+oEKkK6JqOjrv9Ly9s0m/L+0HSZvlc6tTBWIrKa2bzRIxgoc1uYpN87mAsZxK0K8Nu/9lGWVkRvduI+zz6r3iRjSIZIfybjEx/5bUDKMT/8JnGZRkGNloL4X7y6xuameOKiQcaNlcvSj8vy3Vvf9Bb+9vf/IABf+uIXSHZE/G6FIX1l2hefWOJzT0qof8WL8dVt7hYK1JQpHT56jG/5ju8C4C0H6Kc3hMmwlo7aRrZaLbaG5a19D5vIewb9AUZVrNjuAslXGnVctXu5npen3Fhrd7/Xfx3HMNS6MscRkHkU21xzq6xj8u8dZ7faK8YRMDogSdgd/9tQosxuc2eHuqZ8tDo9XF1XGS7dYVVnB2ymB1zJZW1Lvn/6uStUSoruOYgZmjGCostL50VVmy3LHqhVfObmFI/9ygpGbVFr6+scPiwHY5oZQmWEvW6bJBviWPep1UXHiTJLNzq4Kp4Myx1lNi8PXqlLXwFW19bZGAZYrmxi1aZWLJSItSyahbyeY6Xg5zl5pWKRYlERJDUB9ur6am7/+tZv+zYeeeQRAJaWrvGrH/51AJ565iip5t5tr+4QbSpgY1qjl4it9uL2Uh6qcSsaqW8jGtGI7iraV1IKNM6jF4Z5NnPmFujF6qHxbV4R1PcrBJpC0qhXcjzf3sJhZhZF1Ly+tsH9b3sUgM76DS6eewGArsJpem6fhiZhGjKWr4sofvXKDk5B3l+fLTOtsTlmMMBsyffj2x4LM2K0Ozx2mAsKGfv+b7v9IByalSC3U8eOM6zP4TkZrp7ajuvk0KZBsQLqUZqfX+B9H/gAALVymUZRROQXn3+GcxcUDWHhGAMVJ1wVj58/d5YXz4loWz52hhs35HfjY+PMqIpTrpbYUjjTzesXWN+Q8RykllhP2uWmxyOPHTydpqPlfFqtLl3Fb+52B3kByPpYnUJpVx0cptCUvABfYVZd18XX09X1PInpYYiJPiz1I793HWdXBU/TXNqx1hLr5xSbn9Ce5+0GFhaL+SluszSvoHI7Gp8UqaVer1DU32+12pR07OMozRN1Pd8h0FM7SmPWtKryIHGYUOzzwyemiNWj12o3uazwv8G0qpw2oVpWlXNmnHpJ1man2eLylcsAnLz3CJFKGVE6yCvZ9LptjuhaLhUDQk3/OAhtKNZZnAykoglgk5Snnn0egAceegtPPSvAfDEOkcaFRbHL8rKo3INwkGsmvrtbdcsP/HyO0yGKwKDPhMYhTk1O0tZE3rlDcznO0sc//hsMOhL/trnZoasYZF6pkKNsjM9OM6PoD7eifZnS7LS6BTc36afSuG4XrAaCeZ5HvS4iauD79LvS0JLvgdZye+LxxzlxWu0k11bynK9ywcdV71tJo1O7nX5ehjpJIqq6QR55070U1WOQuEmeH9dfGuC0hUHMlGu86V5xyc+MzfLk8v5Ro3tpS7Of3/mOR3jkvQJOVyi4eO4QDMwh08lxcYm12mo/6rF5Td6zNYjZ2pDnXLzwCjfWhClWZ+ZBPRImUK9UEvKJzwgu9tGTD7A4IXaAouNRVq9lOGhzsSVMu1qrk1rZGCvbHaamjgHQizN++zNS3fd/+qt/6bb93NjUhRylDAa7qIh+cWijCvLxd1xnFyHQcbG6qJI0wRmWKyoXdvGWrc0Z1JDMnhLfsFsCO01TvCHDcUz+DGPMnoIPJi8iViyWDsyU2vqOLIuZ15LqQamcR+pXynWMN4yitviB2ncSh15fbWWlIlUt+xQ7CYmnEd1jZTK1wbRVTTp14ijJipb86vbZUQSFU/ec4tqSeFXjJM29bJ1Wj0wVlGq5nDO0breHWz54hdzUDO1SQQ4A2O90WFmXOf5XP/5vuKIomJ0o5cJ1YRw2szk+fJxmGLX3uji75dj7KVYzDvLZs5ZSRe7d3NzMQxxaOy1CzV28fPlaDh8UZ2BVBbSInQqgUqjS6+5vWhmpbyMa0YjuKtpXUjqyKNytYYpcWFID2rol0pSPatWjq9CwadbBVR63tb5Ju6P4NfEOrpV7atVxVlfkJLnWHZDp6Ts7LdKWyWK2tVpEoVJgTHN3AtchVOkEz6cbynuijk9FA/nuWZxjXjGXlq6tsrl+8MqxFfVgbbYGPPWs4MHMzIwzO6PVTOKYbY1/YTDAU7C2hePzLGrppevnlulq6ZiZ2TnKkyL+u8U6PY1LOXRI4jNWblxjY1PG5NB8F6PSQSeMQQNA4yyloBJkwRiiTTnpcHxmF45J/8OIO6gklefpYR08PfELBXa9MQaGTi7X3UVJSK3JT1fXcXHV6O74DsEQ4cDunsB7y1uprRrHcfLimnEcE6rzIzU2N25ba3MVL0liSIeQxrvPvh2VK8N4umgXGM93c6+eGOFVAvbB83elu1Dn1Xgu5Ybc326382o06+tbeJ7M97iiY5TH6lSLIl3OTjfYsLJ+y2WfGfUAt1sthsvXMVBvyDjU6iVaO5r/ubGBdQ5eIXcYdwUufVWZwkoVxwzjp5pMajpUY2KaZAjHayOSWKupJElu9M7i3TEOw2i38KRqCA4OTVXZPv/453n/+wUR5IUXX8rhbaPM4jIsI+YQq3aVhjEolvvSlSXcwv4S4b5MqT6uZYXWe4zPqChfKbOxKp0aRBFeoEGFEWTxUCwM2enL5FRKBQaa69MfbBDpPWmcYrV8cEeRCuv1EvW6gu/3e2xo5Gu1WtkV8RNLoPpxoQiBbpBj9xyjrxVJP/vZF3n23Nq+Hd9LBV2Y4aDJ44+Ld9DGA+qarBnHCQNVazwcjmqJoTe+8z5OHpEgu+bSNVa2RVcPSgVOaimp9fVODm52/wOCmfyL//k/4WmybdwdEEValCFJQZNF3UKBY8elPNHa0st5faZSpcCZMwKmN+h1WDw0c+B+Tk7KJnHwSRVLOU4yUrX7DAZ9jDuMMHZyd3WUZrhD+BV2vWuZTYk1hmOvmja0UWWZJVFxPkt3bUdJkuQ2pThLcu/bXvXNdV0cdu1RWXYwz1SxJOPqmIC+ehULmUupMKzvlhAMQcZcQ12DBwetHSJPsdoLGX2dE9cN0D1M1LcsD2SOJxZE5Y6X1ygNMxxqLtMNhWTZvMqE2kdxfDqJPOT0ofm81HmvF9PTAMuJxhh3ANGdR8FnWYanfSsUynkA5Pj4lITCI/MwHOMk6pFpcnKaZvm4WqtAhUCn28nLQA0PsjRJ8+8+8tGP8vyLEh7wxJN/gNFK2ikm936mNsPq2sjShGHXHMelaPfHzx+pbyMa0YjuKtpXUvIURqRYD5ioqiW9vv/UnAAAIABJREFUH+KXhAO2tj1I1ehZnCH1h+Jak0DB0nwvwNWqmaHNcggQa81uPXs9ldIB+F6OQ0pTg/D6UUxjTCuBOA6O5hf1SFjd0Mq5nYS2ws5+8tNnWT249kZP41ZwHD7wDR8EIIu6uHpyZGmGzWNsgrz6yEqzT7spXrStfoIpikH75acvsvkFUbdOHD/N2+6RIMhI1bhSUMAOc/z6AxxFGsgM9PXk8tKEo4dFUhp0NrmvLqrcl558ihtXJBO83+1ie9sH7mddAzqz1OT13MMopqWYzZ7v4qoUkaYpGgqD7zgk2fDUS3dhRIyDsUOxaI/KZoeVOzKsnnuZzYj6w0qqMdnQiu2Y4SeyLMuDa8vFIoFKVo4xuQRwOxrm9JXL5V2Vk1SgbIE0jUk0YNK6Du22vKPfauFqh4tFL8+hi/sJvR2REAKvRE3zHlFvZNzr4wbqmS0EWDXg1+olCtr+sYlprJbrMk7KQGF9+r2UolbxNcZwJ7q40YBX33dy6ZbU5JVhsIJOAFBw3Vx8DTww7Jauyp0TdleampyayD2Ow+owIlXJ+HS7PVYUNePYseO0u8O13GfonUiszT13NsteFYuWAxzegvad6U5HO+hWqVa0pldpt75ao5HRUXiJTmuVjgYsxoOUWiCqQtH3SVTs8zyHYTCnX3DzaqLlqtah8sgxZYKSR31MJmxrq01bO1ifmMyjcM9f3uTscxJRPTtRZ/aw4nI7GVONg3syKlW1nVmoTYtqFIYhRd1QgQmwalcolAOygWzidruFqzlsMyfHOFkW0f78pVdguGjKBa4vS2LxpObJTU6NE/XVDhDu0FX8obDXIdYATK9YZnZeEi2vLK+yelVCDAadHV554Wl5zuQ0dvzgYPNmmFxpLJHqJIOwn4vojuvuupfTLI+4D5M0L9BgHCdH4XQch0yDAi27npq92OZ5yIDJcsxx391NI7Z7Sqynqd3lbTbL7SPYjCQ+mE2poszCy3sr4QXD+oSu6xKoJ69UKe9+dqCv9p3ZmSMMlEGNVYr408GwGcTDSrc5fE8FXz1oGIh1bKamqwTZMHTCp6B7xtqQsub0lcpBDlXT7+96ng9CQ9OHzXY9nMbsYsb7vg/erlo8nDM8Ny8t7meWWA/HNN2tmmwzm2dZDPvpurtwLaXaGAtHFOsrs/TVYBbHSf5+4zr5vGZZlqv8aZrettrxSH0b0YhGdFfRvpLSNQlzIGwWqU1rrEYppqFOgokJj45mYjebPbY3NeVjk9wwmu3xypClORc0jslD//vpMPgLfPWAJL0tUlWrUs+nqXEhUQpbKp1dvrBJU8HKom7KXEOMy2eOLtA6+KFDr31O2+fgG+nc6uoO51+8LH32SgTqMZmaGWd+SjPBHYfJxjCVAAZq3J+ZqbMwLxLM8soK5879f+y9acxk13km9pxz99rr2/de2c1ukk2JpEhR1mbJlpexE0fOOIA9A0yCxEGCIEACJMgvZ/mTnwESJEgyAyRAxjNjIIAyXuSxbMmiLZIyN1Hi2vv27V99tdfd7zn58b73VjXV+rqYiYE2UO8PdrG+qrr3nHvOe971eai95HRMyAVRFGHALMO+f4A+UwxF/rDAjzacMj54n7J/cRRjiYkh1688jaVFLmJbXIHrTI88mZ9iURQX1lEch4hzJtckLeqxBERxurmOA5nTAqXpAyeg4AC8gCg6+W1jHBQPuVUoTdPihDaMMfpiFEXwGclSCAGXXWBDSqQcqJZCwnWnq1Oy+HdllsJmt3jy3pRSsNnFSdO0YDlxpUS9WuHrAS7XlKk4Q4kJNZMoLogXIw4il2yzKCwd+X5RTxfEKQKeV0tbMHiepOHmEQ/4gUKXs81pmsK2T26/mJQ45ForIWDk3occJycM04TgZ6ahC3dZCAnJVpDlWdAG7bcc9QA8Y/nzybOhSRwXayNNU/hx7tZlCLmFRQhR8LLpLCt6F23bfsD9LpVOZho6USllFqfE7RcQKV4gaQtunS7cWHTRZITJOV+h2yYXp9syEIy4IjS1i/iFShVCjqvYtl1kYwYh3XwwDGExhlNVVqEkbdYkMeGUOcNhOWgwa+dZNPDMs7QpL155FqfPU+X4i5/3sb07PQi74piWhISZMCSqpfD2DwlQf/+gBcFFjS+++Dy++PILAIBer4efvEPg6KMwxDVuzr115w4CLmjTWsCtkRvWzymSOy2M+rQYBQCTH2S9WsLaGVJczflVLK2Rkl377DOY45iSbRiFsoAwirmdRnJTPUnicS+Z1uMFI43CBTMMo9jIWooiW2aaZnHICGgYRs6JZjyQ2gcolpBvNCnlAwoqj31MXmey6tt2nQICRQDjnrhHiMeMyVmWQaucKtsax9OUKn6r2+1As1Kqex4qdl5BbiBgeBahNFRCbnm1XCnCPrkzOYojWAlnqYMIqWS6794Aw2Nav43GAo5H9LxdT0IzWmqn7RfFnp7nFaUH00hezAqIAlIXYlz5niQJMi6psGyrUFYmrIKJN9XjZ6Wgi1iPEOOCVoshgg3LLuZtMhuapAmkymOvGdI8jqcF1EQF/2SZyKNgaGbu20xmMpPHSoT+NNV3M5nJTGbytywzS2kmM5nJYyUzpTSTmczksZKZUprJTGbyWMlMKc1kJjN5rGSmlGYyk5k8VjJTSjOZyUweK5kppZnMZCaPlZxY0f1//oe/qwEgGMUFr73YXEWXcYau1G3c+wmxvv7R6++iyzzqhjFmpbAcF3OLjJvsSTyxRdXNX/25Fws23FaPqq+tahMfMYTnd7//OsDXdCyJOlcA22aGmL+XJqIAoXIMBz5Xg3dCDcmQLX/06g8fWQr8+z/6+xoAXv3eAarukwAINtVixLNK2cJCnXCTmqUNNOrM6Nu6h1tHP6axrQ8xvz7iMfsIRtTc6bo2DEEtKoqbG7NsgGaN6G8cpwQTTM3Uj3B8QNcMh3X4EbU9aGh02sRs4vsR+sMev5+i06a5+6e/99ojx7l5/kkNAFJbMJgKZ/PiaoF/dOfmLhQ3kVbrVVTr1PJRsQ2srlJ1eXc4wDG3RszNLyDucEP2wTGaVWqCXjlFWEPDNESPIXiHgxEMXm5JlKHXpzF4TQ9JlleaJ2OWE5XB5o57z3URMyjcj19998Rx/vd/fkcDQKayohnYAmDneFyGjZgxzgdxULRoIPRRY7C/WsVFXvA+SIyimTVBVgATFugIJ0hR2Q5VIAAQgsLEdyfKBPM981//yulH/vjv/d5/oQGgt7+HkBu6TaeMnD7m3PlzOHvuXH4j2GEm5w/ffBN3bt0CQAAfOZOz45XQ4BaZWr2OGq/xZkE1NYdShV5Xq3V43HrjlkpwGYzQsD0oHpvCJ5oNsnFrkuRJ/9yzlx46zhOVUmeH8KfNTMEy6Ud3dITrAS2iK5fOQnF/0vLCArxgjBSYT7AfRegx1ctQZIhCWsTPPvcSEgZ/y8Hcll0PKmacb0dDcb/5UrWCp89SC8nR4Q6CgDbxcDgk+EAAjplibYUmMrGXcIP71qYRhgpHeWGIn7xNFN6bK8+hWiblG8YGggGNP2gIpIJaA5prJp7Y5P499wADRYpI9W04GbPBOhpJxp3/BimZudoCSgx3kYyq6I+IuGBw3Me9a3f5nhRg0Xxu7+yjykgGw0GGNM17pDSmxD6jTzODbaYyBKwI9vc6WFqge3VNCSmYCEIZiDo8zsUSNpapx6/smfAZhgPREJcukQJa+cKTBaa6U2GccRUjikj59ruDQskf7R7h9l3ui5qrwXC5r07E8GqkCF3HRpXJKizThFLTFflqbntREIUfEEQpQmZotZUuEA9MaY7ZhiELhTMKQxjMiyekVbRFSIgCAmES1O6h94GxG2JIAYm8kz5DMvHMHtBtU7bSAEBzkQ7JxfllbG2covfmFhBzX5sw7UIphmGAiyunAQDnnryCW0xa0eu00W3Ts7x39zbu3+P9LibadWJaA5ZpwGViDNNx4Vbp2XjVChpMG96YW0O9QZ+p1GuoMsidV6nC4JYhwzRhTvRGPkxOVEq3Q1pcftCDLUiBIKtD8gNr3T3A27vEk/bxYQeaAcQnGyuTNCu0t+s56Ab0RN547zpW50mJRGn+MDRyOnXLGi+Ai+fO4fQWTXyjWsL+3h0AgEpCVJq0oTPLQ8khq2FtoYL7xslNf5Oyw7THa2eaMAw67ecqZwHQxt25fQu3d8hSWV/zMdIMiWp2kNY+BgDIyjEi7oEadFPMmUzsZ2vUuIO56tEGjZIEccqccqlC74AeaueWiWtvESxJeTPF+nlCMXTLFvoDbtoNU4AXXuv4CDETEU4jTt7blYkCeRKpgaUmWbJh20fAMMau4RWNk5cunscTF04DAHrDASyXt5vUuPwMvX/m9BriiCxFzf2Q0gBMtnBVnCEZcePvaAWfDy8BII56yVZbZieQOfqMJWDzOOUDhAInS46EqTNdqA0pjeJ9pZIC0RKGpE5qALbtIOXTyU9SeBYrIlOh+CWlHiQ2yP+dvDVWLJM9dlLICatJPwCbNDmuT9NdceEizd/1q9fR6tEhXarW4Xg0Z2E4LPoOVRxgxJA4i0ureJnhlHfu3YHPcC0v/9wXsXdAPG225aBRoTX+/k/eBAC88t1vIzskC0tKUWA1GY5dXMdQAha/Nh0HJT7U6/PLqM7R2m825woE1OeffvKhY5vFlGYyk5k8VnKipRRw93pbZgUVy7xpolIjEy0c9dAdMB1ymBTUS1mWwQgZtB0SYLdhFEeo8Gnwxo9/ggvc1f/kOQLUN+0STp8mP3ikLBzsEXpjfxAAbMq/8OUrePdN6t4P0hSDhI7W41ETc4xAsG4MEA6nN4WvXaOT5vTZRZy5SPdy6/oNjBiRsVwtYRBQDOT9q++hskZIkvPVGKmkk3b71jGg6V6a9lrBKOzai5irE9TIsEenyMcfpWiWKUZTrUkk82QpjHZWsH9A8aczGwZKjPaZqhJiBpYzbYkO85P5ozDHkptKyg163KaSqGZkyXqOC8F8AiXTRRiSReYPW9Aluv7hrosfsQsaxhHml8iCW91YweoaWVlew0XuVOYEqK5tQOdY4KMIYPzsyJbQEQPSZybg0LPylupIPYY0ETG0GMchctiMR0lhkXzC6hBigp+O3Qetx2GGJApggybCNl1YE99N2GTXeIiH9dA3Udw3QIzEBQCelhjD4D2IfvBpulDz+N3Z809g+z65/O32AWpV8j4c14Nt0C+WbYmAKbV0Jop4Wb3eRBwxpVkWY5NjUJ7bQKXEUD2bhFrha43vfOsPAABGqmGzm2ypGIoRRWWWIGTXWAmBo3zeblwH2HMxpFEgGfy7/8l/9NCxnaiUHEH+5mopRYMf01zTw21Nm6LsKTi8cErCRFJm2IQ0QcjochkkPHYDbMfCyia5W2sbm2gNSYnsM/jRSy+9iPYB8aV98zd/Dt/+4z8DALz+2g+x9TTRY3/tyvO4uUNm5O1X30QvpoczTBUufY4+EyQdLCy4Jw3tAbl/L6cxDtCfp4BgLHvITFKsjeYcnrhID+fgsIcRu0w/+eAYKSvixsITAM+L5YRozhGeUqW0gEGfHlROuKBiE26N7rsfN/FeSLC30dw85BItsJJ7jE6X5n9vd4CUXeMkCjFkfr00TeHa0+EMAcDpp0g5OqFCyjGynZ0urv6E3FepTURM4iDSAJIVx+23erjHrl+qFRaYT62zsYKyugIAWKpdwgoHw0sO/bYjNOIBB8LjFHGfFu/wzhH6hxRHjAchAnaTFy5sQjbJ5HeXKhANxmqSApacTvsmOW7QhCIQwDhYnaQF9IuQEhlylhagZNFnyh6QMqRIJEuIML52EWgolOSj70trXSick1206Q/Sj97jBMv8EjxOCHWODxGwglhaWQd4bSZaImaEUKE0JMfnLMtEs0lxn1df/UtUOSZ4+akXEbESyVlYaosrSJiwo9PpoGTS+EuGBYehb4TpjKGN9TheprUC4kEx/oF/svqduW8zmclMHis5mba7TH8+W13CGQamqtsu0KPgdqnhYGQzI6mV4YXPkKWyvLSEWzduAADu39uBZFNPpyFc1t4vv/Qccmq2N175PgDg6tWtAm0S5Sa6I7IshonEjT06zUfKwChlt6IbIXIpiPzEqbNoLFNG4uj4GF/72lNTT0Ia0f11D2Mkfs47p9FcIWtHOymWztN1+mqIIZ9GHuZwfExWU9WuY22Dec1wiJ6i90ftFlyDTGpmyka15iC16TqHoyV8+1v0e0rv4pxNVoihDbR2ySKKQw2D8a3DJCmCjJVqfarUdC6//Btfonu6c4jX//SHdJ1oBL+fu90SHp919ZKFskXvzxslNJiSHaYBMBCe3Onj3T9+FQBw990P8dVvfAEA8PSTpwEAZcuA3ePkRyvD8T2y/MKP9zDaJ9c8jALs9inYevf6fZic/ChtNXH5F58BQDjnOYfYoySfDimAHLJOCxR431qgICcwLZMyagAMQyPhko1wOMBwlxIbCxeeRoLcjUaRBcyvI5QoAtcCD9o6k9bRAxbSQw0F/amIA9pdmr/33/0bWBzEXzlzCjG/LlXKKJVW+Zcl0pwqPBjkbF1I4ggf/5h4Dt/5/ndQLlOIZHVxFcubZBXlZRnPXH4W5j/8jwEAO/fvotcl4LtBv40hP7/RaFTgjCdJUsyzELKgRbMt618PeXIY02atG2UkLdpE97s7+OKzFDUP4hHWebBuSePzDRrU5cUF+PzwWo4Dv0ffzWLAZDPu1L3b8LqMWrnIm/n9HxUK7PUPP8LV3V0AQJhG2LlHivDw+Agvfvbz9BuNTfyP/+z/AQDEwT7efpMm6uDgJp77+sMj+w8Th7M8SRCiuUIuyM7BAfohZSO0vIZnnyZCgZd/aQVlm1yvxK/i2jV6CP3OETw2fzM7w3afyQKqCdaaFEupzvGDgcSIzemb23dx6wcUr4oHNyE2GSb3sI/VU/TwvIYNSEbHNGyUShaPOYYlp88yPv0ZSt/fCCL0ON0/X6qO68UGbaw26F7PN6ow2bWxhIkmp+ptr4wc1Nh1PZTLtA17h21c/eO/BAA09tmla9aQcixDxQJWwG6d0vB5UUMBWY8hlVsDlI4og5d0B4g+S26tcdoc81I+QnZu07wbQsHKgfNtC8LIa95sSK6FsiIJxa6Ha4iCJy3VNhxOoXf8CCNWaKZhj+NcBWywHCMpqknFIoqsncYnEnRywrHkYh4NDSWmHCRQ1BHd9odo7ROzSKASVBfoUBNCwOMM+PziWkE+GgU+PI7tXb/2EV7/wV8DAGSWoduiZ7K7fR9OlTJkNpMcNOpNfOmrX6PPSoEgpOfk+wOMGNr5YPsu7tymsoLrN24USm5jYxPz8xQ68DwPc3Mnk13M3LeZzGQmj5WcaCktGqRp12GgxoHZdzvb6ESkGU+trOLfPqQAsNUfYf46WTPOzT1kTABwWgAWF65J00XGVkn0xjuoM2uo4uK9LFUAuxI1o4JoRNp4zgBKmi2S/btYZ4bYatnFi+fo9D/sxdhncgHfb+PW9etTT8KgQ5mt2oLGcZ/MdrciMBxxcDnN8PGHdALs7dxDtUrzsry8iaXTzEF3d4T7R0SD5FUV5hcpgNishZCS5sW02dqQdaQxZa1UIgBFluSlZ3p48gxTnJciNBcVj6eMOGYa6eMDZAza7tmlolJ2GqnXub6pdQxL0pxXDBcdxX6lDmGzX7JVLcNjfOZYAhETGgx6PmyP1oK2BEqCxrS0sACbC2z9+5Ss2Ds8QspsrFJ6ANMCmY4orMaoH6HE9EPtYQ/+Abl49aqHimDLU6Y56/Mj5Z17ezyWrLBgLCFhstVimhYstnYsAwjZaFmq13B6jp7ZimuiUqL5CcIQgkkwOv1ewZybcQrLsGzYnGzQ0AUZRhRGRYGlFGJMU56mRe2W53qQXFCqAaSfxkRgevdGcw4Ht+4AANzAR3+bLMWDgwO8/c47AIDLl59FqcxM1lGI3FD7yTtvoMeuV5pmUFlu/Y0D8klM+3ioR8i9Lsfy4PHv1ZtLcLnQ0pYW+mz1fu1r57C8TNZRpVqD6TIRg1JFDePPHNpJf3yySj9UPm7B4NT3hY0NDA6Y114LrOfZN1vA8JlwT2lOrgKRlAVxn6U1THZuLZkgqXJq1mcurUgj4we5LGN8jcvXY2EjW6MBunfuwM9zz7UqnnqSygpW/RirDIh+4dwazi9Mz8suVK40BYYBc38tL8EAmci7uwn6miay34lhujT+49ER6lUqj3ArHmrzVCDmOSaWuaiTNnbeRpFX9R5Dc3Fev7MIxrTHV39xHg6Ibnx1pQKblcK19xTa7G6F/YDovQHUFypj0PgpxOPnINIMgw6NUxouTHYbdCqRphW+RwvlEj8rQ2LABIq266FaobmwbAOjERM0ZCbm2H0vMq8ZCh67cNTGYECvS2UbzQpd57Afw+UFq9UAIW+C+/f2ceY+zfPS6Q1k6mSusFxEmckiJzJekUaxHjNoos0BUFK6aHEp+yE0V6I35kysVjnW1Kig1aOx3zz0cYPZc0RRlexD8B5wDKvIEsZRWFQKCKBQSkmSFMrSdT1IkZcnKNhFIu/R8dCQ95HtlgpFmCYxNLus+7uHuHmbMsmvv/7DIixiGiYWc0LNJMw7uTDoDzDPbC62YxfEAXnbj4ozWBbzIzaahQILwxDXrhJbz6vf/x7u3KHM+NraOlodigNrCJhc0mNaVhEu+Pov/cJDxzZz32Yyk5k8VnKipdTeJa0XpQKBwa5EvQLPJ00XfnQTmcHBwbIJadBp4KRqTA2sJ1hSLWuckQBgLlEgs9ol3Ri6QHyKLI9mOkSZg6RpN8XwkAPAu69i7y2u0XjqAo45ixOX5pCyF+Ifd9C38rPx0TIcUPDdGElUOduQ+D4kmP7GiSDZTak2G8gMOmmD+Aj+AV3nzPpTqHvULoJEI+nRqdMslwC+F5+DgzBTKOYku3XDQnOZTujnnp+HByrMTLIhwhEdtWlygJj7/RzDgcf1YIYBCPkpmt/YkrQywOLzqFGvoqRobPf7I0QZ3dcgzGBZ3LPnOEiZJ25jcwP1eQpUto6PC/641CRuMICCyQAQBlGRTfX7Efptyibq1ENlkZ5zkqQY5rTPUYaEEwBha4Db1+ikX3h5DaY1XZ2SZitNq3FhpJrk7xUCefFiKhRcpiCXKsV+jxaQUinudLlOSRno8v31/BQ+u8t9nksJCc1JHVNq5FaxhITIaas1AJ0zyppFQSnSDDqnQNf607S+ocEB7YPrHxW9ZGHgEy83AMsU8Lhna+hHhXWiTBt9TjJk4Qj1BllNsdKFhTscDmHy+hyG9F6tWoPipr3W/gFGI1qPV699hLfeJJqxW7euYsRMxLfv3oTFe0lpAWlwK4phFDRa/+1/9988dGwnKqXjIZn490chUm5ctMUKStwrdRwMsML9Ql4okfVp4FGcAAv0mfKF8whTutFhqw+HXSUjihAd0cDg0AIVjQrMPLvRD+E9RUoLdgWlQ1owo50ddD+mcgN17wDVOYpvtBsKx/t0nb3DbZyxV08a2gNiOLRBgzDB8C7dU9QKsLTGFbGegx67dVUzwtwyLYKjIwdGRsoniwyEHNNyRBnSoIfdbvkwy7Twjrn6PRgOAZP+fn/HxOoGKVy30ofJijgIStARfWZjPUa9TC7O/t0RytyhrWWct8FNJX1ufB4dd9As0by5toM4yhdsBl/QPHciiWqNq3aFQI37mBr1UtEc3OtmOOZufwMVLM49SJUehhHyYFAcKwy5WHY4GsLhsu9MCrT4UOiEEUJe+GESYXeHNk8cJVDmdEGlPNYDjBtvlRp36Qspi1hPKjSqXKLiSqDFzy9MLEg+KP1YU2YOVKVc5s/HSV5G4RQKXiODyj+rFXReRa5Q1BBonSMFsIjxMT1l0ToAYHPzNADg2puv4bhHzyDoRNg4TR0JUowJOIUYF3sqnSLlisiy56LPcz8YRfD482+/8w7usBFQrdPeLJfKRS/itWsfo8MlCXfuXEenS25aprNCQUOg4AfUiujFafx6xvs2k5nM5O+WnAxdwoym+/4QSZ9cj4XlRehNMh2dZhVOn5lTd48Q80kzhEJWYQiMU1swBWvmho/kGmUHkjhByKBH1S9fBgCqXblKXfdIJbBHJ2WkurBWqDBy5Sufh+ORpdK+dhMNn17XTzm4x/UanqGLoNw0IjjwqcMMizWy8IwgRTrgDnfHRBzSidJqjaDzdgSrjMUluq+l+QUsNmhekBiw2FxNjCH6IzpVtg8og7e/fYD2AQ8zuoJqg/6+3/oQdUFWUMm+jKU1yjKurVchUnKxBpc8xGx5ZsKHH03PT644iJwMfMxxF3iv28cR9/UtnGqiWbb4HvdRC8nadEwL8xwcrZRcmOzK12oudu9xkehIFL1ew9zi8H0o9qI7/RDdQV4kGsPcp2drV8sYshXeS1NEbFFESiDkrFeqNLJkOnd8kuU194fEBMoAvU+fFZDIuE7IkQpDLvDrJwplj7N1tobDbkgvSFBmNzJn073TieHz2W4ZAoJfCwk8UFU5AS6QWwJa64LF99NKiTPjq5unkXB9XBoliNgy7fZDJDyXludCcGA6CyOkDPejDQemwwHwKEPEc/H+9es4fpvQKkoeB79Ns2DlDQIfit1OrVXBkgwYgBz3Hkp2AWGIYi70J/GkHiInKqXNTcomyds78HjtZ7Euig07oz5eu0/p7rVwgCdBH4riBMEOFR7G73yIIK/sXF9HeIGKE/20hCvnSBmNJA082L0Du8fUzjUb8T1WYAcjWEuUlfKXl2DNMQDV159D9z6lgBsLBp6rELzJn/+gA6exeOLAHxDuZbNNC5U8U5iZhZkrnBAl5rI/PkyQMVrIpbObWJ+nkgjTtBGO2N2BB8Fm/DDWuMoFfXtd+lcmGVSXewl1gAtNrhj2Q8QmLTYjaRUZENsLsbxAsaaF2hb6TAEdJRHK5vzUwzTzzSPMUYz/AAAgAElEQVRMxAE3Ug+GCDQpqy/+4hfw1GVSRD/4/W+jtUPPc7VeQ50zM3EcIkpzmuYEUcTKIlM4ZmweqDyuk2E0pM3Q7YXIOMUvTQv7TGm92qgBDBo4UANEiudCGDC4cC8TKDJcj5bcTRj7Qj8FD8IuRiYEQt6s6bAFLWhdWU4FyzU6VDxD4hSHIs4slVBm2BbWy/jrG/v4/nVSsO1YwMBY+aVpvhEnGm+1nqDcLm6FvzPlEAGEnMlcX9tEpUExvuAgQLtDB8zIj8bU7FJAcZZRZSlivsdOvw+b0/lCCgT8LIdRiCjhjDhndw3IcRW7lEUvITVL55cZDyZ7oAL/wfl/1Dhn7ttMZjKTx0pOtJRWuDZosNNCqZmrSQcWm8h7rWP8kx9/AAC4OF/Bf8q1CCUJaK5fab/3IdqLdALdikaFll67sIatJr0f75EvU7m/B5Hb+wMBR7I5HfjIGMJT7+6jU6UTt3xxA2tnCG4h3D/AIhe8ffbp89g8szH1JNTqDO1Z9qC5x6zcqCDN8tNihCEXhRlDDYfNfAQWENApKsxFZFzj41iVov6l1wF0nwC5vIRONE9bcAwq+tzvvoXTJrl9G+7TSNilDfwhejFZgardg1BkWTTKfShJ4x/0M9jl5tTjdBhaZWXxHN7OaM478LH2FF3/C1+9jCcvkTs6XzLxr/75dwEA/e4Q/ojmtt3qI+ZTVJsSgyi3CBM02fpyuD0lS1N0+USPUw2Li0fDJEEn5BqoWCNgRM4AI8ScGfPTIQx+zqWyi2xakLcsz35N9LvpT5gjuSshgSwHFcQQLzToes8+/wKWannmSMLm2qPNRatoUUlTLgS9uIx+QO/92c1ukU0TmYKZ1yBJWfQrQisgh0VW2diVw0Rb/RSSI7iahokmQwmlYVAYJX4QwOaapSAMoTj7ZhqisFSk1AhDej5SyMJUy6GHJ+dO6Ww8BqUw6XQWn4F+0H1+yGcm68d+lpyolHoZuQmm7sHiAq3Y0Ohy7r0daKTcqNu3POxYtOgbOkXM6INaR+gpGvj24Qg1SQuz4wF/uPOHAICL67RBz825mHfIvRvd2UEWMJJhlqLTOeLXGjG7UkmvhfgnVLldgkbkkil66vJTSHbvPmLoYzGi3JxPkbAr42vAHzLmtq1R41iPIw3YKVU7lo1TMCJSiipYhmdxUVomITjzsFo9hZUG9eoFGcWlRu0Atw/p/prmB6izsthaOoeP9qkqXIomLC5qjKMMIS/8oPI3yGxW1qGLQZcrmJ/5e48cp8/ZUenUELFeXTu1iV/+d+j+zl9cgM14Rk998TJSXh0/+Md/hHdv0qEgIpMq7wHANtBmRTTXdGFyT1XQ54xObwgGm4RhmIhS+p9eGMLnjf7RzhHutej9QaaKzFQEgdoCHVqVcgltfhaPEs1ugxYCWv40qqNWqsi+aSgYubtcPQ3B+FHRqIe2SUq4WnJx/YgOhDc/7mJ0TP2YpRVy22UmkHCJTEUqhHmWSZjjjauToiwGWkOl7EopBZPdfKqiPnE7PiA+N47fvXMdnkvz3qhVEbHykV1gkUs34jhGwFAscZIg5tiiaRowuCcwSdLCVcsmspW5ClEK40zhJ2J0msc27un7aRkjb+KRjccz920mM5nJYyUnQ5dwsNBUCRY4Yh8bKcwkLwZUWF+kgPLGmU3s5NgcWsNmq0WkJmIOfK7OL8Dk2Fv/aB+6Tdp7l0v3eyUbW1wzI1s7QMAFaqlEkHJXchZDs7VVCgT2dhhGRQiMOLDXiFIsXLkw9SSoQz4NPIWYu/Ftz4ZtURBZxgqaT3mVmlha+wwAwMou4miXs4ymidRjszyOCrAt13MheZbrDQoi2zUDbe5rs8sl9EM69Q6C91FZ4RaErIkoJLfGyNaKjvP99o/gWIwjPncFMpm+nWb7mHrSXnvvNSyeIyvkt373mzh7OXdBA0SMsx3HGZ5+ntzOu+/cxF/8wffofuMykohbD3SKukv3tbm6XpykQyaT6IQK3YiD2wAsi/4+sCJYDbIO728fY5/rtxa2lrC7TRZxmhgFFny/M0CYTtdmUsCVaF1kAyehQybRJqE1hKJ1et+38HGP1t6Hx/dR55orlWl0uagy2f4QZucOAOA3focspaOdXZyrk1Ul3Speu0vP0tBAnTN0VceAw9jVwjCojg9A4AfohTSXR9H0VhIAvMHoqzv3bhekHqNhF6ZL67FSqWBjldZbr91Fhy13z3PQ6XKLkQRStiyDYAQjxw59iCUjBMaR+E9gpj/M7pnMeH4S2O5RWOQnzoQX0MLZTetY4s3aDLowD8llSAcdXLpMD2fr4hNo//gqAGBVGAAvQEtLeGx6m9Aocabl2s07WBjRBjx7mszMbTvBwQ36bW/QhsjR8jIDIVdRx1Ii5kbZdjZAqUSu1CCOMGI3rL1zAHNr5cSBT8rljecBAFnJQcbNkquNBbjMxiCUwNERZc7aoxSGS/12YdhAwJk71+sh5mbNYORjxM3EWZYh4xhCjSFMvYqHnSPGFjJK2OOSgcqxhsHIi0n/DkqSFknTOw3T5oK/yEbZIWW5sfIELKxPPc6VcxRnSysxPvPCswCA88+uINMU/0uyEHGOEWII2BVaHlvPPIHhtwiWxEw0+oxzZZsSn3mSClxPnzmLHscRR1zouu8nOPBp0RtGBsMkt66yYuDnfpWwlw7+6A3sJuQS/Zu/8wv4q++9DgD44St3scMKKom2IKbE/TUm4hu2MUbLzDOGtCHy+IaEYCcrUgLHHOeyDYEqV99nKVAJKbsW6j4SPqjTDq3T/ftXkXJJycs//8tY8LhBuWJhc56ft6XhcrGoaZqFK5dGEW7vk4L4Jz+4g71w+vKAm1ffBwC0Wy2cPUtZZ8dzEca8T+IQVk6LhgwGK5SBH0Cz6+y4HlKuzNZZhphLMyib9omYEMZxIiHET8WMTpJPKqFZ8eRMZjKTv1NycqCbe36+30uRcjnMz6kY3iG5AW7i47PPE/DT2uZ5/NEb79H3orDAt06EAY+zCuH2PgwGeDrbXECYUU2FWaZT5MoXX0SbrfT224eIuABCmQ4C/o1yeR5g9IDANqDmmcQABvbZ+uh1W+h8TAHwX5tiEq48+1UAgKxXISv02w23BIMBzg1Y+ODqWwCA43sHuL2fc2GF8Cp06tjJADqhcYx6AVLN1oRtwR/SaXTrDgWxK66NjEkfh0mMowGV6Z9LTqO9Q/N2785HsGL67UblAGunyd3qpW0odn3mrCNUnAdbO06SxirN/b//n/0j2B4HOOWg4CSTMOHlsCQ6Q8pu99qpFVy4RHVS2+8dQXNW0rC8oq7q3Zt3cdil57l/RBbTUS9Gny0caQSouDS2l37+S3jxV14CALz+49vwb1CPW7lh49e/+WUAwLUPvoV33yJr4Ku//gRWTk+XZcyREoXMUOeiQj/VRfB9sqYRAGwjbxERMNkK2qo5uLzMbUKdLnqcQUxUhsM+je37r5D79PQLL8PhHrNmpYTNZQpnLFYsNJjcUgqFEgejpSGLQHN36OPqfbISs2QMkTKNtLapDlBlAuC15JUaODyicEbFq2AwJFfSsgVCLoQOYsBj76LX60Bz0L3klYssokp1UYdU1H1hAvP8Z2XWQHRW+XsPc9OmsbJmltJMZjKTx0pOtJTiPmnxG8cHCNgKaGws4Flmbq2aKc5sbgIAapU5RAzoFfkxbMZ3DnUMm2Mjdpwi4KpfaZpQTAFzwAHYzkcfosRsqQO3goFHFkFUqRYxmtLCHNocuxmkGWRC8Yu9/SEk10n1kxhlbhSdRs5f+RwAQFtuYeGZxggG0xAJz4D/Po1n5/4x2iFZNtVKBek+nzROBUtzVO8zX6tj6OcB4xAJN9kOu5RaDlUKyfVYw/B+0WbRVwMITmNbYhkf3iDLqr4wQMckC8YqJxgmdOofd4Y4s/wCAOD55X/wyHGOImahmXOhuJtd66yAik0jBa3HlTM50WVjuYpf/81fAQD8i/0/hN8dM3kcSxrHwlIdw5TmPGIMb7NcgsexwKXFZbz0MlXwf/4Xnodo0HXWzsxBcbD5xo3b+PW/9yIA4OLFVbz9DsUot+/s4dT5tUeODwDK3LhsGBpthmH2Yz0m35RyItCtirqjTKV4jjHWv/zEHBSzx/RMIOMkhz/ooVIji/XZ52neX/j8F1FhiyiO4gJADXrczmI7DhJO1W/f2cZfMcrFW3sDfNSl6/fiMqQ5fZymz3RiJctFnwPXpueixDEtyySgOQColMoIua5JRwkSprfXaVxYjZnWRdAbEBAPq/HKh/YzAtVaaxgcL1JaFw25nxT1CFrnE5XSN07RJj9qV/DmbTJh//xOD95ZLpKsOKgyFUsyCJFxj9soCuFykDEzJDcCAUpKtDkYqsMUNnOgJ12epJv3UGLjLS7V8B5nXO60DuHmpTEqgOWyiZ4IhExDNNJVmBWmGrYETjUbJw58UkqMd5wqiSxfF1YKpWnMbsVAwsHog+sfQrOLt7jyFG5cJcUdCA+CA8DmuobgYOrevTsY+cylxjxyRpZBaK67cbvQHFy/v38fTc7kbG5tIOJioiAeIo7ou9U5CyFTH8X9HhyQ4sLTjx5nmmcQJQAu8jMTA2lRQ2IWtTJJGkIzLEpqRdi8choA4K3U0PuIXAdhWth8iRId/8ZvfQN7BxT8PTykTTIYJUgFbe711QVsbZHSjs0EnYAU+8apOZiMgnnr2g7Kf5+u+cJz5/Gjd8gFD0YxsuTkhZxLv09znSUKce56SJkjevA4abwSgMEZw/PLZfzOVwhcrTcK0WHm2KZjYmdIyvbK05fx0hcpXNGcI3fSMy04XNvWrLlw+UK2THHcojXzwcdX8devE1HDq3/9KjqMEDH3hV+Dn3J/pciAgkL80RKwC2ggRbtFa3BxeQXra8yq7NhoH1OAvnV0XLSZlGRSFIMura1gv0Vj6/SHn1BKDyrISbfrJKWUB/HlhPLPsuyB4PbMfZvJTGbyd0pOtJQurNGf/73SFjYdOh2/d3WI794hrfuZU2sY3qTO9y4kDNaS3djHIuP1ZNpAwnjdR1qhxU2WoZmiyvjE5TrXhMQpwI2ajlPGNpucx5nGClsTpXIFVWZJ0EGIVpyX2/swuO7paW2jMpieGSKnnNFZOgYty0IomzHEBwnEkE72dHiA5iJZB9HRAUaHFKRNlUAypHs/PjqAwVC2QTBAEND7A59+w5AmYJCVuHHGxNIqBR5LzvgUGiX7OMPYOGa2Dj+mdh5pbiPOyIIqVzagph/mGEcoSWByC4JSgM9wxGQlcTd5msDiWrNYAh4TQ1bWGtjnNHK9XsPSObIY6qcrcNcoNX1e0L9JEBcgYSpLIRmLSGgFh3G4FhbnUc2ZUqwySszw+uyLT6D5LQomqwQFYNmjJM4xfLSCye6QMERuGCKFhJ2f+GmGZcaG+rdePIsNZnLx+0MsN2hNNh0DC+WXAQCXLl5CrZ5XSXNLjZFBsqXUPtzDXU5mvPHWO3jzHXLTbty8hQGvjQwGmi/9BgAgyFwItl4tQwJ6ehshZWQHBQlknPrXKUwmiVxZXcHSArWJ/enNb2NtldxfzwJ8DieMkgxpnkyChCw6/MfXmSwDyEUp9SDqAsbV2mrCUspFyjHjy88KgE/KiU86immTz7kCL1+gArvWSOHtHZqQjw46eIIVR2yb0NzhPQgj6IgesOWaY+AnpeExSPxAh+hv0aTNP0V0SIYC3vszWoibYYSN5mJ+I3B5sntJgNEx3ddKqYK1BaaCkSasNhMaDIbYbEzvvuVg8HGQIWQll+kAaUquYYoYfo+zN46AyXx43VYfrT3KdsQ6RMrU1pXGKtKQN33sww+Y4ywjpANhWzC5jmthYxXnL5CS2z8+gM143UIeIB7R9VeazwCSFpWu9HH1Y4qVrC4uo+xMT7EUMKyFYUjYOa4zNHzuDg/CwcRi0igbeZe+hOQ6tcZqEylDVUjLwRy7MUmWIs5RF9ntFkgKltY4iQuOOg0Nm6FdKrV5NBe4Nmx9DRm7cvNbGlvn6NnqTMCcsi5GFKV8KYTOXSkT9RJThkMU3fNGkmGDqdEvrjYLamuRRShzfPLUmVOQZ6kWzLEdZDmBQovioG/fuIEPPqAD40c//jFuco/mYNAvAOdUloHDp3Dnl1FdpN/TaQrFLpuGgUk670fJ1gI99/m5EhpN2kdWqYaQ47pHrUOcWqcWqM31LSwu0H5IswS7HxCmdqs7AHNQEPjdBODcJ2VSkZAiypUVxq+hJt4fPy8pZcFKXCAXnCAz920mM5nJYyUnWkqCg9UijbDaIAvnC2fq6LNlcafrw+eGwqXNTRg2ae8w1QgZZtNMMtgWuRt1ACkzodSytOCtb3MQs9FsosFBcSv0sc5umg0JwbjUwipDDuk0WDZLcPOO5yiDz9esGz7ObZ1M4zIpmcohTAE3J5qMRoi52bWddFGap5PmK9/4Ena5GfJ+eweL5+i+lJDIEm56xBDlGlk2h/f3EMY05ic+wyR8nsZxj1y5xpKHHNM2GArMLdKYU93BwjK5MouLElKSpdoNSljkzJVjlHC4Oz3IW8iunlQKCXJ2jag4IW3HLthRlNII2YIKY4WEV0q1XoHBtBuW68Gx6L4iXyGV7Koxg4mpDOQYZhoCKeNa+4GPiDOy7fYIQZyznHhosbWbJhnK7MqNRhl8fzo/1ckBxwzgAgd9z60u4tQcrYfucIQedxjYaYhqQs8yDjNEnHGrVksosQUqFFAucxN55xB/+ZdE3vjaa4RL/dHHN9FimOE4jcaNt9m4ctwwzGJvWPNbEPxaqrjYY1oraD19oPvcJs17qVqBxQwud3dbOB5wUmUU42iLLe31VRwdkWV368597DCuPYQBnSMZTGCa/yzJrSUpRZEsgFJjfCQhJsDfJMYW1zgTmf/vSXKiUsrBqLTKYHMh3eU5E0erZNaPoggppyYX5hfhVmgRdZUu+KLSOEHE8RMpDNTYNnNB2SMAABd26f1DbPAdW0aKKvvNS4aHDgO5O9UmVJKDonXR5w0QZYDivq3Vy0s4szU9yFsc53xXJgS7oMhMWIxG4DaqqIxIWQ1u3ccLT9Fvn3vKACSZznEg8eZfUXyp1bLgcUuJHwxRn6ONcuVzFGu5fXgVYAqfta0VNJmOqVJeQ5ASpMjAj6A0fW+79T7mGvnmr6PuscsUZEXadxoZcQtCmsQwmeJpMOiiyptucX4e2hr3K+XuTOAHyBjVLFMpJLe8dId93L1NG7K5WoXhcWY1BxRLDAzYvQ/jaMwlliRI+Tr37u+hxxtJWhJ9Bp6X2kYQ0meu39hBrz+dUvrKFSrybJQ0zjH3XjnLUOemy8Q0EDC6ZjoaIfL5eUtZ9O6VbAmL4yvD1i6GTJ/+3b/5Ef7p//0nAIDWIW1spTiuA0AJo4gvaSgIi9aP7ZQKMDVzaR3gglOoDArs6gpRZESnkXLeb+c04HNMSRkSJvcLeo6BwYgB3xIft+5Q7Lfd7hdxJIoyTmbUfjru84Ci4vnRAjBz/HPoAiVATZQSJFmKjItRpaDC3PzzD++WG8vMfZvJTGbyWMmJlpJirZfBALgcvW4KfJZNx+NBGzHXpiSjEWxmvAiFRMKZBKkSZEkOfCWQ8m/GlgBAp5fIcVwMG3n1WZam0GxBuZkFzVmxfbeLhJsblQNYfOr5flygGixurcA1p8fozhj2NgtDmNxxLcwA1RqNJwu62LlHwcHr799A1aXAfDi3j4Dva97bglR0v4vNC3C4FSZKFOocZEw40zIYtLC+schzEuKV75ErYJUUlrboXmzDwf4u00dlx2gPyR2cc9dRr5AFkJoS6SMK0SZlwFaIbdlwmFvetp2CpVUIs2gq9n2/IM+EHp9tiU5hMCRst9vBn3z7LwAAtflfxemzHBjngHeapfAZb2kwHBZBTsu2ILlgcu/gGDE/f9Mxi9dZHBVj2723i+Pj4VRj/K3PUdLAdjTu7tH8vfbKX+OpJXqWwrIR84l/8+r7OP8EoUlIpOjuUOZs1Olhf4+SEtdv3sT9FmdeSyuYW6ff15w9zOK0YLaNkhipTyEEzxKQbPmE/giZS3vGay4VlmSqMmhu8RFCFI3b00h9gRrO7+0NinFmQiBmZI0wiNHlOkBhmYiSHNmBmoIBalHJs2WEl/TTEMJjiwkwjbwwUkGz6hCWU2BYGUJAcfYzzcawv0JLiGKNKUCcbBGeDF3CG8twS4i7DFafZFjj3qtneiE+6pK7sb97D31OfQ+VQphTJiuNNOfW0iZGbA76WhSY0YqLAVUUjoGilEZo5r04KUZscoZOVGR0XMuB4mxDWUU4v8xpXFvDP2ZKpBOHT2JxhXoy9GFyvCTMWtg9+AkA4OO33kOVM1HlxMVH3ydQdee0wDErztK5Bk5v0LxsH0TI2FUybRvLW/mCoDlUvo0So0fevnodr/0NZfA2LptQVZ63dB5pnxTr3KKJO7dpw3zca+MbP/8lAMDKhodRejzFCEk8Vuaua8Nm981t1uGwAg+CED3uXwsCHxVWflpl8BkkDBIoM1LnZz/3HO7cpwLHf/w//1/4ypepGvvJK1TlX192oHOqbsMtOvLTOMURFyfeuHmnsNcznRbxvSCO4XFmzBqYGAXTEQcEnHFrj0J8vEcK4tX3P8Q2s/3OVzzUrTFqg8dxq+29Fq7fpbl8+913cH2bChIHoSoosr/22cv41UuEiuDmYQjbwc4hKbDtwxb6DN9z7YP3cfXt1wBQ9s1eJbdSGQ4yZpKGMCCZ4IKU0vTuG4e/sL17iO2c+1BproylOS5xTNZMFbIkB7nTkPzstRqn8AnOnw2JiXS+UmOlVGQ2J6q1DWmMseQhoY2fho5RmYLiEgqpFaQxc99mMpOZ/B2SkyvSuKpQCAs5LHUoE1g2abqt1RJub9MJFkcjZNzP1U1jtNhcqxoGxETQrMcW4n6cFRjKxgQ2ca4lLRg44MrAHjIwKQbWpUCDTVGjPcAyBw2f31zBuU26yVIwLPrwprGUOsl9HkOAERsEB933sNuhmqnWfhcrFrUgzAsDfSamtPZrsNlc3s6u4eLXKJB9rLro7NL4F1czXPkcWyUcUG61tnDEiAblShWXLhHOUW3Dh864Tioxsb9DgftR20QckUXWHfawc4mJPqtL2Gv9ZIoRklg5GkAWwzVorvRkoDLTcLiOzLZteGwpDwZDZFyD5ZZcpOx2n7t4CheeoUD/n/zBK/jWP3sVAPCN0XMAgBe+fgpK5jz32UQ/lcDhIVklg2GIzVNb/HqAfQ4gm9JEfZ6+K60lDEfTweH+cJeZXsIIewdkKZVKQJvdqtv7h1hjZpZv/saXcPkZwpWyvSrmV8nCW3ryIn6eLd2luToaHt1H3SvBcWl+yvyvJSWGzCzb9mPsdek5/dXiAgK2MnaPj6HZOvDbu0Urk1eqFNhGk6Bo00jACzVJkmIfEQ0VAzMassCWMjVgc0BbOU7hIj/A/aTHGG5SCnwyKiCl4DokwEAGmbfqZCEM9m480yyKcoUwClZeal/JExUKhvGvQbGUm4JR4BeKQ0gBzZm1SrmMhRpt/vbRIQb7ZMb2DInXWKE0NVDjlHdZCCSc1einGmHuT/PlDClhc5FVCQIAm/5CocTfU0mKmJ+qhwz1CtuxSR/DDk1av2ZBcAxs4cQBknSGFBcb9fcLXPDu8CYUZ47qJQ2/R6y85TkDkt0ay62glpD5L5dLaC7SQq3VBe5d7fLYDLQPeB5T6kVaXtnA/R1aVMetETTTei+5gOOMi88idmv3rvVRtui3L3zmDIbci9XqaFjO9CZ/yvGiNNbgtYNSySs48gxpwubXWusis6fiDDLjjFWUIWHigHbnGC9/mdApX/riC/jhK1REePsuuaMr9x04FVIA9focYo6/9fsjDJgb7onL59BoUHyk1jTQ7fX5XgxsPUFFhqEv4cfTKaVOm5RSmgKCYze2sBGzu7wyp7FxnpBDzz77OVS5cltKiVqF5n55/iLsfINqVbgtAmJMYMDwLXGqIDmtXrItLNdpS730wgtwKhRL/OPvfRf3GDM+UwFSfpbSsGAy2qOUxqcCTgsZDicNgoLTzYAu4lJCGND8nEw5Tslrxy1gdeI0nQAkQVHOoLT+KfBJrQUUf1YCKHExc8kSqHFDcqnkQvL+NU1zoop7XDZgGhKWPQN5m8lMZvJ3SE60lDI1Do7l8Ba2aUMHbIppYIkB2t55730cc7YoFSaOWKv20xgl1uQlATg5qJZtT3Cd02dN0ypqG/pZUrAraK1QKNckheLfkKYuIDi6wy4MLj5zZBVCTY95HAyosEwYR7CqZE3USwLRLQroVhcTJAvkbglrDmtz1JK/vbOP3nWyWi6vX0alQvO1uRHjeJc+f+tDjaBPp4dRIuvA9gIsr1Eh5f52C5FiK0DrwkSuNRyc4b6yoxv3kSZ0uvbbEfb3yJqIsi7mF6ZvpxlxAWKSJkg4ZRTHAiVmMMmyrKD5MQyzyEomQQJ/SHN7sHOM5UWyP5v1Bnw+jU89s4hOSBlFm2FYh30gYVYb20uRMbyx6ZSwvE4u6+mzDmJ2lYQEYoY96fV7KDPLsudqmKWchfVkWeX6nSTLkAiaG6fcwD0u57LrC/jSlwn+eK5aQZLm1sE4RGCbEtWJ5K3JcyINWbgqOfIFVAatxsHd3Btq1Kq4eI4ydR9eXcXODllKqcpgFEBoYuw9KfVI6qFJUSmt07maVfS7RSmgFbd3GRbsPMMqDWSc7eylKVwGwktdUdTopYkuXLZssrcN+XrQsDnxVC+7WGZC2LpnwuXkkDTHyACGYcLk6wshCkgewxgzqPwsOXHnSm6CtfQEu4phAhx5z0ZDrFa5B8fKYLG7U1MCIT80KSRSnrSRUgjymc8SGOmDA5dpPE5FClUYlpYwYHHlqyckOCmDsshgFd5Lhohdr9EQKMlP0RPWJqpww4kQcVrUrrpYfYrS8EmSIXU4Uz2VFuYAACAASURBVNiroX/I9ORdH8Eejfm9N69hvpbHQCr4/Ffp+qfPLGNukXZEbYnMXG/ehZTksrR2zuCwTa6hcu4BCW8+ZcPmfi3hANVKnrodYMgKIpUpXAaKn0ZyAHwAyDjm5gcZBPdfRWEAg+fZcV3YzBY89EMk/Kyqc1W8/BXa1FunVyE5k1WdK+MznyO8pBJXLNdqNUTMmmxIE8LMKbKNYjOGcVhgDbmehyoXndqOA4NhQOIogs0ooI+SswvkWmcqRpfXnV9v4IkmKfhzzz+L9XWKYcVJUsQ3dPEfyjiNs4YSRkHFPdF8qsexmFyUUkW2yjEN1Ep0kJzf2ip64rbbfWiOg0phFZtYSjHuEZ1CBB/Gi3M2FucZ/kRlkGC0VDne2nRffNj5MSynXFwzYlzwOBq7b5PFkzmSpG1JeDaHbUoOSl6OWyULDCVpiOK6UlooijGhJ3wy+chxzty3mcxkJo+VnGwpcZGVoSUVNQCAYQIc9DSlQEXQifvlp9bQ8+n1j+610OJCilBpRHnk3zCLkvxM6YJ7fJKxMxdDSHAdIzxposQUT1VToMrgY/MmUSsBgIUUdg69kKUF0t40ssLZFd8RMEGnmDYl7CYjIHSq8CmGj85Hx7CHFLytRfNIueYj0jFURqdH5yDEgIO6Z88sIOKer/Z9yjjJ4SFcNvfOnHkWy+tk7XRCB0dHFMBUsQuDo63PvnQaBhODKvgImAxUwC3M4mlEcVDVMi2A53M4CpBxDcloOILB1kyzYRREjXAcuOw+rdgmygtUb+VVJTJOhpjKgdnkhAafxJZpIuHiSZkJpJw17Q96iPiawpQw2SLSCnC4tce0LIwYUkVKB8NBONUYF6rMBhObGPo076Wnn8cmW1AXzy7CzutxLBMWrz3LQBH8FxAweW1KMbk+ZcEAm5/2GhnylrVEp9D8dwMGyowRfuWZS4jYpPrOD97CYS9vuxIwcjfwIcBqJ0qeWTMlTH5mluXCMnKLcpzNy7KscJENaaJao3WqdAyBHBfcgMihZQRVLdFrUdxrcadiEtJk0mWzCtdUSqsomJRSFPWHGhKPYgI+OfBi502taZHWh2kizRk+YUJztH+1BPzas5QtWbYUbhxQ3ONglKCTMnGAMsAsSEhFCp27eByxNwxj7LIpDba+UTZMOPxZRyjUDJq8pilQZv/UtcxiUSVJAv8RVaOTspCSaR+t1nC4TVmzw+0DpCXOXsR1yB0u2GynAGdykNZQPs/u6zkNIyY/G4dd7N+iotKs42PpDGfoGBjei1bR7jH7bnYP88uUVl+Zu4wsJNyq+zsH8Cr0281FB2lIz8K0BNCiSYx6GZJw+irgmAvo0iRBwHHB0ciHk2ffzDLYe4MWBiKO6UWZQsLZL40MDrupqQgRs/mfRQoRI2/GBmMEmRZabdLmc81GwX7b2jtCyNTQC6sryHhRt/sd5P6QNEzscXpfKY1MTfc8NcOmhFEEjw+Mp85vYa1J8+fJDJJdNkOMY0BS6yJEIYUo1rvOACXHLlvKfWZ58WCSaYw4LjMMIwScMc20iaDoVLCwukHlIvPNOzju3y+uL/iwJ1iX6ZVSXrBoGEbRV+e6FkxuSBYYV2tnWVaUfZQsDxaPP80yCD7gDYkx5baURUiluCU9xlkSYuzWSSHG8TVpTLw/8VrKcRmAlhCPcNBm7ttMZjKTx0rEpynYmslMZjKTv22ZWUozmclMHiuZKaWZzGQmj5XMlNJMZjKTx0pmSmkmM5nJYyUzpTSTmczksZKZUprJTGbyWMlMKc1kJjN5rOTEiu4Xv/aiBgChNGSWd1MDHsNs1uv1omp0MBgUbSOubSFkECrPdmFzi79TNuEwlkwYpgiZLSOMuG1CClTK1MLhuG5ROR7HMRyH2geOW10cME2TYToQXMFqGEbR2BnHMTodqgbeu7/9yDLZ/+F/+d80AGwt12GmjLFsJDi1zqyi5UXs9Oln/uIH72LINEDVWhN/2iLCROPyV9B/858DAL5u/gj/6B/8QwBAUGpCKWrLMHm624cd/O//6/8BAOh1uvgv/6v/HABw5sxpvPXWWwCA8xeegMfNtpVKBe02oQ4Mh0MsLRF1ULvdLtoynvvcS48c5+9/PNIAsdXmIsSYvlFAYLLTQfz/XMI2VU1cwVuqkDBiQqoUwNXov/vC4onjnDMnsFYnSpAfxmU/+d4nyRYf1vIx+X7+rxIP9OQ+wALyMHbZT4qaaILNq8RbfvjIZ/kf/E+v0LNU2bgOXGTIciA9WLDyptrRHsoxdQp89uIWese0f37w6quIuR1rbm4OrjsG+HN5XV26RE3WtuXByOmrhEYOJvcgOJ0uqsInx5ZlWTE2pVQx77/927/90HE+giGXNpNjmEWLgGFYBdj5yB8UAGFeyUGUKxdToFIn5WJLE2BESlsq1Co08GB4VACre9wjpADEDK6PmADIAFB/F5fjV6oltFo0liRNiw5urXWhlJRSBTj6NPLZS0QEYEmFw11mgl1ehyqmR2K+Qb1Tv/ZLX8cB4zdv7+7jPCM1Dq02lk9xh/pegh+8QSiM3sIGLpwjRMNKk6A0Xv3oDbzyCqFaCqXw59/5DgDgm7/5TTzzNCFchkEAoXISAYkqz0XFc1Dm17axUNCMTyOmkdPiTBjIExtGTPy/ACAn0Qc/RVtWIXrypX6g50k/BKhDYMwnprWEzNecFjne4CPlQUqgn75pLca3oSc+Mnk3Y3jByXdAfW055ffENX7W1BT3IvQEc+/E7+kHlV/OIjuNCG64k3r8kKQeQ7YpZMgRQmzXhsXQsX/6nT/FzQ+ITnw0Gk0A+o8nOE2Tgr9t7YevAwC+/vVfxNNPEUpnPIFwmaZjBALDkEjT/P30AUWf73UIAsY7SU7mfWPlkykgZchP1y3BYGwWz6ugVqONOByNEDPGi1Ny4THnlaGBKGAYVqHR69KJr7IEFkOjJDyThjGm9zVNAxEjJaosgWK0SaUAhwHw0yAtJiSfCPqM+lTNjZtLhA+UZWnBYydkifgEQXDAZScHeNeonyUsoK21JZy3qG/u4+MIzS1SKJWjCHvMMOG3W9AbBFOSW3ubp87g1BbBZ0SjIZ555goAIAx9eA7DCJdcpNzHdf/2DZQrDOlh20hC6kMzIJCq6XvfCq4uPV3zp5hUBP8flJL4xP89oIYKBfWgciq2lQYED01oDSU//Q38rDHqyVcTjaXj741B9Cd/QQnxU6r0k9blw+/jwauOL/+JGfoUa9aUk8qRx6ABAznXWoz+IWE43bz1Ho63rwEA0v4hqi5TpperiJMcs0wX+GWGYSBnEb11m2B1hv9yiJCRSJ955sqYEUUlxX1nmYLW498rxqUVwhEZOK1OG4esA36WzGJKM5nJTB4rOdFS8lzqUk/CCFLmZp5AbroZpizMPCE0PAbGj9MYtpVrUo1qg7rkTSPF7g6hPDpOCTLvaM5NO0PA4M7uJI0xynnKpAErt7xMiRq7hnE6RMQd2pZlFdo7iqICLGwaSWNyO30/gMd+tSmNcXxAWogZQK7X7mCZYzpuycC8S9dc96yCQlxVL2BjgVy2XhBCMf11yvf61NNX8KUvEU3SQrOGX/rlXwIA3Lp1Awe7hBdeLTkIRhTfanc6qDfIIsuytED0S9MUPp9ATzz7wiPHmZvzYhLR7AT5Wz2xHmEUKK2h2PKV/297b9ZjyXWlh357xxxxhpyzsrKy5iqyOKhYZEuyJkuC3IPRsA0DgmH79gXcfrv3Ahcw7Fe/+SfYL/flGm40YMttq92G3OpJ6pZaFCmS4iAWa2KxsqbMyunMJ+bY2w9rxT6n2GRW1oOBMnDWA5lMnowTO2IPa/jW91UVhDhaaKM+3d3OP07ndyY5GAfSPBRAVfWILUMJY6OaeG9CQBlPpJ73yuibAZjKqUx7o/oxt9M4EVO0KNB4KpI3u55rchKPSgnYglIYtz98E5988NcAgLizA8352+X5BlZXSJHZ9gM4Dj3X4XCA0sgmSeQFrYlax63bG+C//+CPAAAPth7iMgsutNtzcDnikYJYYgGgLDN0Wa5rb2cPu48o5dEdDpBWh6sdH7op1XGokkDEwoxJMkbCWmfD4cA45UorE0pEUWBCvyD0YfEGVUGiubRivno4YJlnTl44lkTBsXKlKyytUljlwoKqE+1KomBumKoqoZiKtCxLsynleY4wPDrz5Hu/+iWNbZwAHO8GnotWk3JAC3OXkQzI5bx/+yYEh6lRGCJwRvxzAIu5Vuy5RTjMl5Teu4OtbSLSD+eJ2qU7yvHcc88BAH7r17+FVpu+Z3FxCTsP7gEAentbaLG4p9QV4kGPn2eAnDmKBIRhjTyK1aoXWlSYTm9/1g4hYHRBeVE+liD69A+P5YP+5pU+47fTv566FZNTUpXheSqyEuKo4qJTVKuTzWeyQUhhASxkIRzfcEZZzSUcu/xVAEC0chEPOzzH9x9A7pAQqd29A5HTQqtTDqoqzQZFG9HjHGGfuhP+z5qLaYq3CALTYpBPHiZtJkoruJwBs6oEN977CQDgwzf+HOmI6alLwAaPWfpYXCaqnOZC01AX9wcRYi5ORY3ICBDUaRitgR4rysTJEO+++wv6/34ImxPXS4sLCJhtc7ezg61tEsro9nOkfMRZtgc/aB86tln4NrOZzeyZssNLVHyiNBoefA4ZHMdBoWhHdWyJnInjIYQJ5fwgRMFJsXGSYMwVzrDRMDpg41GCoEU7ZjzmxJfSaLYo7MryHDkTgWktDF90lqbw62qdqkyZsqpK8xnP80wl7ih278EdfhgSIZ8M2XiibQWhYTs1qRZ5i2SVkUdq+xqan5f2AsMvvXHqFEIuBsDnse1t49VXiee62ZozarrH11aRDk7TvWgFjyMWDW20uhzbmiip6klV8khmCOsr2KzdRkWxv3k2CT3R46uEwGfR2k9Ed2pPiaz+7GNlcf05PpOeVGZKALBqj6ZCkZEXmqeA5x/RU5oqyU/uc8qkZbwuy/MQNkns4Lnv/CO0r3wbAHDwaB++Q/M3aZ5HtkShSrZzDf59qqraMXm/lcggWZVXKg3N4ZMSnypdThGkTbyj6cegDXHbUayWWo8gUKVEKHj1nZ/gV29TtSwZHhhiN+F4sLnI4kURzpw9AwCYX5qDxWGxUhpJwkyrWYb7D2l8MSv+rq+swmOl4DzKDPynPxzg2iek3nz+/CW05ym62d7pYq9L76/RWsZ8g9IPrXYb7Xo9fI4duikVHDJJG0g5ZJG6gCro50wrODxYy3XRYI0vAQtVVVMYVkagrt8bQrBeVjoaocmigAsN2pyEymHV4VgFxDF9z7gsMddmFjtHomCIQdDwEI9oEggpJtU3TfiRo9prX6CqmdAwL0loDY9pXYVUaLN6w/nnLsKuBRUsCz5vsn4QQnOYKqQNR3P1ptGEiEi55GDITIgX57C8SGNP8hJZwnCHloVz588DAKq4gsUTXIvK6I0JrUyuBVqZ0vBRzIiCQkLoqRzHZ3zWEkDO2mLCEnADes+V1mYT01MLfzp0k09ywCdRy2OBoYYAdA3xcJGMKVRK48SwYz7J6mqs/NQCn2yOJaRiRlHhohXQ75fSLbjXKPRJ+hkuerRwhlaE+6xhuF0uYLT8LQCAn1NI53auw8mYwlholLXMNcrJIKGnQraJSSkfqx4/TYWTdSyQdzv4yY//gO517w5C1g0M/GOmYhs0IkjOhTUdG6srFL4FkYfApTnuez7abZrjRVlgkYUO7t2mf8/Pt+BmrIgyiJHzHtAd9NHp0fjPX3wBGydPAwB+/ubbmFuiCvOJjZNYmKPNv9Fown4C9GEWvs1sZjN7puxwnBKfrFmeIfSYFD4MUDk1j7IDmyt0j/b2EWdj/kwLPiO3yyIxOlNQpeEkDhyBinfbBsu15EmOnMM+y7bg16dzWaI+T8IoRMoS1q1WA+MRJdMCP4JWE1EC9RRw5EtnzpvvnEjLSCg9qcbUkjJySRpPybXtCdJd6PqQh4Q2GnSwXRQs91Qe0MkfRU14NaG44+Ogw97jIMMcJ7eVSCB0Wl/Q4LSkkIb7WFUa6gkk7NPmgDxMpeypalZlAHgaEw+jv7eDv/g+ncDNRgMXn6fEfDDfRrRMp17YWEDF36/FBJI58cL0Y66BOQGnblkLgUpOPl8DJrW20T2gSuTm7Q/x1a/83SON0YSTeoJBkph4idAKmuWlimSAgx16TwfXFb71yssAgBOtUxgWNK+29q8ivkP3YZU+kktUNe2tfAcAkH/yAcJbPwAAuMPbkEX9PVOJf1EZT4mgUTV4Uj8WZj4NC2zcoZTDz3/0R+h36F4tO8CxDepCcMN5NFiWSiNHyqHwaquFkpPYeQZ0dohD/cqVK5hrc3FICPgBve/1Y+Tlx8UQn+wQ7ikVAwz6jDcMh1jbII/s4oUNXLn8RfrOSsJ2ybu1HQGLee2llJP0w+fY4ZCAelPIUwNqtCwLAYdptuuh4DjJcRxo/rJhtwdb10J4FaIWl/OFjSQjV3hlqY2UXd2SS4S245gNJ/B82Kb8KkzLSb+fI+Xqn+O4Rn3jsbyPliiOSDQPAB9fvUH3GrhotOkBLy0vQfKD9L3QtIhQ4mNSPak4lBKWoLgRlLMxGhGeD5s3kTZDJlxLQfPfPdgb48YDits31gO0WIDRsnNCrYLCrbraInRVfw200lDl0SeyVCTmYIv2VC5ITaGoNSzOi/X2H+GDN/6Sfp/muPMBQRxa66s4zWDPr3zjNyEEjakSyix8+djiqsOZaeTzJAOlhTDhYJWPsbNFpePVlQ1UXOna/PhdtEIKM/B3Xjt8jPU9CDHJL9X/DTzWOiOrCiqhxbp5bxPDNZozdvZLjDucG0pTPM9V6NbaGvaW6J5e53fz0DkLMfcqXTvtwiq3eFwWtK7zmurx4uXnbD5Psym9+fqf8iASXHiRNtMiT6H4MCyLAiOuXpb5CIrTJq31dUSc09nffYSb1z6i8W/dQSNs8H3Y2HlEeao0p/yx8irc2iEg5fHTqzi1QRXjIPSBhDafcXYPWlwBACwvzSHJ60NQQVd0nQr2E8Eos/BtZjOb2TNlh3pKNdanl45RlhO3uvaatCbAIUAelM/VLxQKFe+wwlFYbZNLeWdrC0vcQzY/P48Bt5/EyURO2ma5GIWJYqdSylQGPM8z7SmqmmheKVUZdc6yzKfqQk+2//Q9ClOee/4CrrxGlZYo9BGFNM4yTaE53PKmql9yShIKADR7M57joctu8fBRD83jVO0YsNzQH//4z9BnqeADfQzBHHkhx4+9BItPyzLPoKq6+qhRcVigqtLIWgldGbDaUWzrPp2KaxtfhKqlfaAfk7ypr12VGdqcNJWVxniXqjEHg23s9Qj/EtgtfOHVr9FnPG2waeIzppUU02lfbbwYrYTxdh/cvYE3/upPAABf+tLXce/2VQDA3tZdvMUacMC/OHyQ01W3z2iInUZlCQij+6ZVgflF8ogutRz85F3yeMKgZaqwRfwI3vv/FQDwUvAe/X+cwX3QnI6blxEUlPS1iv6kD01PAMfTIdt0M+vTCnh09smTOXliDd0+NZ83HIH4gMDJRVmi1aT7Wp1rwxEc9RQp7m89rAcNm9fjgYjx8W1qRbmz+Qi9HQoJ/bqoE1qAQxHK2sYSlvqkYZgkQyQM4L330V/BKhmM2S8N4DdPXQSMc7T9lilmfJ4duinV1SwhBAp2xQaDHBaL2ZF4HT3MIAhQxLQRLS3Mw7Lpb50qRj6gF5UME0QsK7y3tYcea9tLbmp1fNcslqoskHAo50rLVPaiKMKAr+c6AeIxfabfH5kQz3FdlPnRq1K/vEaTP1po4hVNoclo0DVASkukpjnYsmxU/D2lLiDYjS80sNNnzbj9LuIhbaKNIMSKpL/9/d/7DwCA13/2OqrGKQDA3Lmv40pIZdSk8xBFm2P4gz3kBeXolMoMbKDKc2iuBulq0vS48srXnjjO2zd/BQA4vn4ZUtQd35OwSlkSJef0br7/DmRB73Ol0cDm7jZ/PoLqUxj4oz/6Q0QOhVUvXHkZZV1Rq/uyBOX3AALD1iA7IaQBclrQKDO63o33fo6P3v0pPf/+Q2zdYyBpv3vkcFwakOQELj0NTdBSQNf3YUlz2CQQuPmIxv7rr13Gq4LCkwf7Ce7tUMh2kAyQl/SO5wWF/H8r2MVyg3Jsn9hLkJJCKb3/LlTJUBcB1EHJdLVtWh5bKfVUG5PLMIrO5m2zD4ftBo4vsFBqq41lzv0FgWeYOHYPDvCrX9GGFjQi3B/Qz+NkiPEjOmy2O/uwLW4079Bak/sark3f+de9DxF5NI65to+AsSuNR49w9d3/DACIxznW16nKd9ApkTl0X1/+2lextnb80LHNwreZzWxmz5Qdid8jiiJkMbloZZkydQFh8czGb0m0GQxZpBn8GqiXxnh0jxRB5+bWkI7opOn3BxhxSNJa5bBLKuTsndmeC5c789PB2LARxHEMh6t5liXhcVVQqdK0Rbiug+oJLuK0JZz9LCTQXuDWklYE16qxMRIWtxKMBn2TaBcALK74FcLG//iLvwQA/MVPfg7HpZPhyvMX4HpvAAA++IA8lZUTp+Cf+go9n/YF7D+kBOIbf/EO7C+cAwAM93qI2LVuNV1Y7B2pqgSqyc+P4VyeYP19Ch+rdAA7oHYfpQDB0utaOujwZ25/8BaaDABtex4O9rllod/DQkzfOb8kcONt6q/65Nr7aLC7fvk1Svw6gQ9VV9YEjLeTJTmSIc2nUe8A9++Sp/rR2z+FSuhk3n24iSF/xo9CSPto46xbjR4DbkppPChIafpnLMuCxZ6StAO8e4/mwUfOC/jS7/4uAGBj6wDeLz+kv727iZLTEiXfpxru4hWPQttTUYZ3QWtglDZgjbglpbKg9OEUM4/zEj3Z5hmoNB82cHyNWCiiRgNLS+RpKz0JkW3XgmawZZ6luHGD5hssGw+7VFG7uB7iynHy2E8sh7i1RePb22JvLyVALwDs7sWG/kagDwmOeGRlFHodx0FwnUHJVmSqxPfv30S7TfP6n/2fv/OZYzt0U6q4r8qWgMWNe9LyUPBmEjgWfJ64lmNDF0z4Nh5BWfWEDhEzHUj3/hZsXlx+4CFkIqm5JXIzdw52JmXUojJuqe1YiOMx/2ybptnRsA+be50810XODa9ZlsNzg8OG9ph5Tbre0tqikTS2pW2qQkJrKMMh1UfGPULpKMbDXYrnC9vBW7+gzefe7VvY51zbRzfeh8Pl99V1CtnWVk9hJ6Vxthc1rt8gYre+HOLMPG0Wv3z7l+iktIGvLjTx4vmzAIBXvvACNFdSdJWbUPIo1jmgxXPnkw/w3ItU2hYygFOHUlrj/uYmAKDX6+HkGk1SjAvTRKqr0oAa5xfmkfVpE/vwrV8YMr/ux8TX40cRAq4mQmn09jgPMRzjAYdmo+EQcDnEK2NIRkKXskTDo0poUmkolRxpjBc4jFVao+Dwu1JqqndyEiZN035Iy8WIK8m/94N3gHnKA7760sv46gL9fKbbQTykRTrcp5zTaH8buk8bthvOo5lSf+Of/QxI7zMKP91Hqet6rDB5wE9vQp8GfB5mX3rlIgBg4/gxKN4IhoMhQq5SVqoyY5a2hOAOhzjOsXmHckrtxVVYjG5/7eU2XlqhQ+XP3+miyU5Ac4nWbjHShl/L07FxAFAKQNFncpTIFKV2wtYcLrxEudJLF1/Gzn1CfY+7dxE1Dt98Z+HbzGY2s2fKDk90M8matoTZvpS2jAeRZDmW27QzN5oRHj6kpFnlCFQMnCqDEC53BXeu3YLkk301DNDgpFzdkeKGIQqm+SCGNdqao0aA4ZDcSdtxUTD5WVVkEBXjp4REkXMCusoNw8FRbHGe7m95eR6asR1aAeBQQEIYKkLLseGyN+UGAX5yj7yCd67fwN1NcledKocqybPb6aeYDygkPOiS56PvbcFbJ+/QlQmu36Br2CfWkQiuTp44hz/9/u/RlxYprl+nk2bj9DmsrtBnimw4od44guUJeXVbDz/ChedeAUA9iCV7O9KWGHFVJ8szZBxudfd30efwPQwj2HaN+clRsde0HDmwuH2je5vC1CyJUXJvpNZAwFTHC80Q6oDaF8o4x4Xnqc3Hd1cwSsgLvbvXQa+g7xRRCb95tHH+u//v3wKgXq6CWfryojR9lFVZPkbNWtOFKDWh6eh0Oibc2O8dQHO4HLoCuxxS3n9Ez7IZNqAWuI0qHuFYg35/+dI5vMNjyfZuwMr4GYvKAE6PQh/zeba0QF5kqx0iyei+c9cx9MhZnhvq0KKqzNhQlRCgtak9GwmvmSsvH8O3XyQQ8f//37+PgSCvKWxw4aVKIDjKqdBEJWvcnjA5HFtXsNnbbDYaOH+aUhGnNo6j2yMvvdSLaIeHtwwdHr4xfy8sy9DeApMKgqoKjEf04PMiIy5l/nzJeZpxkWNpnhag70XQkq6poWAxMjxjtGmRJ6YkbUvLxMR5ksKv0aFCmhCvzHJAcVgJAZtDRkgLaXI0dx8AQq7+FUkKptGBVJhcQwpU7HL3RkMIfi7HFlawcoy4aT74r38Ijytax49toLNJi04IgUZA19fM27QyFyJapM38rZ/+OYY9Rg9HFr73Q4InfOvLv4Zza3TtzTu3cY/LuFevX8OxY1/h2xKwnoL2N+c8yL07V/HJLerd8qxlfPyLvwQANAMHsmDwXRnjzQ/eBQAsN+aRcLWnGo2wtELvsypyjDlHuDg3jyrnxcZhNJISYc1L5LtYO025D6sc46FPE3yQFVC8YTQbEU4sEef5QnMO//GHf0bP68IK5tYPp7uorfOANngtBVwO4RcXl2A161yTA4crhgRtoXtWU8yLqlpAfQrtPtpFn3mBRqMYFVex2g2aj9IVePd9etfvv/crWDx/3cBBoOi9q3ANmUeLXCfbsBIKYzUEpjNlT5NT8tyaUqQy+U6pKqhykm+s56xSGuD3ZwkNj9dS4eKjIgAAIABJREFUVo2gJTPKOgXA4FrXEabybFs18DmHZCrQSklz51JONvNKa2g+sBKVYMj5t0yNITgneDCIcfnC4ZvSLHyb2cxm9kzZocesqPE4WWE+6Xo2nIBDJtuZYEFgYW6OXL29/Q7CJiW8XCEQNenEWphbwLhHbmxZpBgN6MSYW6WTt5eN4LG340gBxbv1eJxi/fi6ua/9PU4s2i48h74nTfsQDOuvSgXpHJ2EfcSh4d7uHkp2Z0fDIX7+HgHkLM9BxoIG8aiPK89fojFohYWFeX5YEkNObi83NFw+YfwwwDyzYKYcAuW9XfQSIpbr3N80bnGnt4PtPTonhoNT8GoCvbLEiL2shzvbE/CmkE9FXVK3YPQ6j/CIPa9vvPYCLn2LksO3P3ofo4fktdmyQo/pTdqehbVzlKS/f+02spQxaAseHPYytS2QM3BOuPROMgxgcejjWy4afLpbyLDMbKR7wwPs9yjkEVUGzZiltcVltOtQJM4Q8M9Psvd+8ToAIGw1UXtBS4vLBghcFCUiDiODIJgAOrWa4od3DJ/6fMNCwJidB0kHKycoFHedZf67Cg63dty4fhs7WwRe1J3E9LQ4dgDpkgeIsImyJM+rnBZ9eMpIrk6KF0VR02lDQKHia6qqMBGNwISNICtLVDyXRVKZdZ1ZTQxz8vTTOIMV0A3VLWKqSCE5z+IoAS2ZpsixDX+9dDz0uAjk2hrjEaV/RqMRHEmfaQdtfP3ry4eO7dBNyeWQTclpkFcJh1HXAIzL63shasrPpeUlSJ7Qrm+hYqoRW2ossqJHd7yLXpcG0OASoawqNJhuocoL06cUOQHGTI/geR7A6HLPcjBkwGKe5iYfVWkJyzp6WJPyi+yPhhhyHuDBg228/yHlRpzQR5wyQl1XuHD6NACgqEo0gjpkW8K771Fp+4G2UPLzWogiLM/TxtUtaWMZ7N7D9ojyT9kwNdxGoeXAZcDiJ1ffR2ePJngJhREDSeMsM265LcRTUahWOS26TFiGDbRUCVzmKmqFNta4MnJmOTQN0U7zJC6/QqGkSiVyhkTYUkLzs9vv7WJ7nypTIfdQeboAONzxCxf9Dh0moojhOXTtPC8Q58xPZUfodmlTHHUewWXqFhlEaC0eLUcY+lx9KmB6EdNRioAZJgPXN9XTwHYRRtyEOoW0lpaAZqqeshBmvgktEPBmtc7yW8P+ARpcgfZsQFqMatcSBTecF0UKkfT4e2gN0YfU5FAR+il6EGByrHGijVRZkRZwLP4eSyKrQzktUPL3pFlhcsXLroNHAxrbf/yDW3hzjqA7wo2weoo20aUVqvreuXEdXa6elkNBDgkA0XCxdpEqgc9feB6v/5jAr3s7W9jcpDzSaNhHLpguRkt0B4cDYWfh28xmNrNnyg7n6PbpxLNsIGVZn6JMkNSSSdIxrmMS5/AZ27C2fgxZQi5qnI7QYNfb94HhAbnnUIBgd7B/QLt+HpcY1GKQjgtb0m4cj8boM2Znfn4eHnfv97odHHTI9Q+jeUMElhaT9pejWMEu2ShPsc99RNeuX8fWHlWiFldXjKd0sLeD2/c2AQCR4+EYe3nf/Qe/hQfbFBJVWQnLqTmRBSqWpypjHpsoEdSJ43EXkk/OBREi5BaOfp4h4TAtVjkS9kgcd5Ik1E/ZmqC56hKPNZKUQYr7d2HX76cR4MolOhm3H17F3geEJdo4v4ZTjFmyvnAWb7/+JgBg2N+HXeNikgG6O4Td2edp1Q4c+DaHAaGL3pjGlgyHGLOzMM5zlDH9vsQyfBbgHB88RMWeZbu1iqBxND/i9m2qUlbCRsDUHfFohJ1H5HU2GpEB3+ZZhjkOIytVGeZSx7GNvFVVAVFI3n1ZFrhx4zp/ngYwiMd47yYBEPcPdlCmNBZViceEJrWe4KTqqjKFAhOKu6dxlcy1ixI5zy9U2qg2KT1hTpWWRK0/WlbKFJNW2yE0VyjffHMTirFPqxdXMWySV/1r33wJAHD8TIWtuzSnx50Yo5TTHBB4KKgFaWdzH3GDPF0lC/TZA7a8HJZF863lO3iHAgr8k88Z28xTmtnMZvZM2aGeksWc0qN4D9Kl3dUPbFMudB3byAQnaY4OS2ULRyD0a5WEDtZWKD69cPE4PnyH8g7xUCFlZHhR1kq8FoYj2l1Lt4TgbX8cxyaxJ5SciFjmhWHpsaSFOtWVl1N8q0ewXt3l/GgLd7YoDt4fDfBgh05XO/Rx7gJhOPb3dyfcUhLwHXouv3blAr7+DeL6eXCvi21uZOx3e/DYy6o4li+tyggtLrR8g6HxKgWf8yCdwS6GAXuSWWHySFEjmhL/qwyH1VHswot0EnZ7MZI+eYEffrCPX+xS8cFJEvyr//f/BgD8w1aEucW/AgCM97cR7d4CAFxspLhN6Rk8uHcX1sZpAEBRamTc2jOqG7DHwuTcpOVjGNOgO719jLmg0Bun4KmF23cfYGORPBfHsZAxItmWluGfepJ973tEuGY7jmGQsKSYvDPbgV2rOgfBRPlITlpULEsa7I1SGgET/FVViQ57spKR0MJtIePSfxwDaczJ68fkTBQmHFyTf07nsQjpffQ5O88eOlQKjz2/NC3gu8xvpSpUHMZoIUzpX4gSFRdWLFng5RP0fu5ul+h0NgEA3Y5CdIZUoxfmaewXFteQnue/Kwtsdeh9fP9PtnHyDHlTMpDQJwj9Hloerl8jSMm5c8AXz7N6dJbj7ubh3v3hOKWaiiMM4EfM/etIdLdoEaMowd0XsC2YxZUNBwgscuvLLMeYF3274cAP+GUOEpQZ63oxo0DUDrG3PeTPtkjyCECRl3A4wz8cD01ysoSAqvmibcDl/p5yNKJGtiNa7cK6jgeHqzTDMkfKG0G3M4BUdO3VuSX4vBEEVoYHPQ4XGgrLy/Sd77w9QFJy+4tnG9a/mtpDlQKdISeLoyZW1qga0ekMsMc0LkmuILnCkZQKAU+2VhhB8P1mhTZd+EexRWYRXFldNiTmg34XewNyv4cPu7i3TRvU8aXj+I1vErvi/fffQWeL8D9yeQ5rS5S4//j2tfp8QgmBUVLLttPYc2j0WQ4q2dmHxe9qmPVhhxQqicBBlzex8ShBxqIMx5cbiFm91QumyPyeYJopOghuxYdamhganLJUU6qwyhRzhGXBMZuSZWTqNbQhO7R8D3m9cVjMoNoOEIZ0eFuyC8XvRtoGZgcKzQ6vkpIe3dErxtu7lHRuRw4Ub2xKTAQLKl0ZhkloYmMASBKp4nVy71GMsU3zrek3IWyunPkCfWbefPsq0d3o3S3s3qVEuA4lihbd6yhTqLYYdxUKU+DxQxsQ9Gxvbt9DMqLv1OMSvWSGU5rZzGb2v5Ed6inZzICVjEpYfMp7toOIy64yV0ZzXDoWmiG5aI7rwOP2h6W5BYTM4x2nKcaM5bFtB7xJI2TY+eJyG70OhXcaJVHMAsinNMotUUFwV7JyHBTcGajkRMnUsgVUefSwRtTuXpHD4bAiLBXmuPF3nI6w36Vyth94iBkZPU77uHlAnpJMFZTFJeByjMEBt0ioDE328urDfjhOkbGL0YwCnNqgcnu2vIIPfkWJVLvZwtpxxm/duIWIE8oLrRZlX83NH/10rWNGjRyaUcBBQ2B1nZqAAxmaTv5RvwvBlMZf/PXv4tZV4sbJigzuW3f5bwOjaNLr94wY6YSPeqpTvSgMz1KwFODKl4m3anlhCX/5p5Q4f3R/Dw87XHRIMxT8/qPFsAbuP9EsnoO2bcNlDJUPhTzlroEsQ1DUz8GBx4KjwpJGsRmOazwlVebw+JrScgFum7EZ0hDMH0MdLpQCmERg0zotE5WWv/H7x34+uv3ghz8CAMw3PbQYK9eIPMy1qDjlB4FJc3i2azzjqtSmzaSXxNj1mAI4HmERzGtWjHH5Es3PpE/v+trdDnY2qXil2z48zpXMr3sYc/olHSTI+OdR3kfZ5tAwrXDzE1r3UdZAxiHw59nhOSXGavhComRsQVrkKJkNILBs0/JRAXC5MtRqNYksHcD8XAsuY4biYd+QuNm2DduZwNYBYNAvDC/28soybI7btzrvwuHwxQoc5Ly4olYDEYdyeREjHpL76fke0vjooEKbZXb8OQ8D5oWGVcJu8X2jwk5F1T8hcmxV9FCXVIxbAwo3tj/ZhszIjT97aR3Fr2iz2n7UQ8kTbqHBYgooMcf9difXjiHkRfyNr3wRDcZ//PUbbyL0TgAAQt/HKrdfrK0swzI4pSMPEQBMlS+vMjhMfRGPRwbDYvku/vCPiFnxytlV7O7Ss1i59A0E87Qpvf36j3Bvn1z7sBkhyzjXFfooGW+1uEr3Ki3L4Flcy8L6OrWZnHjxGJbWaKF7wkaPMWh/svtTFLz7DDOBlVP0+ZWTCxDu4dQftTkhhaiO4yLwWOxBV8hYvyzLNSRvEF7QRGOOnrESdg1HgnAD1DC3Mh3D5k0JWkOjz5+vZeQDKMbhKSHNwfh4VXSa7xKH/Hx029mmkLtKIjx4SFVPSEn5MADtdgtRRAfZXHsOUauWCxPwuCJ6dm0OX/82HXyP7u+gu8/y8qWNV1v0Xvd8mt+d8y1Ex+n9JlmJIXiTL4ChqkkPNWwOvnSRoeBQUngOwhUmjOyXKAaH5wdn4dvMZjazZ8oOl1hipQdZFNBcCRknGSz2iAI/MFWhQZbA5iqAUsrwS3eGA8wxWZkUwrRl5LlGzl7ciNsWBlaGoOYFH/SMAKMVuJBcxckw2WVtVZqqjBAaDebu6R6keJoTaJHJrTpOhjf3iACrDIDqDN2rrErcL2sKXgFRkNd0cPsqbnFbxicf72DeJs/ym1/82zi+Qifw9/7gj1HWkkj8fV989Qs4c5LaNlZXlgFO7p5fXUL4RVKDeOP11/HJx+RtQWusMbXp0vycSVo6UjyV6uaIcVJxGpuobzROqEoAoHJK/PDPfgwA2L52HLvsXairt40XlGV9uAuMxn7URTziZLwusXySvJS/949/AwAgfAFp8WeHJY5xgjyxBkiYASAMQly4RN3kP/urt5ANuVrnB7j4Isn8rCwsIGHu6ydZENG7tF0L4L+5f/cmBgNKC1RVaQpjzngExcntpbXzkBaju30fPifAM+FCMeOE0iUk6F1p9raksFGxpythw64vLiaFDfFY+PZpmwrfnsJpOrZIz/LF58+iN2DRzlLgxscUbt25c8NELm7gIpyjsTUbbWwcJy+9gQz5fXrH//y7r+Lf/z4R9m09yjHnkfezy1W7jnYR1xgoVaLMyeuP8nk0eGx5VUJW5JEFsGGXtMCrLDERVVH1MdajQ8d2eC8GA8gcKRCF9BAqAWTMohcniWk5iaIIshYUgEbAQLTlVsvIbHc6XVicJwhDFydYovv6Jj1IP/SNVHOSJ6jq9yWF0XGz5KTCoHQFaypUr7/f8x2MR0dnCXjuApXKb/buYchVFbfdxAr38smsQpzQmC0ICG4F2bz9AFmfXl47X0Sg6FlYSYoTLF98bHEFD3d3+FnQ9V46fQKLzHPetDRs1nrDsItln8bwnb/1a/jjN96iX2c5mlwByuMYGVNSKFQTZscjWH1o6FhN+qWEhMPwjSAIcOElehZnF9YhB1SJ68kcq4s0nnDxDApWLu5uDTHs0AZdKo1+n97dkIG2lgvkfPKIysFOnxkI3NRsit1xgorDibAZor9Lf1spoLvPVC/FOqzqaEmlr32ZObLLDL94nSANZRbD5TCykpNqvRBAyrQteSPE3HEqg2u/Abtmpyw9A3Uo4UDwxtXgKvLqQhM595Lpbgt6RAewUgUqJj+DxtThMdESphBvehM7us0xJKA934Zg7uwsr/DS8xcAAD/tDpFxXlWnGawBc5GPD/DCWQrFl9pNPNjmkC0t8JvfItjLH/7gOpiuGzsj2gPyQQA95Os1JGqNEFsW0ILWsad8aGaFyF0Bjmphpw49eAAnrGUk1t6hY5uFbzOb2cyeKTvUUyqKOqEcoOCqg5IWMg7NAjGRG6oKhYw7wlthhDZ7QZ60oA1HT0UNtQB838eQr1kw1alwS7Q4fMvjDPGATt5WM4TDTaOWJ5Dz949GfayvUDJ0FPdMo6jrHo6D+LSpDn3/mWgZER/hfunDI/gFvNKGF5CnaEvLqG+UoQPFyD+5NA+f6WBF1kfdsvz82hpGAzrxv8oKrC9srEFyo2pgA4KrPoEjIRx6Pt/62pfwPnuQw80HmGtSVSUZjVinHpC2gH4KPqWS2xEaQWiAgqkqURV125CNeQ6xhskA5y6TFnzVCuBxCNqNR3BCcv/bx1ewxRWZjZVj2O4T2HSbcSvLXgOqZhpohyYJa4cNVFxN9dwQDre5nDh3Ag9Z5gdK4sE9SuYm2fNwoqOxBPzT7/42javXwXifgLCD8Rgph8hQmVEHtuwAEbdSfeXFi/jKt79Ony8akPyZIhmhz15yJQRGjLk7wZivF597DjkzOPz4RxI/++uaGyxDVdVijLnhM5JSmpaTsiwNF5Ga8qCOYqdOEdWs0hpLTCedJAlaTZpLc+0GdplUsBFFePkSeYF24EMl5Kmsry/grXeoMfz29QQvvchCrH6AT25yM/4yeWTf3HgJ724SGPLu3iYuXiZivra/jN07FAkM8j6sJXpPkd2AyOga6/4pSHYav/Py17DduHro2A6f0ezyKqlR1rI+cGBzmOTarhERyHONnMFajpCwmQ2gKkpDzu55PoSk60QNHz3ueds4TQ9VWhpRzUqnFdJdZhFotY3MtbQlfJZ0Kb3ciAv4ykfG/VxVVZlFdxRbiDn2LzQiniSh0HAZzOZAotGkh+06HgoOrn13Hi73ZFmiAV630LYNwSC+tx0LLi/GVRYlWJlrwSpqySbHoOKhhZEEP396HmdPUd7pzoNtnDnJCrWNCKLiXA80MQwe0epJH4YhBH/naNQzhF226yHkKs3CnI+QQ7YechRccbUcH0Pe3BZPrMBp0qS+fPk55B/wIcMkb0uLi9BWzdoYGSZI5TiGW11rbfrdzl86i6tvEkCvEUbmviotTY/ak8xiufjlY8fw279Jua1RUmJzmypUWZGZnrBW1MLL3OH+O3//t3DyEv2co4XQr3XSxtjtMZGhEkg4z2pxyHny5FnETEmzu3MJfZazTpLUEPCpKkMdy7iuazbFoihR8oGtlHpMn+5JtnacQrCHDx8gy+hZRkFomBEW5yOTa1KVRsmMqufPncQ+iwLs7D6AcOk7dw4qvMxrbLHtol+epjELmgON2II9pneZdRTGEef+ghEGHTo8+90DXIxozrqosHWLeidRWTjFFCi9G29gbe7wtTkL32Y2s5k9U3bollWzmkorMFJGeZbD5xAsCHwMa5CgY8GvgYxpYrqsLUei4J6vOT9El13dsUrQXCHX2WE3T5VAxhSaWlZYXCEXuchycwIUSQbHr0UGKwMQy7rKVJGAyUl2FDvFWbuqqmDxGBxpw6k9QssCSrovy6rgRTW1KkyiWVh6It1jN437r6RCWvNDsyfZbLdN4s/2AnM0VHDAESAsV2KuTe50FAZYYVyTJYBRLbekNXR5dDxWWX+PFLD5VHQ9B9mYQhs/9LHAfYp+BlgcSuqiQsCeg6VKE9afOH0Mm6fpJG2v+njxMnkaISfum60WYvZe8zw1nfVCtlCx15SM+8YrCRoujp+h6508tY6tBxQO7u33ER6LjjRGJ6w7/SXOnKP7+b9+J8TOAXkw2/0BhiMKv0+tHcOLZyhEXV1eQcU0uQIOJM/3rJLQsiaLW0SlKJzZr7nMs9TQQGdliSF7TcPhwFDTFllCkxvE8jDdu2h6UYSA60x4yp5kGxvrfI0SN29SyNtTPRMiNyMXLqN1B8MYH90kyt4gdLE0VxPeVVhdojHnZYVGgyrGl14YYlTQOr3DHFjd7h6++gpFEd9or+PHf04khduDAX7r71MoN+evI+JEd2u+iU/m6Of7dzv4x/+QPCikNvrZ4cDmQzelTNW5C98QkRVZDqHrRtoSbs0IKAVcriQErmMejnYkRn2amE6loJil796jfcxzKT5PuXowjk0loaq0WThCSaPWm5e5mSRZFht32rYsQzjnuD6UHh868GmrNx+JiVYYLGnyNcpx4HKV0Q8jQx9iOTZUnb+yJBwOdz3fMw2gZ3b62NijRmWbOaPbC0soGAZhub6Rb0pLYdRvAZIiB4BGswGf0eVKVZD25H4tcfQw1eKKUVzl8Bh52Wi3DMSgqAoIh0GVwyEixaGxB4AJy6RWWFmgDbIMLbz42kUeB3B2nkLMe0xO1+92Tc9ikWUoK7pG6LVQ8btqBhFE3WwceVg/R6H8yQsrGLAowWAwRHxEzvWafxoCcAPa1E+cOYXzL5NQgusAd27TAm222phv1GUkBy5XifNKmqpylnloRCyH1YpQcnjf4+ZzrRUc3sCGSYZ7W1TNG/W7yHmD0mUMrSfv1RAmagU9pdv3NHzrIVdjL5w/j/k25QHvbm6a3NmZxiloZnv86PoddLgy+vZ7V/HSC9Q0u7o0j5DVm7e2d/F7v/8B/e1pG7/7Tyn/eW2T5snHH/dx+Xm67xdeCfB/fOs0PavSR3OenttPfnYPez16T+fWPXz3NwjeMu7FkD4dzDc+7KAfzySWZjazmf1vZIduzX5Iu+ggHpvQzA0DCMYMlUrB4762rChMv5AXRQYomMfZRNJGFCgY09FqzkGX9PUZhzUZKszzCTAXBhj1adfvFxlyTtTlZQmPw4OF+QWjVqu1Np8pCmWS8Uex5gKFiRASLmN5fD+AzWGqdF2TOPd931xbWBbKWpVvih5DSmk8rkarhRWujqScLM6UMMBQVRWo6t4pbZtqjLCkeYZRFBmOaa21adVRSkMctSkMAHPmIUtzlDHzmXsOLJ/DUamM6KgdziHl8MN1PAj2rKxKwOELCUfj4st06qKqqPkLQMxeqsgV2ozHOogTFDmDDKvKqH44lo06BR9GAaI2ne5Lq22sb9B7yYoM3hFzwHXl1ZECKVOwJgVQsDceKjGR3xIePI9TCIFnBFRhuRC1wrPWsO3aa6pQmJ7Kej6EKDGprJXm/0/mg1ISmsUoVVVNIjYIQyGttTbe41GsphO2rQqnTlIItnHqpCl85HmOV65QyuH0xvt4613q9n+4s49bd8iTdWwJ36YoZjQocOsefX576OFLd+g6rL0JXUg83CUvKP15idGQBjFMUpx2yAv99q9/GZrH/PHNa/jX/+b7AIAgiHD2EuGn+l0PnrN46NgOZ57khahBoEkAiIsMIb/4qNlEknOeQEnD0xJnhXHbq6I0C9SLPDiGN9gyzJMxcwy7nms4p33fwZgXqGVpA7qsMmUqQVHgIWaQpFbScB8XRQVLHq2EDADrp8/yOLUJwQAY7HglhZlIqdaQqkbwKuis1lkVEJwQqlCiBsWNekMkMb3snT16w1t7XUQea8pliQGGaiuEx8BQLS0023WDs2vKyMDjvNxPU7GBnqDfCw6Hs7wyz9a2bVT8/YWQyFlVNSlLVFUdYoUo+Pe2ZcHjqqRSyvBsnThLMA0/cM1GGES+Kf0n8ciE47aMjCqutBwcO04TNgw9nD1H4eDu3h4854jUJajpR2xYnCtseEDMG6aoFBa4j9AOW3A4pJVSImO6m1IIaO6vLDUMNUihS7M51/+uZwJA/XaSZbagpTm8tQa0qnviLFNVrBHftT3NuxS8Nm04RpJcQcPmPKDnlfDZYfjG17+BF16kUOrW3bv4xRskrrC/O0TAYN1G08fGGdo4Htzbwf/zL38IAIj5oFfaglR1brRCVtYULRG++R3a2I+tpRhxvu7jW9fw1hubAIDXXnsZrWMr5rk4dvPQsc3Ct5nNbGbPlB1OXcI8DLYlp3Z3bUi8Kg1oTrT6YQAN2knTLAaGnGguc7S4IjKMM6M8mqYZHP56bcQtNcAnYlmWhgpjaXkBUcZKHA92TItEWZYGuObYAcKImQSkh143O/JD0M4EN1JXiPIsx9ioowgkLECZJKnheLakhZJJ2XSpDWZKa208i93tHRywCsQun8p37j80KqG6ygxsTjiRUcbwAxtjTu6mWYYRM3JWoQfF70IIMSHyOoJV7KXqShvKmSTNUStwSiENw2dZVRgltRpHYUBOzbSBBtOoRGFoQpu0yAydRVHV7A8FJN9e0AwQMelXmtjG25JSwGUGCAEbJ8/UVaUKAct0rfnHDKnak0xy935ZatgcmgV+o3YSETjaeGmV7aPkgowlLLDTj0Jroy6bZhncOoy3JmG5baIIbZ5Zuz1nktWVUp/ZykZ/XlePpUl6E4/3U/DK1worAmY+ThdqpOPAsem+Q18galBxYnFlASvcf/rum+8Y4HIQOdhkErfrV3eQc1iZsaJMWgxhcRuVA406jhAW8J//y3+jm9Ji0kbmW9jYIMWXRiNCriZFoLg8fG0euimFdWVHw3DhaMc14UZelqiYkVFKC5rL4NL1JgvXsqF4gfZ6qdFjC/wJFalbx9iVNvS2maogeIEGgYeDLlViwiCCF9Yl/NhU6IjZr564nz0hPs9i3nzKqkLKm89oNKYFC6JWqTelNE3NJqqnKCmm8wEUbjGPkOfgLEsynTtLZdGl1ePwrHoFJBPaUitAxVWumx/fNoydGxvrePiQ0Mn5wQ4yzpVACDiMHf/SEcYpeMO1hWUawPa7HcIZgEr4FjvPB90ehmNmx7Rto5A8GI3N+IsyR6tNkz3NMwOwLdnN12UJ1+fJKBU8l/M0yoLkTaaq1BQwNzfUH3mpDL+R7dgocbRDJmFgZ+S75vC0wyYaJu9ToRiwKqwfmr4/S9qweQdNxwkkP1fbcgGm1hGwDByiljUqowo+N4tX1eRQ00JD1+GbUIbK5/EIbfIfT7sp1U3xSikTBLqWZUI5Asfy9ZQw1XDhaFx+iehrN5Y38GCfmr5H4yEqRcIXF17w4IesGs2hdZxmqDh/61ghCqayjsdDM4q5+XmcP08h4OryEhYY8NoIffg5xaxzAAAAWklEQVQMo7EdCa0PP0hn4dvMZjazZ8rE0+zOM5vZzGb2v9pmntLMZjazZ8pmm9LMZjazZ8pmm9LMZjazZ8pmm9LMZjazZ8pmm9LMZjazZ8pmm9LMZjazZ8r+JzHmoPYU0MbeAAAAAElFTkSuQmCC\n", 32 | "text/plain": [ 33 | "
" 34 | ] 35 | }, 36 | "metadata": { 37 | "needs_background": "light" 38 | }, 39 | "output_type": "display_data" 40 | } 41 | ], 42 | "source": [ 43 | "# geektutu.com\n", 44 | "plt.figure(figsize=(5, 3))\n", 45 | "plt.subplots_adjust(hspace=0.1)\n", 46 | "for n in range(15):\n", 47 | " plt.subplot(3, 5, n+1)\n", 48 | " plt.imshow(train_x[n])\n", 49 | " plt.axis('off')\n", 50 | "_ = plt.suptitle(\"geektutu.com CIFAR-10 Example\")" 51 | ] 52 | }, 53 | { 54 | "cell_type": "code", 55 | "execution_count": 48, 56 | "metadata": {}, 57 | "outputs": [ 58 | { 59 | "name": "stdout", 60 | "output_type": "stream", 61 | "text": [ 62 | "train_x shape: (50000, 32, 32, 3) test_x shape: (10000, 32, 32, 3)\n" 63 | ] 64 | } 65 | ], 66 | "source": [ 67 | "# geektutu.com\n", 68 | "train_x, test_x = train_x / 255.0, test_x / 255.0\n", 69 | "print('train_x shape:', train_x.shape, 'test_x shape:', test_x.shape)\n", 70 | "# (50000, 32, 32, 3), (10000, 32, 32, 3)" 71 | ] 72 | }, 73 | { 74 | "cell_type": "markdown", 75 | "metadata": {}, 76 | "source": [ 77 | "## 卷积层" 78 | ] 79 | }, 80 | { 81 | "cell_type": "code", 82 | "execution_count": 52, 83 | "metadata": {}, 84 | "outputs": [ 85 | { 86 | "name": "stdout", 87 | "output_type": "stream", 88 | "text": [ 89 | "Model: \"sequential_1\"\n", 90 | "_________________________________________________________________\n", 91 | "Layer (type) Output Shape Param # \n", 92 | "=================================================================\n", 93 | "conv2d_3 (Conv2D) (None, 30, 30, 32) 896 \n", 94 | "_________________________________________________________________\n", 95 | "max_pooling2d_2 (MaxPooling2 (None, 15, 15, 32) 0 \n", 96 | "_________________________________________________________________\n", 97 | "conv2d_4 (Conv2D) (None, 13, 13, 64) 18496 \n", 98 | "_________________________________________________________________\n", 99 | "max_pooling2d_3 (MaxPooling2 (None, 6, 6, 64) 0 \n", 100 | "_________________________________________________________________\n", 101 | "conv2d_5 (Conv2D) (None, 4, 4, 64) 36928 \n", 102 | "=================================================================\n", 103 | "Total params: 56,320\n", 104 | "Trainable params: 56,320\n", 105 | "Non-trainable params: 0\n", 106 | "_________________________________________________________________\n" 107 | ] 108 | } 109 | ], 110 | "source": [ 111 | "# geektutu.com\n", 112 | "model = models.Sequential()\n", 113 | "model.add(layers.Conv2D(32, (3, 3), activation='relu', input_shape=(32, 32, 3)))\n", 114 | "model.add(layers.MaxPooling2D((2, 2)))\n", 115 | "model.add(layers.Conv2D(64, (3, 3), activation='relu'))\n", 116 | "model.add(layers.MaxPooling2D((2, 2)))\n", 117 | "model.add(layers.Conv2D(64, (3, 3), activation='relu'))\n", 118 | "model.summary()" 119 | ] 120 | }, 121 | { 122 | "cell_type": "markdown", 123 | "metadata": {}, 124 | "source": [ 125 | "## 全连接层" 126 | ] 127 | }, 128 | { 129 | "cell_type": "code", 130 | "execution_count": 53, 131 | "metadata": {}, 132 | "outputs": [ 133 | { 134 | "name": "stdout", 135 | "output_type": "stream", 136 | "text": [ 137 | "Model: \"sequential_1\"\n", 138 | "_________________________________________________________________\n", 139 | "Layer (type) Output Shape Param # \n", 140 | "=================================================================\n", 141 | "conv2d_3 (Conv2D) (None, 30, 30, 32) 896 \n", 142 | "_________________________________________________________________\n", 143 | "max_pooling2d_2 (MaxPooling2 (None, 15, 15, 32) 0 \n", 144 | "_________________________________________________________________\n", 145 | "conv2d_4 (Conv2D) (None, 13, 13, 64) 18496 \n", 146 | "_________________________________________________________________\n", 147 | "max_pooling2d_3 (MaxPooling2 (None, 6, 6, 64) 0 \n", 148 | "_________________________________________________________________\n", 149 | "conv2d_5 (Conv2D) (None, 4, 4, 64) 36928 \n", 150 | "_________________________________________________________________\n", 151 | "flatten_1 (Flatten) (None, 1024) 0 \n", 152 | "_________________________________________________________________\n", 153 | "dense_2 (Dense) (None, 64) 65600 \n", 154 | "_________________________________________________________________\n", 155 | "dense_3 (Dense) (None, 10) 650 \n", 156 | "=================================================================\n", 157 | "Total params: 122,570\n", 158 | "Trainable params: 122,570\n", 159 | "Non-trainable params: 0\n", 160 | "_________________________________________________________________\n" 161 | ] 162 | } 163 | ], 164 | "source": [ 165 | "# geektutu.com\n", 166 | "model.add(layers.Flatten())\n", 167 | "model.add(layers.Dense(64, activation='relu'))\n", 168 | "model.add(layers.Dense(10, activation='softmax'))\n", 169 | "model.summary()" 170 | ] 171 | }, 172 | { 173 | "cell_type": "markdown", 174 | "metadata": {}, 175 | "source": [ 176 | "## 编译训练模型" 177 | ] 178 | }, 179 | { 180 | "cell_type": "code", 181 | "execution_count": 54, 182 | "metadata": {}, 183 | "outputs": [ 184 | { 185 | "name": "stderr", 186 | "output_type": "stream", 187 | "text": [ 188 | "WARNING: Logging before flag parsing goes to stderr.\n", 189 | "W0720 16:46:59.197520 140735530943360 deprecation.py:323] From /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/tensorflow/python/ops/math_grad.py:1250: add_dispatch_support..wrapper (from tensorflow.python.ops.array_ops) is deprecated and will be removed in a future version.\n", 190 | "Instructions for updating:\n", 191 | "Use tf.where in 2.0, which has the same broadcast rule as np.where\n" 192 | ] 193 | }, 194 | { 195 | "name": "stdout", 196 | "output_type": "stream", 197 | "text": [ 198 | "Train on 50000 samples\n", 199 | "Epoch 1/5\n", 200 | "50000/50000 [==============================] - 20s 394us/sample - loss: 1.4972 - accuracy: 0.4585\n", 201 | "Epoch 2/5\n", 202 | "50000/50000 [==============================] - 19s 386us/sample - loss: 1.1277 - accuracy: 0.6025\n", 203 | "Epoch 3/5\n", 204 | "50000/50000 [==============================] - 20s 398us/sample - loss: 0.9836 - accuracy: 0.6541\n", 205 | "Epoch 4/5\n", 206 | "50000/50000 [==============================] - 20s 404us/sample - loss: 0.8808 - accuracy: 0.6917\n", 207 | "Epoch 5/5\n", 208 | "50000/50000 [==============================] - 20s 406us/sample - loss: 0.8040 - accuracy: 0.7179\n" 209 | ] 210 | }, 211 | { 212 | "data": { 213 | "text/plain": [ 214 | "" 215 | ] 216 | }, 217 | "execution_count": 54, 218 | "metadata": {}, 219 | "output_type": "execute_result" 220 | } 221 | ], 222 | "source": [ 223 | "model.compile(optimizer='adam',\n", 224 | " loss='sparse_categorical_crossentropy',\n", 225 | " metrics=['accuracy'])\n", 226 | "\n", 227 | "model.fit(train_x, train_y, epochs=5)" 228 | ] 229 | }, 230 | { 231 | "cell_type": "markdown", 232 | "metadata": {}, 233 | "source": [ 234 | "## 评估模型" 235 | ] 236 | }, 237 | { 238 | "cell_type": "code", 239 | "execution_count": 58, 240 | "metadata": {}, 241 | "outputs": [ 242 | { 243 | "name": "stdout", 244 | "output_type": "stream", 245 | "text": [ 246 | "10000/10000 [==============================] - 1s 122us/sample - loss: 0.9077 - accuracy: 0.6830\n" 247 | ] 248 | }, 249 | { 250 | "data": { 251 | "text/plain": [ 252 | "0.683" 253 | ] 254 | }, 255 | "execution_count": 58, 256 | "metadata": {}, 257 | "output_type": "execute_result" 258 | } 259 | ], 260 | "source": [ 261 | "# geektutu.com\n", 262 | "test_loss, test_acc = model.evaluate(test_x, test_y)\n", 263 | "test_acc # 0.683" 264 | ] 265 | } 266 | ], 267 | "metadata": { 268 | "kernelspec": { 269 | "display_name": "Python 3", 270 | "language": "python", 271 | "name": "python3" 272 | }, 273 | "language_info": { 274 | "codemirror_mode": { 275 | "name": "ipython", 276 | "version": 3 277 | }, 278 | "file_extension": ".py", 279 | "mimetype": "text/x-python", 280 | "name": "python", 281 | "nbconvert_exporter": "python", 282 | "pygments_lexer": "ipython3", 283 | "version": "3.7.0" 284 | } 285 | }, 286 | "nbformat": 4, 287 | "nbformat_minor": 2 288 | } 289 | -------------------------------------------------------------------------------- /code/rnn-text.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "## 下载数据集" 8 | ] 9 | }, 10 | { 11 | "cell_type": "code", 12 | "execution_count": 1, 13 | "metadata": {}, 14 | "outputs": [], 15 | "source": [ 16 | "import matplotlib.pyplot as plt\n", 17 | "import tensorflow_datasets as tfds\n", 18 | "import tensorflow as tf\n", 19 | "from tensorflow.keras import Sequential, layers\n", 20 | "\n", 21 | "ds, info = tfds.load('imdb_reviews/subwords8k', with_info=True,\n", 22 | " as_supervised=True)\n", 23 | "train_ds, test_ds = ds['train'], ds['test']\n", 24 | "\n", 25 | "BUFFER_SIZE, BATCH_SIZE = 10000, 64\n", 26 | "train_ds = train_ds.shuffle(BUFFER_SIZE)\n", 27 | "train_ds = train_ds.padded_batch(BATCH_SIZE, train_ds.output_shapes)\n", 28 | "test_ds = test_ds.padded_batch(BATCH_SIZE, test_ds.output_shapes)" 29 | ] 30 | }, 31 | { 32 | "cell_type": "markdown", 33 | "metadata": {}, 34 | "source": [ 35 | "## 文本预处理" 36 | ] 37 | }, 38 | { 39 | "cell_type": "code", 40 | "execution_count": 2, 41 | "metadata": {}, 42 | "outputs": [ 43 | { 44 | "name": "stdout", 45 | "output_type": "stream", 46 | "text": [ 47 | "词汇个数: 8185\n", 48 | "向量化文本: [6351, 7961, 7, 703, 3108, 999, 999, 7975, 2449]\n", 49 | "6351 --> welcome\n", 50 | "7961 --> \n", 51 | "7 --> to \n", 52 | "703 --> ge\n", 53 | "3108 --> ek\n", 54 | "999 --> tu\n", 55 | "999 --> tu\n", 56 | "7975 --> .\n", 57 | "2449 --> com\n" 58 | ] 59 | } 60 | ], 61 | "source": [ 62 | "# geektutu.com\n", 63 | "tokenizer = info.features['text'].encoder\n", 64 | "print ('词汇个数:', tokenizer.vocab_size)\n", 65 | "\n", 66 | "sample_str = 'welcome to geektutu.com'\n", 67 | "tokenized_str = tokenizer.encode(sample_str)\n", 68 | "print ('向量化文本:', tokenized_str)\n", 69 | "\n", 70 | "for ts in tokenized_str:\n", 71 | " print (ts, '-->', tokenizer.decode([ts]))" 72 | ] 73 | }, 74 | { 75 | "cell_type": "markdown", 76 | "metadata": {}, 77 | "source": [ 78 | "## 搭建 RNN 模型" 79 | ] 80 | }, 81 | { 82 | "cell_type": "code", 83 | "execution_count": null, 84 | "metadata": {}, 85 | "outputs": [ 86 | { 87 | "name": "stdout", 88 | "output_type": "stream", 89 | "text": [ 90 | "Epoch 1/10\n", 91 | " 45/Unknown - 132s 3s/step - loss: 0.6927 - accuracy: 0.5083" 92 | ] 93 | } 94 | ], 95 | "source": [ 96 | "model = Sequential([\n", 97 | " layers.Embedding(tokenizer.vocab_size, 64),\n", 98 | " layers.Bidirectional(layers.LSTM(64)),\n", 99 | " layers.Dense(64, activation='relu'),\n", 100 | " layers.Dense(1, activation='sigmoid')\n", 101 | "])\n", 102 | "model.compile(loss='binary_crossentropy', optimizer='adam',\n", 103 | " metrics=['accuracy'])\n", 104 | "history1 = model.fit(train_ds, epochs=10, validation_data=test_ds)\n", 105 | "loss, acc = model.evaluate(test_ds)\n", 106 | "print('准确率:', acc)" 107 | ] 108 | }, 109 | { 110 | "cell_type": "code", 111 | "execution_count": 7, 112 | "metadata": {}, 113 | "outputs": [ 114 | { 115 | "data": { 116 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZYAAAESCAYAAADe2fNYAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nOzdd1hUx/rA8e8AIiogIlIsgAULiBXFGmvsJZpiibEl9yYmMd7kmhhzY4zGVPNLNNHENHtJ0yhq1ESxd2yIKNg7TUIvC+z8/jiAoqisLCzLzud59gHPnj37LsK+OzPvzAgpJYqiKIpiLFamDkBRFEUpX1RiURRFUYxKJRZFURTFqFRiURRFUYxKJRZFURTFqGxMHYCpubi4SG9vb1OHoSiKYlaOHDkSJ6WsUdh9Fp9YvL29CQkJMXUYiqIoZkUIcfl+96muMEVRFMWoVGJRFEVRjEolFkVRFMWoVGJRFEVRjEolFkVRFMWoVGJRFEVRjEolFkVRFMWoVGJRFEWxMGm6bL7beZ6QS/Elcn2LnyCpKIpiKVIzs1l24DLf77pAfKqOl7rUJ8Db2ejPoxKLoihKOZeamc3S/Zf5YbeWUB5rWINJPXxo7VWtRJ5PJRZFUZRyKiUzm6X7L/HDrgv8k5ZFl4Y1mNTTh1aeJZNQ8qjEoiiKUs6kZGazZN8lfth9gYS0LLo20looLUs4oeRRiUVRFKWcSM7IYsm+S/y45yIJaVl0a1SDST0b0qKOU6nGoRKLoiiKmUvOyGLxXi2hJKZn0b2xK6/18Cn1hJJHJRZFURQzlZSbUH7KTSg9GrsyqacPzWqbJqHkUYlFURTFzCRlZLFozyV+2nOBpIxsejZxZVKPhvjXrmrq0ACVWBRFUcxGYnoWi/ZeZOGei7kJxY1JPXzKTELJoxKLoihKGZeYnsXCPRdZuPciyRnZPO6rJZSmtcpWQsmjEouiKEoZlZiWxU97L7IoN6H09nPjtR4++NUsmwklj0osiqIoZUxiWhY/7bnAor2XSM7Mpo+fO6/18MG3pqOpQysSlVgURVHKiIQ0HT/tucji3ITSt6mWUJp4mEdCyaMSi6IoioklpOn4cfdFFu+7REpmNv383ZnY3fwSSh6VWBRFUUzkn1QdP+65wJJ9l0nJzKa/vwcTezSgsbt5JpQ8KrEoiqKUsvhUHT/uvsCSfZdIy8qhn78Hr3X3oZG7g6lDMwqVWBRFUUpJRlYO3+w4z0+7L5CWlUN/fw9e6+FDQ7fykVDyqMSiKIpSCrafieG9oDCuxqfTv5kH/+nhg085Syh5VGJRFEUpQTcS0pmx/hRbTkXTwNWen//djnb1qps6rBKlEouiKEoJyMrRs2jvReZsPYteSt7q04gXOtXD1sbK1KGVOJVYFEVRjCzkUjz/+yOMiOhkejZxZfpAP+o4VzZ1WKVGJRZFURQjiU/V8cmm0/waco1aTpX4/rnW9PJzN3VYpU4lFkVRlGLS6yW/HbnKx5vOkJKRzYtd6jGphw+VbS3zLdYyX7WiKIqRnL6ZxLtrwzhy+R/aejvzwRNNy818lEelEouiKMojSMnMZs7fkSzad4mqlSrw+dPNebJVLYQQpg7N5FRiURRFMYCUks1hUcxYH05UUgYj2noypU8jnCrbmjq0MkMlFkVRlCK6ciuN94LC2BERSxMPR74Z1YpWntVMHVaZoxKLoijKQ2Rm5/DdzgvM334OGyvBtAG+jGnvhY11+Z+T8ihUYlEUI0tMz2L/+Vt0bFAdB7sKpg5HKaa95+KYtjaMC3Gp9Pf3YNoAX9yr2pk6rDJNJRZFMYKkjCy2hkezMfQmu87GkpUjeaJFTeYMb2nq0JRHFJOcwawNpwk6cQOv6pVZMr4tXRrWMHVYZkElFkV5RCmZ2Ww7Hc2G0JvsjIhFl6OnZlU7xrT3JiUzm58PX2VUOy8CvJ1NHapigBy9ZPmBy3y+JYLMbD2TevgwoWt97CpYmzo0s6ESi6IYIDUzm21nYtgYeoPtEbHosvW4O9oxqp0X/Zt50LKOE1ZWgjRdNjsjY5kedIqgVzthbaVKUM3BiasJvLs2jJPXE+ns48LMwU2p61LF1GGZHZVYFOUh0nTZBJ+JYWPoTYLPxJCZrcfVoSIj23oyoJkHrTyrYXVX4qhsa8M7/ZowcdUxfjl8lZGBniaKXimKxPQsPt8SwfKDl6lhX5GvR7RkQDMPNSflEanEoiiFSNflsD1CSybbzkSTkaWnhkNFhrepQ/9mNQnwujeZ3G1AMw+WH7jM7C1n6OfvruY5lEFSStYev86HG08Tn6pjTHtv/turoSq6KCaVWBQlV0ZWDjsiYtl48ibbTkeTpsvBxd6Wp1vXoX8zD9p4OxvUpSWE4P1BfvT/ajdf/h3JjMFNSzB6xVDnYlKYtjaM/Rdu0byOE4vHtaVpraqmDqtcUIlFsWgZWTnsitSSydbwaFJ1OThXseWJlrUY0MyDwLrVizU+0sTDkVHtvFh24DIjAj1p7O5oxOiVR5Guy2He9rN8v+sClSpY8+GQpgxv46nGwYxIJRbF4mRm57A7Mo6NJ2/yd3g0KZnZVKtcgUEtatLfvybt6jkbdeLbG483JOjEDaavO8XP/26n+u1NKPhMNO+tO8W1f9IZ2qoW7/Rrgot9RVOHVe6oxKJYBF22nj3nYtkQepO/T0WTnJlN1UoV6O/vQf9mHrSvX50KJTSL2qmyLZN7NeLdtWFsPHmTAc1qlsjzKPcnpWTG+nAW77tkMdsDm5JKLEq5lZWjZ8+5ODaG3uSvU1EkZWTjaGdDn6bu9G/mQccGLiWWTO42oq0nKw9e4aONp+ne2NVi9+kwlbnbzrJ43yXGdfRmat8mFrE9sCmp326lXMnK0bP//C02ht5k86koEtOzcLCzoZevOwNyk4kp3lSsrQQzBvvx9IL9LNhxnjd6NSr1GCzVioOXmbP1LE+1rs17A3xVV2QpUIlFKTfCbyQxYcURLt9Kw76iDb183ejfzINOPi5UtDH9rOk23s4MblGTBbsu8FTrOnhWt5w90E1lc1gU09aG0b2xKx8P9VdJpZSoxKKUC+uOX2fK6lCcKtny7bOt6NbYtUwuwTG1bxP+Do9m1sZwvh8dYOpwyrWDF27x2s/HaF7HifkjW5Vat6cC6ietmLWsHD0z14cz6efjNKvtxPqJnejr71EmkwqAe1U7Xu3egL/Co9kVGWvqcMqtM1FJvLA0hDrVKrFwTBsq2ZbN34fySiUWxWzFJmcy6seDLNx7kfEd67LihUBqOJT90tHnO9XFq3plZqw/RVaO3tThlDtX49MY/dMhqtjasPT5QKpVUSselDaVWBSzdPxqAgO/3sOJawnMGdaC9wb6mk1XR0Uba94b4Mv52FSW7Ltk6nDKlfhUHWMWHiIjK4cl49tSy6mSqUOySCb7SxRCNBFCbBRCJAohEoQQa4UQ3o94rcFCiKNCiAwhRLQQ4ishhL1xI1bKip8PXeGZBfupYCNYPaEDT7SsZeqQDNajiRvdGtVgztazxCRnmDqcciFNl824xYe5npDOT2Pb0MjdwdQhWSyTJBYhRC1gNxAIfAJ8DnQDdgkhnAy8VntgDeAETM39/tXcr0o5kpmdw9Q1oby95iSB9ZxZ/2on/Gqa79pO0wb4kpmdw+zNEaYOxexl5eiZsPwoJ68l8PWIlrRRe+CYlKmqwqYD1YFuUsodAEKIs8DPwGvATAOu9Q5aguwvpTyde60k4C0hRB8p5WZjBq6Yxs3EdF5afpQTVxN4pVt93ni8kdmv7VSvhj3jO9Xlu50XeLadFy3qGPSZSsml10um/B7KzshYPhnqTy8/d1OHZPFKvcUihKgADAMi8pJKrtVAIvCsgZdsBcTlJZVcv+R+HfyocSplx4ELtxj49R7ORSezYFRr3uzd2OyTSp6J3X1wdajI9HVh6PXS1OGYpU83n2HNsev89/GGDG+r9r0pC0zRFdYMcAQO3XlQSpkNnAAaCiEMacc6AVl3HYvJ/er3qEEqpielZOGeizz740EcK1Vg3asd6dO0fH0ata9ow9R+jTlxLZHfj14zdThm58fdF/hu1wVGt/fi1e4NTB2OkssUicU79+uNQu6LuuucorgEuN41NpOXUFwMCUwpO9J1Obz+y3Fmbgine2NX1r3SkQau5XMw9okWtWjl6cRnm8+QlHH3ZyTlftYeu86sjafp5+/O9IF+alZ9GWKKxJK3jkVhf0G6u84pipWANTBXCNFQCPE48GHufYVuAyeE+LcQIkQIERIbqyaplTVXbqUx9Nt9rDtxg8m9GvLdqNblekc/IQQzBzflVqqOuVvPmjocs7AzMpbJv52gfb3qfDmsRbnpGi0vTJFYUnO/FlbOU/Wuc4ri/4CNwGggAm2s5tPc+1IKe4CU8nspZYCUMqBGjRoGPJVS0nZGxjJw3h6u/5PGwrFteLW7z0O3AC4PmtaqyvA2nizZd4mz0cmmDqdMO3E1gQnLj+Dj5sB3o1uXiXXglIJMkVgu5H51LeS+vHf5S0W9mJQyQ0o5AGgCdAXqACF3PZdSxkkpmb/9HGMXHcKjqh3rJ3aiW6PCfkXKr8m9GlLZ1poZ68ORUg3kF+ZCbArjFh+mur0tS8a1wbEct2TNmSkSSxha9VfnOw8KISoBLYHTUsp/inoxIUQ/IcRgKeUZKeVOKWUikLe63x5jBa2UnOSMLF5afoTZWyIY2Kwma17ugFf1KqYOq9RVt6/IG483ZM+5OP4KjzZ1OGVOTFIGoxceQgBLxwfi6mhn6pCU+yj1xJJb/bUKqJk7HpJnCFARbczEEK8Cvwsh7iwX+jeQgTYvRinDzsWk8MT8vWw9HcO7/Zswd3gLi94Ea1Q7Lxq5OfDBhnAysnJMHU6ZkZSRxeiFh4hP1bFoXBvquljeBw9zYqolXT4AbgE/CyGmCiHeBRYAV4GvAYQQDYQQY4UQD6sh/D+0iZ4bhBCvCCFWAz2B6VLKmyX3EpTi2nIqiifm7yUhLYvlzwfyQud6Fl/ZY2NtxfRBvlz7J53vd6meXICMrBz+tSSE87EpfPdca5rVVhNJyzqTJBYp5Q2gE3AAbRmWN4Fg4LHcrixy71+U+/VB19qGNnBfFfgC8AVelFJ+VjLRK8WVo5fM3nKGF5cdoX6NKqyf2In29dX+43k61Hehv78H3+w4x/WEdFOHY1I5esnrvxzn4MV4Pn+6OZ19VLGNORCWPkgYEBAgQ0JCHn6iYhQJaTom/XycnZGxDAuow4zBfmV27xRTuvZPGj2/2EmPJm7MH9nK1OGYhJSSaevCWH7gCtMG+PJ8p7qmDkm5gxDiiJSy0N3qzGOdcaVcCL+RxKB5e9l3Po6Phvjz6VPNVFK5j9rVKjOhSwM2ht5k3/k4U4djEl8Hn2P5gSu82KWeSipmRiUWpVSsO36dod/uJTM7h19ebM/IQLWm08O82KUetatVYkZQONkWtiHYyoNX+OLvSJ5sVZu3+zQ2dTiKgVRiUUpUga2Da2lbB7fyrGbqsMyCXQVr3u3vS0R0MisOXjF1OKVmy6ko3l17km6NavDJk/4WX9BhjlRiUUrMnVsHj+3gzYp/BeLqoOYeGKK3nxudGrjwf39FEJ+qe/gDzNzBC7eYuOoYzWo7Mf/ZVmazK6hSkEH/a7m7NL4mhFClGcoD5W0dfPxqAl8Oa877g/zUm8QjEELw/iBf0nQ5zN5SvjcEOxOVxAtLQ6hdrRILx7ax6PlM5s7Qv3QXYA5wTQixTgjxpBDCtgTiUsxY3tbBNtba1sFDWtY2dUhmrYGrA2M6ePPz4SuEXU98+APM0LV/0hiz8BCVba1ZOr4tzlXU24o5MyixSCk90eaVfIu2wdZvwE0hxDdCiHYlEJ9iRrStg08W2Dq4aS3z3Tq4LJnU04fqVWyZHnSq3K0jFp+qY/TCQ6Trclg6PpDa1QxZ3Fwpiwzum5BS7pNS/kdKWQfogrYEy2BgrxAiQgjxTu6e9ooFiUrMYNh3B1h16AoTutZn8bi2VFOfOo3G0a4Cb/VuzJHL/7DueGFbGZmnNF024xcf5to/6fw4pg2N3MvnnjuWplid3lLK3cCPaMklC/ABZgHnhBBPFz88xRzsOx/HgK93czY6mW+fbcWUPuVn6+Cy5KnWtWleuyof/XmalMxsU4dTbFk5el5ecZTQawl8PaIlbesasnGsUpY9UmLJXcdrmhAiHDgK/AfYDowCPIB1aGt4KeWYlJIFO88z6seDVK1UgbWvdKSvv4epwyq3rKwE7w/yIyY5k3nB50wdTrFIKZmyOpQdEbF8OMSf3n7la8tpS2dQ2YUQ4g1gBNr4igCOA5OBlVLK6DvO2wk8YcQ4lTImKSOLN387wZZT0fTzd+ezp5pjX1FV8ZS0lp7VeKp1bX7ac4FhbeqY7Sq/n2w+w5qj13nj8YaMaKsmy5Y3hrZYPgfcgM+AplLKVlLKL+9MKrmuoiUcpRyKiEpm8LzbS93PH9lKJZVS9FafRlS0sWbm+lOmDuWR/Lj7At/tvMBz7byY2P1hi5cr5sjQd4MewA75kLIUKeWGRw9JKcvWHb/O26tPYm9nw8oXAgmsp1YlLm2uDnb8p6cPszaeJvhMNN0bu5k6pCJbe+w6szaepm9Td94f5Kdm1ZdThpYbb39YUlHKJ122nveDTjHp5+M0reXIxomdVFIxodHtvalfowoz14eTmW0eG4Ltioxl8m8nCKzrzJfDWqgCj3LM4MF7IYSPEOKjO/49SwixQgjRxLihKWVFVGIGw7/fz+J9l3i+U11W/qud2hbWxGxtrJg+0I9Lt9JYuOeSqcN5qBNXE3hp+RF83Bz4YUyAWtW6nDN0SRdf4Agw5Y7D1dEG9PcLIeobMTalDMgrJT4Tlcy8kS2ZNsBXLc1SRjzWsAa9fN34OvgsUYkZpg6nUBlZOWwOi2L84sM4V7Flybg2ONpVMHVYSgkz9B3iQyAZbetfAKSUE4AmQFLu/Uo5cHcp8bpXOjKgWU1Th6Xc5d3+vmTrJZ9sOm3qUPJl5+jZfTaWN387QZsPt/LS8iPY2lixdHxb1dK1EIYO3gcCX0kpt995UEoZIYRYALxstMgUk0nOyOLN30LZfCpKlRKXcZ7VK/PiY/X4Ovgco9p5EeBtmkmGUkqOXkkg6Ph1Np68SVyKDvuKNvTyc2NQ85p0bOCiWroWxNB3i6rA/dbuzgDURhtmLjI6mZeWHeFyfBrv9m/C853qqsqdMm5C1/qsPnKN6UGnCHq1U6kNikspOROVTNCJG6w/cYNr/6Rja2NFj8auDGpek26NXdVYioUyNLEcBcYLIeZLKTPzDgohKgLj0MZfFDOVV0pcpaIqJTYnlW1teKd/E15deYyfD1/h2UCvEn2+y7dSCTp+g6ATNzgbk4K1laBjAxf+07Mhvfzc1BiKYnBi+QDYBIQKIRYBN4BawFigPtDXqNEppUKXreejP0+zeN8lAryqMf/ZVripvnCz0t/fg2V1L/P5lgj6+3vgVNm4C4DGJGWwPvQmQSducOJqAgBtvKvxwWA/+vp74GJf0ajPp5g3Yei0FCHEc8BcwAmQaEu7JACTpJTLjB5hCQsICJAhISGmDsNkohIzeGXlUY5c/ofxHesytV9j1Rdupk7fTKL/V7t5rp0XMwY3Lfb1EtJ0bAqLIuj4DQ5cvIWU4OvhyKAWNRnYvCa1nCoZIWrFXAkhjkgpAwq7z+ARWSnlMiHEaqAD4ArEAPuklGnFC1MpbfvP32LiqqOk6XL4ekRLBjZXVV/mrImHI6PaebHswGVGBHrS2N3R4Guk6bL5Ozya9SdusDMylqwcSV2XKkzs7sOg5jVp4GpfApEr5c0jlfrkJpGtRo5FKSVSSr7fdYHPtkTgVb0yq/7VDh83tQ9GefDG4w1Zf+IG09ed4ud/tytS4YUuW8+uyFiCTtzg7/Bo0rNycHe0Y0x7bwa3qEXTWo6qgEMxiMGJRWi/YY3QWit3qgQMkFJONEZgSsm4s5S4b1N3PnuqGQ5qsLXccKpsy+TejfjfH2FsPHnzvnOPcvSSgxduEXTiBpvCokhMz8KpcgWGtKrFoOY1aevtjJVackV5RIYum18XCAJ8H3CaSixl1J2lxP/r14QXOqtS4vJoeBtPVh68wkcbT9O9sSuVbbU/cyklJ64lEnT8BhtCbxCTnEllW2t6+boxuEUtOvmouSaKcRjaYpkNuANvA68Dh4E1wGi0MZcRRo1OMZqgEzeY8nsoVSrasOKFQNqpUuJyy9pKMGOQH08t2M+3O84zsHlNgo7fYH3oDS7fSsPW2oqujWowqEVNejR2o5KtmmuiGJehiaUj8IWUcrYQojrQVkq5RAjxM3AM6A6sNXaQyqNTpcSWKcDbmSda1OTr4HN8HXwOKwEd6rvwStcG9G7qTtVKqvtTKTmGJhZ7ID73+wPAvwGklJlCiOXAK8BrxgtPKY7opAxeXqGVEo/r6M07/Zqorg4L8k6/JugltPJ0ol8zD1wd1AcKpXQYmlgigCeFEAvRZtk7CSF8pZThQCraki9KGXDgwi1eXXmMNF02X41oySBVSmxxXB3t+GpES1OHoVggQxPLl8AyYIWU8hkhRBjwsxBiLfAv4KSxA1QMI6Xkh90X+HRzXilxoGWVEv9zGSo5gZ36jKMopmJQYpFSrhBCWKOVGwNMANYB76K1WKbc77FKyUvOyOKt30PZFGaBpcRSwqEf4K//gWNNeG4tONc1dVRKWZKZAglXoGot9cGjhD3KzPuld3y/VwjhhZZozkspE40ZnFJ0kdHJvLT8CJdvWWApcUYiBL0G4WuhXle4eQIW9oHn1oCbn6mjU0xNlwaHvoe9cyD9H+1YRUeoWvuuW53b3zt4gLWFfCgrAcXeZENKmYq26rFiIutP3GDK6lAq21pgKfHNE/DrGO2T6OMzof1EiIuAZUNgUV949neo09bUUSqmkJ0JRxbDrs8hNQYa9AT/Z7TvE6/l3q7C9SOQdqvgY4WVllzuTjh3fm9XFSzlw5uB1O5NZu73I9eY/NsJyysllhJCFsLmqVC5Ooz7Ezzbafe5NoHxW2DZE7B0MAxbpr2pKJYhJxtOrISdn2mJw6sTPLMUvNrf/zG6VEi8rp2fn3TuSDyngyDnrq2obB0Kb/U41bH4Vo9BqxsLIfYAW6SUH5RcSKXL3Fc3HvLNXtJ1Oayf2MlySokzk2H9fyDsd6jfA4Z+D1Vc7j0vJQaWD4WYM9o5TYeWfqxK6dHnQNga2PERxF+AWq2h+zSte7S4LQu9HlJjc5PNlXuTT+K1h7R67mrxONcH53pgbb6f7Y25urED2v72ShlwMzGdY1cSmNyroeUklehT8Oto7Y2j+zTo9AZY3ee127vCmA2wajj8Pl4biwkYV7rxKiVPSjizAYI/hNjT4NYUhq+CRn2N11VlZQUObtqtduvCz9GlQdJdrZ6Eq7mtnqNwen3BVo91RXBtrMXr6quNB7r5ab+3Zs7QxLIMmCaEqC6lvPXQs5UStSUsCoC+/h4mjqQUSAnHlsGfb2p926ODoG7nhz+ukhOMWgO/jYEN/4H0eC0Zqb5x8yclnNsGwR/AzeNQ3QeeWgi+Q+7/YaMk2VYGFx/tVpj8Vs9ViDsL0WEQEw7ntsLxFbfPq1JDSzCuuYnGzRdqNIYK5rP/zaPMY+kGbBNCPCmlPF8CMSlFtCksioZu9tSvUc73yNClwoY3IPRnqNsFnvzRsE91tpVh+EpYOwG2zYS0eOg1SyUXc3ZpDwTPgiv7wckTBn8DzYaV7a6lAq2eu3qQUuO01nj0KYjJ/RryE2RnaPcLK6jeILdl01RLNm5+UNXTNEn0IQz9X/gBiAW6AOFCiHVAyh33Synl88YKTrm/2ORMDl2KZ2L3+3w6Ki9izmitjdgI6DoVHnsTrB5h0UTrCjDke6hUDfbPg4wEGDC3bL8RKfe6FqIllAvbtfGL/v8HLUeDjXG3Yi51VVygXhftlkefA/EXb7dsok9pLbPwO5ZjtHXQilXyutHc/LTkU8mp9F/DHQz9qxp717+fuuvfElCJpRT8FR6FlNC3qbupQyk5x1fBxjfAtgqMzp2jUhxWVtD3My257PwU0hPgyZ+ggoVU0pmzqJPaGErkJq0KsNeH0OZ5s+oeMpiVNbg00G5+T9w+npmsfeDKa9lEn4JTf8CRRbfPcaxdMNm4+WktnlKqUjN05n3Za3NZqM1hUXhXr0xj93K4XIsuDTa9CceWa6WiT/0EDkZKoEJAt3e05LL5bVj5tNZNVrEc/hzLg7izsP0jOLUGKlaF7u9C4EuW/f9V0QHqtNFueaSEpBu5LZuw3IQTDue3gT5bO8faFlwa3R63cfMD9+ZgX8PoIap+ADOUkKZj//lbvNC5XvmbXR93Vqv6igmHzpO17q+S6K5qN0FLLmtfhiWDtImUVSxoYmlZ988lbR7KiVVgU0n7XejwqvZ/ptxLCG2pmqq1wOfx28ezdXDrbG6iCdOSzcVd2nglQPtXofeHRg9HJRYz9Hd4NNl6ST//ctYNFvobrJ+kdU2NWl3ykxqbD9cqzH4do83Sf+4P7Q9TMZ2km7BrNhxdqg1Yt3sZOv6nRD5VWwQb29tdYTxz+3havPbhrUrJ/FwN3Zo4+CGnSCllj2LEoxTB5rAoajlVwr9WOVlILytD65Y6sgg822vjHqX1Bt+or7am2MrhsLC3tnilS4PSeW7lttQ42PMlHP5R67ppNVor1HBU2z2UiMrO4N2pxC5vaItFjzZAn8cRbQFKR+AicMFIcSn3kZyRxe6zcTzX3qt8dIPdOq9VfUWdhI6TtEmPpb0MhncnGLsBlj+Zm1zWgEfz0o3BUqUnwL6v4cC3kJ0OzYZD1ylQzdvUkSnFYOjg/T19E0KICsAk4C3gP0aKS7mP4DMx6HL0t6vBdGnaootu/uZXOnvqD1g3Uat+GfELNOpjulhqtoDxm2HpE7B4AIz4Gbw7mi6e8i4zBQ5+qyWVjETwGwJd34EaDU0dmWIExljdOAv4XAiRAswG+hU7KuW+Np2MwtWhIq08cwcx/5yszdqtWBXqd9XGJer3KNtjBdmZ8Ne72lLmtdvAU4u0hftMzcUHnt+irYy8fCg8vcS0ya48ykrXFg/d/QWkxaZa5dMAACAASURBVEHDvtD9f+Dub+rIFCMy5kfcdcAnRryecpc0XTY7ImN4unUdrKwExJyG4yuhySBtEPrcNghfp53s6gcNemiJxrMd2FQ0bfB5/rkEv42FG8e0ipQe08vW5LaqtWHcZljxJPw8EoYsgGbPPPxxyoNl67QleXbNhuSb2pyk7tPunYGulAvGTCwjgCwjXk+5y86IWDKy9PTNqwYLnqXVtA+cqw3GSXl77aFzW7V+631fQYUqUPex24nGVDsrnl4Pa1/Rvh+2ApoMME0cD1OlOoxZD6tGwJp/aZtDBb5o6qjM16U9Wll3wmWoEwhDfyjaOm+K2TJWVVhNwAdYcZ/7FSPYFBaFcxVb2no7w9XD2oqu3d7Vkgpotex5pYUdJ2n92Jd2a0nm7N/arGXQZuA26KndvDpqa2mVpGwdbJ0OB76Bmi3h6cVlf3C2ooM2t+X38bDpLS25dJmi1hcz1MnftTXanLy0n2eDnupnaAEMbbHUo2BVWJ4UYD7wbrEjUgqVkZXDttPRDGxeExsrAdtmaDXo7Sbc/0EV7bVy2kZ9tdbMrfO3WzNHFsPBBdrS3d4dbycal4bG/cNPuAK/jYPrIdD2Rej1QdnplnuYCnbaBlHrX4MdH2u1/30+KZOL/pU5UmoD839P0z68DF+hJjdaEEOrwrxLKA7lIfacjSNVl0Ofpu5wPlhrifT5VEseRSHE7XWH2r2kDaJe3qeNy5zbClve0W5VPW93mdV9DOwcHz3oiM3wx4vaYnpPL9Yqf8yNtQ0Mmldw8crB8y12Z8Ai0edov0sHF2j/508sUOuxWRgzq0+1XJvConCws6FDveqwcKaWAIqzaVWFSrkJpAfwkdayyEsyJ3/TJita2UCddto5Po9ry3UXpTWTk6UtT7/vK63a5+klUL3+o8dqalZW2jL7lappe39kJGqJsjwvgPiostJhzb+1rXzbvaL93FQLz+IYnFiEEJ2BCVLKkbn/XgD4A/+TUu4wbngKQFaOnq2no3m8iRu2keu1pbOf+Na4XUpOuYkqYJw2JnLt0O1us20ztJu9W26XWQ+o1+322M6dEq/D7+Pg6kEIGA+9Py4fn1aFgMcma8uRb5wMy4bCyJ+1ajxFkxavFTxcPQi9P4L2r5g6IsVEDB28bw9sA6yBkbmHY4GmwBYhRKCU8rhxQ1T2n79FYnoW/fxqQPAL2m5yzYaV3BPa2Gqz0b07Qc/3ITlK6347txUi/tTmzQgrbU/xBo9ryaZmCzi/XauiytFpy7L4372rQjnQ5gWwc9K6+Bb313anLAdbyRbbP5dhxVNaOfnTi8yz21MxGiFlYWPx9zlZiL+AOkA/KeXFO467ADuBi1LKMlpDWriAgAAZEhJi6jAeaOqakwQdv87xgVFU2DjJtKW6+hxt/+681sz1I4DUuonS/9Hmzzyz5P7bs5YXZ7fCL6O0taxGr9VafJbq5glY8bS22+HwVWrFAgshhDgipSx0IpKhiSUW+EhK+WUh970BTJFSuj1ypCZQ1hNLjl7S9sOtdK7rwJyY8dqueS9sLTslm2nxua2ZbdoueF2nlnz5cllx5QCsfEabJ/TcH+Da2NQRlb5z27RtDuycYNTv2m6GikV4UGIxdFStInC/chhbQI1mGtmhi/HcStXx70rBkHQdek4vO0kFtHEW/6dgyLdaKbGlJBXQVjQY+6e2Gu+ivnDtiKkjKl3HV2qJtZq39mFHJRUll6GJZR/wihCiwCL+uV1hE4C9xgpM0WwOu4mzTQZNzv2gDZjXfczUISl3cm+qrS9W0QGWDIQLO0wdUcmTEnZ9rk189OoI4/4ERw9TR6WUIYYmlmmAG3BGCLFACPGeEOI74DTgigETJIUQTYQQG4UQiUKIBCHEWiGEt4Hx5F3LSwjxqxAiRghxQwixSAhh9iOqer1k86koZrhsR6THQ4/3TB2SUhjnejB+izbOsuJpbema8ionGza+oZVd+z+jzaZXlXHKXQxKLFLKw2irF98C/g28D/wLiEcb0C9SX4AQohawGwhEW7jyc6AbsEsI4WRITEKI2kAI0B5tdeVvgIHAQSGEWf/GH7uaQHZSDH2SV4PvYKjVytQhKffj6KF9cvdoro05HF1m6oiMT5emFSyELIROr8OQ78rWAqJKmWHwPBYpZTDQUAjRAK2VEiOlPGfgZaYD1YFueXNfhBBngZ+B14CZBlxrIuACdJZS7sm91hngN2AcMMfA2MqMzWE3mVhhHTb6TG0lWKVsq+wMo9dpb75Br8LVA/DYW1DNy9SRFV9qHKwcplUB9vsc2v7L1BEpZdgjT4mVUp6TUu4zNKnkbgw2DIi4a0LlaiAReNbAUPJKce5sLR256z6zI6XkSGgoz1pvQ7QYWf7Ld8sL2yraJmHtXoHQX+Hr1rDhdUi8ZurIHl38BfjpcYgOg2HLVVJRHsrgxCKE6CyEWHnHvxcIIfYKIboW8RLN0LYyPnTnQSllNnACrTVUyJTu+4rK/ep+x7Gad91ndsKuJzEidYW270rXt00djmIIm4rQ5yN47Ti0HqN1i33VEv58E5Jumjo6w1w/Aj8+rs1RGh1Udrc6UMoUgxLLHTPv75z2fefM+xZFuIx37tcbhdwXddc5RfENkAosEEI0E0K0Ab7Mjeunwh4ghPi3ECJECBESGxtrwFOVnkOH9jLUejdZrZ7XNp9SzE/VWtD//+C1o9BipDY28VUL2DwVkqNNHd3DRW7Rtmm2rQLP/w2egaaOSDEThrZYZgDngQZ5B6SU04D6wDlgVhGukTfRobBNwXR3nfNQUsoTwNNAF7QWzyG0LrABUsqr93nM91LKACllQI0aNQo7xaSklPic+gqdVSXsur1p6nCU4nLy1DZjezUEmj4FB7+Duc217ZlT40wdXeGOLNHW/XJpqCUV1RWrGMDQxNIS+P7O5VwApJRxaK2DNkW4Rmru18Iqtqredc5DCSGeAP4AtqDtYjkaLcH8ndvCMjtXQnfzWM5+IuuP1XYzVMoH57rwxHx49bBW5bd/PsxpBltnaCsYlAVSwvaPtD1o6neDsRvBwawW01DKAFPMvL+Q+7WweSZ5zYdLRQlGCGEN/AicBYZIKX+WUi4DeqO1fuYX5TpljQieSZx0pGbf/5o6FKUkVK8PQ7+Dlw9qm7Dt+VJLMMEfamMZppKTBetehZ2fQotRWhFCUff7UZQ7mGLmfRha9VeBTa+FEJXQWkSnpZRF/etyRStbPiOl1OcdlFKmoSUn3yJep+w4vx3PxMOsdxxJjeoupo5GKUk1GsJTP8HL+6FBd9j1GcxpDjs+1fZ8KU2ZKbBqOBxfDl3ehsHz1GZmyiMr9Zn3udVfq4CaQojH77hrCFqLaGWhDyxcLJAMBAoh8sdlhBAeQBPg4v0eWCZJScaW6VyTLog2400djVJaXJtoWyC/tAfqdoYdH2ktmN3/B5nJJf/8ydGwuJ+27cHAr6Db1LK1Hp1idkwy8x74IPcaPwshpgoh3gUWAFeBrwGEEA2EEGNzJ2LeL55stAqwOsAeIcTrQoi30Gb1VwE+NeT1mdzpIOxiTjA3eyi9mlnwMuyWyt1f2xv+3zu0BS63zdQG+ffOBV2Rhx0NE3dWm6MSdxZGrNLKoxWlmAyexyKlDJZSNgQaAp2AhlLKRlLK7QZc40buYw8AU4E3gWDgMSllXh9AJ2BR7tcHeR9twD4bbUb/W8B1tKqwxUWNyeRysiF4Fles63DeYyA1ndRC0RarZksY+Qu8EKx9//d7WoLZP1/b+tdYrh6Cn3ppSWvsBmjY23jXViyaQfuxAAghHIEO3Dv4XgntzXygkWIrFWVmP5Zjy2HdK7yo+w+teo/mxS5mvEe8YlxXDmrdYxd2gL07dH4DWo0p3pbPpzfA6ue1jcpGrdYW0lQUAzxoPxZDtyZuAWyiYFIRQF52MixLKZqsDNj+MTGOfmyJacP/mqolyJU7eAZqa5Bd2quVAm96C/bMgcf+Cy2f02b6G+LQD9o1arbSWkZVVJGIYlyGdoV9AqQDw9EGxn9GW5V4BVp5bzejRmcpQhZC0jXmWz2Lr0dVPKtb0GZZStF5d9S6rEYHgVMd2PhfbS2yI0u0UuGH0evh7+nw52Tw6Q1j1qukopQIQxNLa2CelPI3YBngLaXcCYwBQoHnjRxf+ZeZDLs/J9PzMZZEedO3qfvDH6NYLiGgXhdt/5dRa8DeTZvM+HVrOLZCG6srTLYO/ngR9s6B1uO0xSQtabdPpVQZumy+DZCR+30I8F8AKaUUQvyCNhCvGGL/fEi7xd8eL0Ik9PU378SSmZlJfHw8ycnJ5OTkmDqccq4mdJqnDehnJEKGDkJ2gZ0jVKh8u2RY6rWlY2o+BfVe0O6PPGva0JUyxdraGgcHB5ydnalY0cCu1UIYmlhOAuOFEL+hrcllL4RoK6U8hDbzvvgRWZLUONg3D5oMZPnV6vi46mjg6mDqqB5ZZmYmV65coVq1anh7e1OhQgWEmg9ROqTUkktyFGSna3/Z9m5ga68te+9UFZyaQmW1RJBSkJSSrKwskpKSuHLlCp6ensVOLoZ2hX0ENAd+kFLGos20XyOEWAq8AxwsVjSWZvcXkJVKQrspHLoYb/bdYPHx8VSrVg0XFxdsbW1VUilNQkAlJ6jRCKrVBQQkXIaYcMjJ1Kq+VFJRCiGEwNbWFhcXF6pVq0Z8fPHXrTN0guRmoBfanBOAF9EWjBwFJABvFDsiS5FwFQ7/CM1Hsim6KnoJfcy8Giw5ORlHR0dTh2HZ8hNMY6jmDRUdobqP1v2lKA/h6OhIcnLxV3t4lK2JtwPbc78PBxoJIZyllGVkeVYzsfMTQELXt/lz9Q28qlemiYf5doMB5OTkUKGCWl+qTBACKlXTbopSRBUqVDDK2Ogjb018J5VUDBQbCcdXQpsXSLR1Z//5W/Rp6l4uuo7Kw2tQFEtlrL9foyQWxUDBH2hVO53/y9+no8nWS/qaeTeYoihKHpVYStv1o3A6CNq/ClVc2Bx2k5pV7Wheu7B9zxRFUcyPSiylbdtMqOQM7V8hJTObXWfj6NPUQ3UhKYpSbqjEUpou7IQL2+GxyWDnSPCZGHTZerOfFKkoinInlVhKi5SwbQY41oYAbeWbTSdvUsOhIq09VeWOoijlh0ospeXMBrh+BLq+DRXsSNflsCMilt5+blhZqW4wRVHKD5VYSoM+B4JngUtDaD4CgJ2RMaRn5ahqMEVRyh2DJ0gqjyD0F4g9A08vAWvtR74pLIpqlSsQWNfZxMEpiqIYl0osJS07U9ucyaMF+A4GIDM7h+DTMfTz98DG2jIajTPWnyL8RpKpw3gg35qOTB/oZ5Rrbd68mQ8//JCTJ09iY2ND69at+eSTT2jZsmX+OYcPH2batGns3buXihUr0rJlS2bMmEGHDh0KXKso53l7e+Pt7c2OHTsKPLZnz55kZ2cXOC6EYMyYMfz444/MmTOHpUuXYm9vz759+wo8dsWKFXzxxRdERERgb29Px44dmT17NvXqFdxt8q+//uKDDz7gyJEjVK1alcDAQD788EP8/LSfZXBwMD169GD27NlMnjw5/3FSSry8vHB2dub48eOP9HNWyibLeFczpZBFkHgVek7PX8Z877k4kjOz6aOqwcql4OBgBgwYQGpqKrNmzeLtt98mIiKCPn36kJCQkH9Op06dOHXqFO+88w5Tp07lypUrdOnShZ07dxa4VlHOM5Rer2fQoEHMmDEDX19fevcuuN/90qVLGTVqFE5OTsyePZuJEyeyc+dOBg4cWGDJj2XLltGnTx9u3brFzJkzmThxIocOHaJdu3ZEREQA0LVrV2rXrs2qVasKPMf+/fu5evUqY8aMeeTXoZRRUkqLvrVu3VqWmIwkKT+tJ+XiAVLq9fmH//vrcdl0+maZmZVTcs9tAuHh4aYOoUyYM2eOHDBggLx582b+sV9++UUCcuPGjVJKKX18fKSTk1OBc86fPy8BOWzYsPxjRT3Py8tLdunS5Z5YevTocc9xQFauXFn6+/vL6OjoQl/D1KlT5dChQ2VGRkb+sU8//VQC8tSpU1JKKVNSUqSTk5OsX7++TE1NzT9v27ZtEpBTpkzJPzZlyhQJyIiIiPxjkyZNkjY2NjIqKqrQGBTTKOrfMRAi7/O+qrrCStKBbyEtDnrcbq1k5ej5Ozyank3csLVRDcbyaNKkSUyaNImYmBg2btzI4cOHWbZsGQAxMTGcPXuWs2fP8sILL+DufrvVWq9ePbS/V01Rz3sUOp2O33//HVdX10Lv/+ijjwC4evUqISEhHDhwgKVLl+a/Bl9fX/bt20dCQgKTJ0+mcuXbu1F27979nviee+45Pv30U1atWsX06dORUvL777/Tu3dv3NzcivValLJHvbOVlNRbsPcraDwAagfkHz5w4RaJ6Vn0MfO9V5T7Cw8Pp3Pnzri5uTFs2DC2bNlC27Zt8++PiYkBoFatWg+8TlHPe5CkpMLHtQYPHkzDhg3v+7i9e/fSrFkzPD09GTduHAcPHiQgIKDAOYbE5+fnR4sWLfK7w/bs2cP169cZPXp0UV+KYkZUYikpe7RNvOg+rcDhTWFRVLa1pkvDGiYKTClpQ4YM4fz58+zbt4+kpCT279/Piy++mH9/Xivh+vXr9zx22rRp+ecW9bz70el0+eMcd7O3t7/v41JSUhg0aBB6vZ7Q0FASEhLYsWMHTz75ZIHzHhTfhAkT+N///lfg2HPPPUdERATHjh3j119/xcnJiUGDBj3wNSjmSSWWkpB4HQ79AM2Gg2vj/MM5eslfp6Lo1tgVuwrWJgxQKSm3bt0iMjKS7t270759e6ysrNDr9SxcuDD/HB8fH3x8fFizZk3+p36AuLg45s6dS2hoqEHngZYobty4USCWr7766r4tlgeJiIggPj6eJ554An9/f0DbdjqvOy9Phw4dcHJyYvHixWRkZBR4/IIFC7hw4UKB80eOHIm1tTUrVqxg9erVPPPMM9jZ2Rkcn1L2qTGWknDHJl53CrkUT1yKzuy3IFbuz9nZmdq1a7N27Vo+/vhjbG1t+eWXXzhx4gQA6enpACxYsIC+ffsSEBDAhAkTsLW15aeffiI9PZ1Zs2blX6+o53Xr1o158+bxf//3f4wYMYKgoCA+/fRTvL29DX4N9evXp0qVKvzwww+4uLiQlpbGsmXL8hNF3muoUqUKc+fOZezYsQQEBDBu3DgyMzP59ttvsbe3v6fF4u7uTs+ePfn666/R6XSqGqw8u9+ovqXcjF4VFhsp5fvVpPzzrXvumr4uTDb8358yJSPLuM9ZRqiqMM3x48dl9+7dZZUqVaSzs7McNWqU3LFjhxRCyH79+uWfd+jQIdmrVy9ZpUoV6eLiIvv06SNDQkLuuV5RzktISJDPPvusdHZ2lg4ODrJ3794yNDT0vlVhY8aMeeBr2L59u2zbtq2sVKmSdHd3ly+//LJcvXq1BOTLL79c4NzNmzfLjh07Sjs7O+nh4SGfeuopGRkZWeh1ly9fLgHZoEGDBz6/YjrGqAoTspjVJeYuICBAhoSEGO+Cv46Bs3/DpBNgf3scRa+XdPgkGP/aVflhdMADLmC+Tp8+TZMmTUwdhlKGRUVF4eHhwcyZM5k2bdrDH6CUuqL+HQshjkgpC30zU11hxnTjOISvhcfeKpBUAI5fSyAqKYO3mjYyUXCKYjrr16/n6tWr7Nq1C1tbW8aOHWvqkJQSpBKLMeVt4tXh1Xvu2hwWRQVrQY8mqmZfsTyXLl3iv//9L66urixZsoQ6deqYOiSlBKnEYiwXd8P5bdBrFtgV3GZYSsmmsJt0bOBC1UoVTBSgopjOxIkTmThxoqnDUEqJKjc2Bpm7iZdDTWjzwj13n7qRxNX4dFUNpiiKRVCJxRgi/oRrh3M38ap0z92bwm5ibSV43FclFkVRyj+VWIpLnwPbPoDqDaDFs/fcrXWDRRFY1xnnKrYmCFBRFKV0qcRSXKG/Quxp6P5u/iZedzobk8KF2FTVDaYoisVQiaU4snWw4yPwaA5NBhd6yqaTUQgBvf1UYlEUxTKoqrDiOLIYEq7AgDlgVXiO3hR2kwCvarg6qjWRFEWxDKrF8qgyU2DXZ+DdGep3L/SUi3GpnIlKpk9Tj1IOTlEUxXRUi+VRnVoDqbEwfFX+Jl532xR2E0DtvaIoikVRieVRtXwOXH0LbOJ1t81hUTSvXZVaTveWICuKopRXqivsUQnxwKRy7Z80Qq8lqm4wpVDHjh3jo48+4uDBg0U6Pycnh+zsbKPHMXHiRBYsWIBOpwO0JfEnTJhAUFCQ0Z9LsRyqxVJCNodFAagyYwsTHh5OZmYmdnZ2iDu6SD09PQvsCz937lxWrVpFZGQkgYGBD7xmdnY2Q4YMwdHRkeXLlyOEICkpidOnT2Nra4utrS3W1vduHKfX68nKysLR0ZG6devec/+xY8eYP38+devWZfDgwXh4eCCE4PDhwyxfvpwDBw7g5+dXjJ+GYqlUYikhm8OiaOLhiLdLFVOHopSiiRMnsn//fipVqoQQAp1OR3JyMuHh4flLkZ87d45Vq1axYsUKJkyYwJEjR2jduvV9r2ljY0NgYCDTpk3D2dmZr7/+mmPHjtGjR4/8xGJjY0NaWhrp6elUr14dgKysLDIzMxk/fjzffPNNgWvq9XpeeeUVpJR88cUXCCGIitI+DH333Xf8+uuvODg45B+TUpKVlQVoSVJRHuh+G7VYys3oG31JKaMS06XXlA1y7tbCNzsqr9RGX/datWqVBGRiYmL+sQEDBshu3bpJKaWcP3++9PLyktevX3/otV599VXZtm1bmZKSUuj906dPl9bW1kWK6+OPP5aAwbfAwMAiXV8xX8bY6Eu1WErAllOqG8ySrV+/noYNG9KoUSPi4uKws7PD0dERgG+++YYtW7Zw9OhRACZMmEBwcDAdO3Zk2bJldOrU6b7XnTNnDjqdjkqVilcMEhQUlL9t8KJFi+jTp0/+fevWreOll15i//79+dsa6/V60tPTqVy5Mg4ODsV6bsUyqMH7ErDpZBT1a1TBx039EVqioUOHsmPHDgBiY2Nxc9P24AkKCuL1119n6tSpNG3aFAAhBCtWrKB169Z07tyZYcOGsWfPnkKva21tXeyksnHjRoYPH06zZs0AGDduHB4eHvm3l156CYD27dvnH6tVqxYNGjRgxYoV2NvbF+v5FcugEouR3UrJ5ODFW/RV1WAWKycnB3d3rbUaFxeHu7s7GzZsYMiQIQQEBDB//nyEEPk3Ozs7Nm/ezOeff05QUBBLly7Nv1aHDh2oV68ejRs3pnHjxqxYsSL/voSEBEJCQkhISChybFeuXOGpp57ip59+AuCPP/4o0IWxatUqAM6ePZt/LDs7m9jYWLXro1JkqivMyP4Oj0Yvoa+/6gYrYNPbEHXS1FE8mLs/9P2kWJfQ6/VIKfO7vuLi4nBzc2PAgAEsWLCAoUOHkpKSkl/JJYQgJyeHjIwMvL29efrpp6lR4/a21n369CE1NZXMzEzmzp1Lampq/n0HDx6kT58+bNy4kX79+t0Ti06nIyMjA3t7e6xylxyaMGECEyZMICwsDIAhQ4YU+jp8fHzuOdaxY8f7tqYeJCUlhZSUlAeeU7VqVYNbY5s3b+bDDz/k5MmT2NjY0Lp1az755BNatmxZ4LzDhw8zbdo09u7dS8WKFWnZsiUzZsygQ4cOBp/n7e2Nt7d3fos0T8+ePcnOzi5wXAjBmDFj+PHHH5kzZw5Lly7F3t6effv2FXjsihUr+OKLL4iIiMDe3p6OHTsye/Zs6tWrV+C8v/76iw8++IAjR45QtWpVAgMD+fDDD/Mr94KDg+nRowezZ89m8uTJ+Y+TUuLl5YWzszPHjx836Gf8qFRiMbJNYVF4OlfG18PR1KEoJpA3HyTvTTIuLo769esD8K9//Qsgv2qrMHdXXL333nuAtrXv3LlzqVixYv59dnZ2Bb6C1loSd60EcfHixfzxkrsVZYxFSklGRkZ+cjLU559/zowZMx54zqJFiwxqEQUHBzNgwACaNWvGrFmzyMjIYN68efTp04eIiAicnJzyz+vbty+urq6888472Nra8v3339OlSxe2bt1Kly5dDDrPUHq9nkGDBrF792769++fXxmYZ+nSpYwZM4bu3bsze/Zs4uPj+fLLLxk4cCChoaH5ZeTLli1jzJgxNG7cmJkzZ6LT6Zg3bx7t2rUjJCSERo0a0bVrV2rXrs2qVasKJJb9+/dz9epVXn/99Ud6DY9CJRYjSkzLYu+5OJ7vVPeeP26LV8yWgLnIzMwE4Nq1a5w5c4YbN27g4+PD+fPnqV+/PmfOnKFGjRokJiYyatQofvnlF+rUqUNwcDC7d+9m/PjxD9wP/s75Knnf3/mGb21tnd8ayczMJCMjAw+Pe7tlc3JyAEhNTS3QlZbXIkpKSipwPK9VBeDl5WXQz2TUqFG0a9fugef4+/sbdM2TJ0/St29ffvjhh/xuR09PT4YNG8a+ffvyW3AvvfQSlStX5vDhw/nnDRkyhPr16/Ptt9/mJ4yinmeo1atXU79+fc6fP4+rq+s99585c4ahQ4eycuXK/A8NFSpUYMqUKURERODr60tqaiqvvfYa9erVIyQkJH8+VLt27ejRoweLFi3ik08+wcrKimeffZZPP/2UyMhIGjZsCMCvv/6KjY0NI0eOfKTX8EjuVy5mKTdjlhv/HnJVek3ZII9ejjfaNc2JKjeW8vr164WW6TZu3Fjm5ORIV1dXOW/ePJmcnCytrKzk5s2bpZRaKbGjo6NMSkoq9LoXL16UgFy2bFn+sd27d0tAbt++XUppWLnxoUOHJCAdHBxk9erVH3qrVq2atLOzk2+++WbxfkBGFh0dLTds2CCnT58u69WrJwG5aNEiKaWUkZGREpAvvPDCA69R1POklNLLy0t26dLlnuM9evS45zggbWxsZERExEOve+XKFblmd9qkzgAAEzpJREFUzRr51ltvSXd39wL/r3/99ZcE5KxZsx56nbCwMAnI999/X0oppV6vl7Vq1ZL9+/d/6GPzGKPcWA3eG9GmsCg8qtrRvLaTqUNRTKRmzZr3/JGNHTuW6tWrY2VlxaBBg9iyZQv29vbUr1+f06dPA7Bnzx6GDBlSpHLey5cvGyXOL7/8kmPHjhEXF5d/69y5M+3bty9wLC4ujvj4eNLT0/nss8+K/dzGEB4eTufOnXFzc2PYsGFs2bKFtm3bFjgnJiYGgFq1aj3wWkU970GSkpIKPT548OD8lkNh9u7dS7NmzfD09GTcuHEcPHiQgICCS0UZEp+fnx8tWrTIL8LYs2cP169fZ/To0UV9KUahEouRpGRms+tsLL393LGyUt1gli4nJ4eLFy8SGRlJcnJyfp9/v3792L59O1lZWfj6+nLmzBkSExM5efLkfQfS73To0CFatGjBxYsXHymuhIQEjh07RmJiIn369CErK4szZ87k327evMnNmzcLHMu7hYaGcvjwYW7duvVIz21MQ4YM4fz58+zbt4+kpCT279/Piy++WOCcvK6n69ev3/P4adOm5Z9f1PPuR6fTERERUeh9DyrPTklJYdCgQej1ekJDQ0lISGDHjh08+eSTRX4dEyZMyJ+TlOe5554jIiKCY8eO8euvv+Lk5MSgQYMe+BqMTSUWI9l+JgZdtp5+/qrM2FLFx8fTsWNH6tatS6VKlahXrx5LliwhOTmZqlWrAvDYY4+RkpLCnj176NChAx4eHmzatAlbW1t69er10OeYN28e9vb2ha4NVhQHDx4kMDCQwMBAOnXqlH/z9/fH19eXo0ePcuLECZo2bYqvr2+Bczp16kTHjh3vqYgqbbdu3SIyMpLu3bvTvn17rKys0Ov1LFy4sMB5Pj4++Pj4sGbNmvxP/aAVVMydO5fQ0FCDzgMtUdy4caPA83z11Vf3bbE8SEREBPHx8TzxxBP5Y0yZmZksW7aswHkdOnTAycmJxYsX549z5T1+wYIFXLhwocD5I0eOxNra+v/bu/voqqozj+PfXyNBCCl6g0iUARxYCHag1qZqEVEsBXTNOINj6VDtAE6nIyiWKZXxvfWly5fqmlGc+sJYUQTBloIyLuNbZSG6qEIrFgZFsdEUeQkQRAkvQZ75Y58LN+Em5CaHHMx9PmudddbdZ99zn5wF97ln7332ZtasWcybN4/Ro0fXGeDRGrzzPiblKzfQpVN7vt7z2KRDcQlJpVKcdNJJnHPOOQwZMoSBAwdSWlrKoEGD9o8MKykpYerUqaRSKaZOnQqEL4Lhw4c3Otw23SF/7LHHMn/+fLp06UJFRUWj8ZiF+b127dq1f/jziBEj9o9cS5s+fToTJkxg2rRpzJ8/n7179zJ58mRGjx7NmDFjuOOOOygqOnLmvEulUnTv3p0FCxZw++23U1hYyNy5c1mxYgUQZmhOe/DBBzn//PMpKytjwoQJFBYW8sgjj7Bz505uu+22nOsNHTqU+++/n3vuuYcxY8bwzDPPcOeddzY46q4xvXv3pqioiOnTp9OlSxdqamqYOXPm/kSR/juKioq49957GTduHGVlZYwfP57du3fzwAMP0KlTp4PuWLp168awYcOYNm0ae/bsYezYsTnH1mINdb7kyxZH533N7r3W74bn7Lrfvt3ic32Reef9wfbt22c9e/a0a665xiorK23NmjVWWVm5f/vwww+tc+fOdtddd9lHH31k77zzTtZzDBgwwAoKCuyll16yK664Iqf5vbJ16NfW1lp5ebmde+651q5dO5s2bZqZ1e2EXrRokZWWllpJSYlNmTLFlixZYrW1tYf1ejXVW2+9Zeedd54VFRVZKpWySy+91BYtWmSS7IILLqhT94033rDhw4dbUVGRdenSxUaOHGnLli076JxNqbdt2za75JJLLJVKWXFxsY0YMcLefvvtBjvvx44d2+jf8corr9jpp59uHTp0sG7dutnEiRNt3rx5BtjEiRPr1C0vL7ezzjrLjj76aCstLbWLL77Y1qzJPh/hE088YYD16dOn0c/PJo7O+8S/2JPe4kgsz/1pvfX8j/+1V9dUtfhcX2SeWA6YPXu2DRkyxE455RQD7KGHHrKxY8c2KwmYhS+ge++918zMNm3aZJWVlbZhwwarqqpqcNu4caOtW7fO1q5du/88K1eutFGjRlkqlTLAhg0bZm+++eb+44MHD7bTTjtt/+vq6mq79tpr7ZhjjjHAOnXqZPPmzTtMV83FZf369QbYLbfckvN740gsCsfzV1lZmS1btqxF55g8548sWlPFm9cPo11B/nZbrV69+qAHwPLV2rVrufDCC+nfvz9nnnkmV111FTU1NUCYBj+bffv2sWPHjqzPncTFzBg1ahQlJSVMmjSJU089tc7xsrIytmzZctDggJqaGubOnUt5eTmzZs1q8G9wyVq4cCGVlZUsXryY+fPn8/777zf6XFQ2Tf1/LGm5mWVd7dD/dbTQ7r2f8/LqTZw/oFteJxVXV+/evVm1alWdssLCwkO+L90XcrhIYsGCBQ0eb+hHVseOHRk/fjzjx48/XKG5GFRUVDBlyhS6du3KY489lnNSiYsnlhZ6/f0tfLp7r0866ZxL3KRJk5g0aVLSYSQ33FhSf0nPSvpE0jZJCyT1asZ5FkmyBrYZsQdez3Mr11Pc/igG9Wl4/ifnnMsnidyxSDoReDV6eQdQAFwNLJY00MyaPg94eP+MemXdgVuBmhaG2qjaz/fxwv9t5Fv9u9L+qOY9V+Ccc21NUk1hPwVKgKFmtghA0nvAHOAq4JamnsjMyuuXSUpP4/lCiyNtxO8/2Mq2mlpGejPYfmbmE3A69wUV12CuVm8Kk9QO+C7wbjqpROYBnwCXxPAxFxHuVp6P4VwNem7lejq0K+CcvscdunIeKCgooLa2NukwnHPNVFtb2+xZHTIl0ccyEPgy8EZmoZntBVYAfSWlmntySccDg4DnzWznoeo31+f7jOdXbeS8fl3pUOjNYADFxcXNmtrCOXdk2L59e5MmQj2UJBJLr2j/cZZjG+rVaY5RhL9rfkMVJP1Q0jJJy6qqqpr1Ics/rGbzZ7sZ+Te+UmRaKpWiurqazZs3s2fPnthuq51zh4+ZsWfPHjZv3kx1dTWpVLN/1++XRB9Lx2ifrc1kT706zXFRdO6FDVUws4eBhyE8INmcD/mSYEjf4xja7+DFe/JV+/bt6dGjB1u3bqWiomL/YlLOuSNbQUEBxcXF9OjRo84qpc2VRGJJL9rdOcuxzvXq5ETSscC5wKIcR5blrKxXiscvO/3QFfNM+/btKS0tPaxPjzvnjmxJNIWl53jO9lM/3Qte0cxz/x3QjkaawZxzzh1eSSSWlYTRX2dnFkrqAHwNWG1m1c089z8SJvNreM4K55xzh1WrJ5Zo9NeTwAmSvp1xaBTQHpjdnPNKKgKGA0vNbH2LA3XOOdcsST0geSvwHWCOpLsJT95PBSqBaQCS+gCDgSVm9n4TznkBcDTeDOacc4lKZK4wM/uYkDSWAtcSpnP5HTDEzD6Jqg0GHo32TXFRtPfE4pxzCfL1WGJYj8U55/JNY+ux+AIizjnnYuWJxTnnXKzyvilMUhXwYTPf3gXYHGM4X3R+Pery63GAX4u62sL16GlmWWfgzfvE0hKSljXUxpiP/HrU5dfjAL8WdbX16+FNYc4552LlicU551ysPLG0zMNJB3CE8etRl1+PA/xa1NWmr4f3sTjnnIuV37E455yLlScW55xzsfLE0gyS+kt6VtInkrZJWiCpV9JxJUVSv+h6fCpph6QXJA1IOq6kSbpakkmakXQsSZF0paR3JdVI+oOkkUnHlCRJPSU9JWmTpI8lPSqpzS1D630sOZJ0IrAienkPYWbmqwlrzAw83CtXHmkklQCrgCLgFxyYqfoToK+ZbU8wvMRIOhl4izDj9mNmNi7ZiFqfpBuBmwkd1SuBy4H+wFlmtjTJ2JIgqTvwR2AXcB9hmZDJwKfAqRkT8H7hJTVt/hfZT4ESYKiZLQKQ9B4wB7gKuCW50BLxL8DxwPfM7EkASbWEpRG+D/x3grElQtKXCDNz7yEklrwTfYneANxhZtdFZS8A7wITCDOb55tJhCfuzzazJQCS3gF+DYwH/ivB2GLlTWE5kNQO+C7wbjqpROYRfqFfkkRcCTsj2j+fUbY82vdr5ViOFP8OfBP4SdKBJOifgELgoXSBma0hLD8+KamgEpb+/7A8o6xN/l/xxJKbgcCXgTcyC6NVMVcAfSWlkggsQb8FbgQyl5M+Idpvbf1wkiWpL+Fu7X+AFxMOJ0lnEZp4iiUtlbQr+nV+br42jwIbon23jLIT6h1rEzyx5KZXtP84y7EN9erkBTObZWa3WdRZJ+koQlOHAU8nGlwri5rAfkWYXHBKwuEk7SSgFniG8EPseqAj8JSkMxp7Yxv2S2AH8KCkgZK+AfwnUAU8kmhkMfM+ltx0jPa1WY7tqVcn70RfrA8AXwfuMrM/JBxSa/sR4Zf6SDPbnod3r5k6ASngdjO7G0DSUmAJIclcmGBsiTCzFZK+Q1jlNj0A6FNgmJlVJhdZ/PyOJTc7on3nLMc616uTVyS1B2YDPyC0q1+TbEStS1If4OeEpsEVkroR+hMAOkjqJqkwsQBbX/rH15PpAjN7DagBvpZIRAmT9A+EpPI8MAb4Z0KCeVHSN5OMLW6eWHLzQbTPNu48/SVS0TqhHDkkdQTKCQMbbjWzyy3/xrEPBjoAFwHroy3dFzc6ej0omdASsSXa76pXvpUwMiqvSCog9Lu9B4wyszlmNhMYQWjtaFOjJ70pLDcrCaO/zs4slNSB8CtstZlVZ3tjWxU1f80DzgEmmtkDCYeUlBeB8+uVHQ/MAF4iPPP0divHlKRVhGbBXhxIMgDHAH9JIqCEdSU8pvCKme1LF5pZjaQKoE09UOyJJQdmtlfSk8Dlkr5tZulRP6MIDzvNTi66xFwFjARuyuOkgpmtA9ZllmXMxrDOzMpbO6aELQR+SHg+YzmApDMJfS+/TzCupFQR+lPOkNTRzGoAJJUSHhr9c5LBxc2fvM+RpBMIvzwF3M2BJ823AQPa0tOzhxLdqX1E+IHyE+DzelU+M7PftHpgR4gosfyZPHzyXpKAl4GhhBFPq4EfE365n2Zmf0owvERIuhm4ifD0/UygHSH59gbGm9mM5KKLlyeWZpDUj9C0cTZhWO0rwGQzq0gyrtaW8cXZkA/NrFerBHMEyufEAiCpGLid0MfUmdCUfH0e3r0B+5PtpYQHRPsCewlNhneZ2bNJxhY3TyzOOedi5aPCnHPOxcoTi3POuVh5YnHOORcrTyzOOedi5YnFOedcrDyxOOeci5UnFueaSFJFtIZ9tm1OwrGZpBlJxuBcmk/p4lxutpB9Zcg2NSWHcy3hicW53HzWlqbecO5w8KYw55xzsfLE4lxMJPWK+jrulPSwpGpJn0r6jaQeWeqfI2mRpBpJWyTNlNQ9S73hkl6N6q2XtEDSVxqI4WRJ5dHn/kXSTVnqXCJpeVSnStJCSafEcxWc87nCnGuyaN2MAuAbWQ5vBroT+lp2ERZ8mw6cSJh08GPgq+nZryVdDMwB1hIWgEpF9bYDZ5rZR1G97wOPAe8AvwIKgSuBYqDMzN6N6qUnQ/0K8CxhBt0JhCnZR5nZgqjeKMIqly9H+w7AvxGms++Tns7duRYxM998860JGyFZWAPbqYRFrYywSmIq431XROU/jl53IKzPUQF0yqg3CNgH/Dp6XQRUA+8DHTPqnRed746MsnQcN2SUfTUqm5ZRNi0q65xRVgY8CPRI+hr71jY277x3LjebgLFZytcSVggEeNrMtmYcmwvcD6TXNT+LsDzvfWb2WbqSmb0uaSnwt9FStoMIKy7ebRl3Emb2O8J6QPVtAH6R8Tq95klxRtmrhDue+yQ9BrxlZsuAZQ3/yc7lxhOLc7nZaQ2sJyIpnVg21Du0hXAnckz0+vhon22J3kpCAupMWBQL6q1M2YjlZrY7/cLM9oUlQA4ws6ckdQF+AJQD7SStJKwJMrOJn+Nco7zz3rn41e+A70r4v5a+i9nYQD2AvyL00XxCuDuC0E9Th6QHJP28XvHmpgRnZr80s9MITW1lwAfA45IGN+X9zh2KJxbn4vf3ko7LeP29aL8k2r9GuIu5TFJRulK0JvwZwLNm9jnwOmHJ63GSjs6odzJwOfDXuQYm6SVJiwHMrNbMlgM/iw6fnuv5nMvGm8Kcy00nSeMaOJZOHAYskfQQ4Q7kSsJosRkAZrZT0kTgSWC5pMxRYVVET/ab2Q5JP4ret0zSo0B7wmivz4D6dyxNsRi4WdLTwAuEUW6XAZ9Hx5xrMU8szuWmBHi0gWMDov39hGamG4B2wNPAZDP7NF0x6uvYTLhbuBXYCTwDXGvRUOOo3uOSNgI3ArcRRom9BlxnZu81I/5bCU1s/0pITF8idPJfGHXiO9di/hyLczGR1ItwZ3Kzmf0s0WCcS5D3sTjnnIuVJxbnnHOx8qYw55xzsfI7Fuecc7HyxOKccy5Wnlicc87FyhOLc865WHlicc45F6v/B5AzHK0TQ+o6AAAAAElFTkSuQmCC\n", 117 | "text/plain": [ 118 | "
" 119 | ] 120 | }, 121 | "metadata": { 122 | "needs_background": "light" 123 | }, 124 | "output_type": "display_data" 125 | } 126 | ], 127 | "source": [ 128 | "# geektutu.com\n", 129 | "# 解决中文乱码问题\n", 130 | "plt.rcParams['font.sans-serif'] = ['SimHei']\n", 131 | "plt.rcParams['axes.unicode_minus'] = False\n", 132 | "plt.rcParams['font.size'] = 20\n", 133 | "\n", 134 | "def plot_graphs(history, name):\n", 135 | " plt.plot(history.history[name])\n", 136 | " plt.plot(history.history['val_'+ name])\n", 137 | " plt.xlabel(\"Epochs\")\n", 138 | " plt.ylabel(name)\n", 139 | " plt.legend([name, '验证集 - ' + name])\n", 140 | " plt.show()\n", 141 | "\n", 142 | "plot_graphs(history1, 'accuracy')" 143 | ] 144 | }, 145 | { 146 | "cell_type": "markdown", 147 | "metadata": {}, 148 | "source": [ 149 | "## 添加更多 LSTM 层" 150 | ] 151 | }, 152 | { 153 | "cell_type": "code", 154 | "execution_count": 5, 155 | "metadata": {}, 156 | "outputs": [ 157 | { 158 | "name": "stdout", 159 | "output_type": "stream", 160 | "text": [ 161 | "Epoch 1/10\n", 162 | "391/391 [==============================] - 1811s 5s/step - loss: 0.5664 - accuracy: 0.6991 - val_loss: 0.0000e+00 - val_accuracy: 0.0000e+00\n", 163 | "Epoch 2/10\n", 164 | "391/391 [==============================] - 1513s 4s/step - loss: 0.3780 - accuracy: 0.8415 - val_loss: 0.4220 - val_accuracy: 0.8242\n", 165 | "Epoch 3/10\n", 166 | "391/391 [==============================] - 1461s 4s/step - loss: 0.3037 - accuracy: 0.8775 - val_loss: 0.4419 - val_accuracy: 0.8112\n", 167 | "Epoch 4/10\n", 168 | "391/391 [==============================] - 1450s 4s/step - loss: 0.2260 - accuracy: 0.9142 - val_loss: 0.4181 - val_accuracy: 0.8461\n", 169 | "Epoch 5/10\n", 170 | "391/391 [==============================] - 1434s 4s/step - loss: 0.1900 - accuracy: 0.9297 - val_loss: 0.4600 - val_accuracy: 0.8032\n", 171 | "Epoch 6/10\n", 172 | "391/391 [==============================] - 1436s 4s/step - loss: 0.3483 - accuracy: 0.8509 - val_loss: 0.4899 - val_accuracy: 0.7797\n", 173 | "Epoch 7/10\n", 174 | "391/391 [==============================] - 1425s 4s/step - loss: 0.3232 - accuracy: 0.8670 - val_loss: 0.4060 - val_accuracy: 0.8207\n", 175 | "Epoch 8/10\n", 176 | "391/391 [==============================] - 1437s 4s/step - loss: 0.2113 - accuracy: 0.9182 - val_loss: 0.4142 - val_accuracy: 0.8446\n", 177 | "Epoch 9/10\n", 178 | "391/391 [==============================] - 1445s 4s/step - loss: 0.1486 - accuracy: 0.9465 - val_loss: 0.4460 - val_accuracy: 0.8392\n", 179 | "Epoch 10/10\n", 180 | "391/391 [==============================] - 1431s 4s/step - loss: 0.0923 - accuracy: 0.9696 - val_loss: 0.5298 - val_accuracy: 0.8310\n", 181 | " 391/Unknown - 435s 1s/step - loss: 0.5298 - accuracy: 0.8310准确率: 0.83096\n" 182 | ] 183 | } 184 | ], 185 | "source": [ 186 | "model = Sequential([\n", 187 | " layers.Embedding(tokenizer.vocab_size, 64),\n", 188 | " layers.Bidirectional(layers.LSTM(64, return_sequences=True)),\n", 189 | " layers.Bidirectional(layers.LSTM(32)),\n", 190 | " layers.Dense(64, activation='relu'),\n", 191 | " layers.Dense(1, activation='sigmoid')\n", 192 | "])\n", 193 | "model.compile(loss='binary_crossentropy', optimizer='adam',\n", 194 | " metrics=['accuracy'])\n", 195 | "history = model.fit(train_ds, epochs=10, validation_data=test_ds)\n", 196 | "loss, acc = model.evaluate(test_ds)\n", 197 | "print('准确率:', acc)" 198 | ] 199 | }, 200 | { 201 | "cell_type": "code", 202 | "execution_count": 8, 203 | "metadata": {}, 204 | "outputs": [ 205 | { 206 | "data": { 207 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZYAAAESCAYAAADe2fNYAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nOzdd1hUx/rA8e8AIiogIlIsgAULiBXFGmvsJZpiibEl9yYmMd7kmhhzY4zGVPNLNNHENHtJ0yhq1ESxd2yIKNg7TUIvC+z8/jiAoqisLCzLzud59gHPnj37LsK+OzPvzAgpJYqiKIpiLFamDkBRFEUpX1RiURRFUYxKJRZFURTFqFRiURRFUYxKJRZFURTFqGxMHYCpubi4SG9vb1OHoSiKYlaOHDkSJ6WsUdh9Fp9YvL29CQkJMXUYiqIoZkUIcfl+96muMEVRFMWoVGJRFEVRjEolFkVRFMWoVGJRFEVRjEolFkVRFMWoVGJRFEVRjEolFkVRFMWoVGJRFEWxMGm6bL7beZ6QS/Elcn2LnyCpKIpiKVIzs1l24DLf77pAfKqOl7rUJ8Db2ejPoxKLoihKOZeamc3S/Zf5YbeWUB5rWINJPXxo7VWtRJ5PJRZFUZRyKiUzm6X7L/HDrgv8k5ZFl4Y1mNTTh1aeJZNQ8qjEoiiKUs6kZGazZN8lfth9gYS0LLo20looLUs4oeRRiUVRFKWcSM7IYsm+S/y45yIJaVl0a1SDST0b0qKOU6nGoRKLoiiKmUvOyGLxXi2hJKZn0b2xK6/18Cn1hJJHJRZFURQzlZSbUH7KTSg9GrsyqacPzWqbJqHkUYlFURTFzCRlZLFozyV+2nOBpIxsejZxZVKPhvjXrmrq0ACVWBRFUcxGYnoWi/ZeZOGei7kJxY1JPXzKTELJoxKLoihKGZeYnsXCPRdZuPciyRnZPO6rJZSmtcpWQsmjEouiKEoZlZiWxU97L7IoN6H09nPjtR4++NUsmwklj0osiqIoZUxiWhY/7bnAor2XSM7Mpo+fO6/18MG3pqOpQysSlVgURVHKiIQ0HT/tucji3ITSt6mWUJp4mEdCyaMSi6IoioklpOn4cfdFFu+7REpmNv383ZnY3fwSSh6VWBRFUUzkn1QdP+65wJJ9l0nJzKa/vwcTezSgsbt5JpQ8KrEoiqKUsvhUHT/uvsCSfZdIy8qhn78Hr3X3oZG7g6lDMwqVWBRFUUpJRlYO3+w4z0+7L5CWlUN/fw9e6+FDQ7fykVDyqMSiKIpSCrafieG9oDCuxqfTv5kH/+nhg085Syh5VGJRFEUpQTcS0pmx/hRbTkXTwNWen//djnb1qps6rBKlEouiKEoJyMrRs2jvReZsPYteSt7q04gXOtXD1sbK1KGVOJVYFEVRjCzkUjz/+yOMiOhkejZxZfpAP+o4VzZ1WKVGJRZFURQjiU/V8cmm0/waco1aTpX4/rnW9PJzN3VYpU4lFkVRlGLS6yW/HbnKx5vOkJKRzYtd6jGphw+VbS3zLdYyX7WiKIqRnL6ZxLtrwzhy+R/aejvzwRNNy818lEelEouiKMojSMnMZs7fkSzad4mqlSrw+dPNebJVLYQQpg7N5FRiURRFMYCUks1hUcxYH05UUgYj2noypU8jnCrbmjq0MkMlFkVRlCK6ciuN94LC2BERSxMPR74Z1YpWntVMHVaZoxKLoijKQ2Rm5/DdzgvM334OGyvBtAG+jGnvhY11+Z+T8ihUYlEUI0tMz2L/+Vt0bFAdB7sKpg5HKaa95+KYtjaMC3Gp9Pf3YNoAX9yr2pk6rDJNJRZFMYKkjCy2hkezMfQmu87GkpUjeaJFTeYMb2nq0JRHFJOcwawNpwk6cQOv6pVZMr4tXRrWMHVYZkElFkV5RCmZ2Ww7Hc2G0JvsjIhFl6OnZlU7xrT3JiUzm58PX2VUOy8CvJ1NHapigBy9ZPmBy3y+JYLMbD2TevgwoWt97CpYmzo0s6ESi6IYIDUzm21nYtgYeoPtEbHosvW4O9oxqp0X/Zt50LKOE1ZWgjRdNjsjY5kedIqgVzthbaVKUM3BiasJvLs2jJPXE+ns48LMwU2p61LF1GGZHZVYFOUh0nTZBJ+JYWPoTYLPxJCZrcfVoSIj23oyoJkHrTyrYXVX4qhsa8M7/ZowcdUxfjl8lZGBniaKXimKxPQsPt8SwfKDl6lhX5GvR7RkQDMPNSflEanEoiiFSNflsD1CSybbzkSTkaWnhkNFhrepQ/9mNQnwujeZ3G1AMw+WH7jM7C1n6OfvruY5lEFSStYev86HG08Tn6pjTHtv/turoSq6KCaVWBQlV0ZWDjsiYtl48ibbTkeTpsvBxd6Wp1vXoX8zD9p4OxvUpSWE4P1BfvT/ajdf/h3JjMFNSzB6xVDnYlKYtjaM/Rdu0byOE4vHtaVpraqmDqtcUIlFsWgZWTnsitSSydbwaFJ1OThXseWJlrUY0MyDwLrVizU+0sTDkVHtvFh24DIjAj1p7O5oxOiVR5Guy2He9rN8v+sClSpY8+GQpgxv46nGwYxIJRbF4mRm57A7Mo6NJ2/yd3g0KZnZVKtcgUEtatLfvybt6jkbdeLbG483JOjEDaavO8XP/26n+u1NKPhMNO+tO8W1f9IZ2qoW7/Rrgot9RVOHVe6oxKJYBF22nj3nYtkQepO/T0WTnJlN1UoV6O/vQf9mHrSvX50KJTSL2qmyLZN7NeLdtWFsPHmTAc1qlsjzKPcnpWTG+nAW77tkMdsDm5JKLEq5lZWjZ8+5ODaG3uSvU1EkZWTjaGdDn6bu9G/mQccGLiWWTO42oq0nKw9e4aONp+ne2NVi9+kwlbnbzrJ43yXGdfRmat8mFrE9sCmp326lXMnK0bP//C02ht5k86koEtOzcLCzoZevOwNyk4kp3lSsrQQzBvvx9IL9LNhxnjd6NSr1GCzVioOXmbP1LE+1rs17A3xVV2QpUIlFKTfCbyQxYcURLt9Kw76iDb183ejfzINOPi5UtDH9rOk23s4MblGTBbsu8FTrOnhWt5w90E1lc1gU09aG0b2xKx8P9VdJpZSoxKKUC+uOX2fK6lCcKtny7bOt6NbYtUwuwTG1bxP+Do9m1sZwvh8dYOpwyrWDF27x2s/HaF7HifkjW5Vat6cC6ietmLWsHD0z14cz6efjNKvtxPqJnejr71EmkwqAe1U7Xu3egL/Co9kVGWvqcMqtM1FJvLA0hDrVKrFwTBsq2ZbN34fySiUWxWzFJmcy6seDLNx7kfEd67LihUBqOJT90tHnO9XFq3plZqw/RVaO3tThlDtX49MY/dMhqtjasPT5QKpVUSselDaVWBSzdPxqAgO/3sOJawnMGdaC9wb6mk1XR0Uba94b4Mv52FSW7Ltk6nDKlfhUHWMWHiIjK4cl49tSy6mSqUOySCb7SxRCNBFCbBRCJAohEoQQa4UQ3o94rcFCiKNCiAwhRLQQ4ishhL1xI1bKip8PXeGZBfupYCNYPaEDT7SsZeqQDNajiRvdGtVgztazxCRnmDqcciFNl824xYe5npDOT2Pb0MjdwdQhWSyTJBYhRC1gNxAIfAJ8DnQDdgkhnAy8VntgDeAETM39/tXcr0o5kpmdw9Q1oby95iSB9ZxZ/2on/Gqa79pO0wb4kpmdw+zNEaYOxexl5eiZsPwoJ68l8PWIlrRRe+CYlKmqwqYD1YFuUsodAEKIs8DPwGvATAOu9Q5aguwvpTyde60k4C0hRB8p5WZjBq6Yxs3EdF5afpQTVxN4pVt93ni8kdmv7VSvhj3jO9Xlu50XeLadFy3qGPSZSsml10um/B7KzshYPhnqTy8/d1OHZPFKvcUihKgADAMi8pJKrtVAIvCsgZdsBcTlJZVcv+R+HfyocSplx4ELtxj49R7ORSezYFRr3uzd2OyTSp6J3X1wdajI9HVh6PXS1OGYpU83n2HNsev89/GGDG+r9r0pC0zRFdYMcAQO3XlQSpkNnAAaCiEMacc6AVl3HYvJ/er3qEEqpielZOGeizz740EcK1Vg3asd6dO0fH0ata9ow9R+jTlxLZHfj14zdThm58fdF/hu1wVGt/fi1e4NTB2OkssUicU79+uNQu6LuuucorgEuN41NpOXUFwMCUwpO9J1Obz+y3Fmbgine2NX1r3SkQau5XMw9okWtWjl6cRnm8+QlHH3ZyTlftYeu86sjafp5+/O9IF+alZ9GWKKxJK3jkVhf0G6u84pipWANTBXCNFQCPE48GHufYVuAyeE+LcQIkQIERIbqyaplTVXbqUx9Nt9rDtxg8m9GvLdqNblekc/IQQzBzflVqqOuVvPmjocs7AzMpbJv52gfb3qfDmsRbnpGi0vTJFYUnO/FlbOU/Wuc4ri/4CNwGggAm2s5tPc+1IKe4CU8nspZYCUMqBGjRoGPJVS0nZGxjJw3h6u/5PGwrFteLW7z0O3AC4PmtaqyvA2nizZd4mz0cmmDqdMO3E1gQnLj+Dj5sB3o1uXiXXglIJMkVgu5H51LeS+vHf5S0W9mJQyQ0o5AGgCdAXqACF3PZdSxkkpmb/9HGMXHcKjqh3rJ3aiW6PCfkXKr8m9GlLZ1poZ68ORUg3kF+ZCbArjFh+mur0tS8a1wbEct2TNmSkSSxha9VfnOw8KISoBLYHTUsp/inoxIUQ/IcRgKeUZKeVOKWUikLe63x5jBa2UnOSMLF5afoTZWyIY2Kwma17ugFf1KqYOq9RVt6/IG483ZM+5OP4KjzZ1OGVOTFIGoxceQgBLxwfi6mhn6pCU+yj1xJJb/bUKqJk7HpJnCFARbczEEK8Cvwsh7iwX+jeQgTYvRinDzsWk8MT8vWw9HcO7/Zswd3gLi94Ea1Q7Lxq5OfDBhnAysnJMHU6ZkZSRxeiFh4hP1bFoXBvquljeBw9zYqolXT4AbgE/CyGmCiHeBRYAV4GvAYQQDYQQY4UQD6sh/D+0iZ4bhBCvCCFWAz2B6VLKmyX3EpTi2nIqiifm7yUhLYvlzwfyQud6Fl/ZY2NtxfRBvlz7J53vd6meXICMrBz+tSSE87EpfPdca5rVVhNJyzqTJBYp5Q2gE3AAbRmWN4Fg4LHcrixy71+U+/VB19qGNnBfFfgC8AVelFJ+VjLRK8WVo5fM3nKGF5cdoX6NKqyf2In29dX+43k61Hehv78H3+w4x/WEdFOHY1I5esnrvxzn4MV4Pn+6OZ19VLGNORCWPkgYEBAgQ0JCHn6iYhQJaTom/XycnZGxDAuow4zBfmV27xRTuvZPGj2/2EmPJm7MH9nK1OGYhJSSaevCWH7gCtMG+PJ8p7qmDkm5gxDiiJSy0N3qzGOdcaVcCL+RxKB5e9l3Po6Phvjz6VPNVFK5j9rVKjOhSwM2ht5k3/k4U4djEl8Hn2P5gSu82KWeSipmRiUWpVSsO36dod/uJTM7h19ebM/IQLWm08O82KUetatVYkZQONkWtiHYyoNX+OLvSJ5sVZu3+zQ2dTiKgVRiUUpUga2Da2lbB7fyrGbqsMyCXQVr3u3vS0R0MisOXjF1OKVmy6ko3l17km6NavDJk/4WX9BhjlRiUUrMnVsHj+3gzYp/BeLqoOYeGKK3nxudGrjwf39FEJ+qe/gDzNzBC7eYuOoYzWo7Mf/ZVmazK6hSkEH/a7m7NL4mhFClGcoD5W0dfPxqAl8Oa877g/zUm8QjEELw/iBf0nQ5zN5SvjcEOxOVxAtLQ6hdrRILx7ax6PlM5s7Qv3QXYA5wTQixTgjxpBDCtgTiUsxY3tbBNtba1sFDWtY2dUhmrYGrA2M6ePPz4SuEXU98+APM0LV/0hiz8BCVba1ZOr4tzlXU24o5MyixSCk90eaVfIu2wdZvwE0hxDdCiHYlEJ9iRrStg08W2Dq4aS3z3Tq4LJnU04fqVWyZHnSq3K0jFp+qY/TCQ6Trclg6PpDa1QxZ3Fwpiwzum5BS7pNS/kdKWQfogrYEy2BgrxAiQgjxTu6e9ooFiUrMYNh3B1h16AoTutZn8bi2VFOfOo3G0a4Cb/VuzJHL/7DueGFbGZmnNF024xcf5to/6fw4pg2N3MvnnjuWplid3lLK3cCPaMklC/ABZgHnhBBPFz88xRzsOx/HgK93czY6mW+fbcWUPuVn6+Cy5KnWtWleuyof/XmalMxsU4dTbFk5el5ecZTQawl8PaIlbesasnGsUpY9UmLJXcdrmhAiHDgK/AfYDowCPIB1aGt4KeWYlJIFO88z6seDVK1UgbWvdKSvv4epwyq3rKwE7w/yIyY5k3nB50wdTrFIKZmyOpQdEbF8OMSf3n7la8tpS2dQ2YUQ4g1gBNr4igCOA5OBlVLK6DvO2wk8YcQ4lTImKSOLN387wZZT0fTzd+ezp5pjX1FV8ZS0lp7VeKp1bX7ac4FhbeqY7Sq/n2w+w5qj13nj8YaMaKsmy5Y3hrZYPgfcgM+AplLKVlLKL+9MKrmuoiUcpRyKiEpm8LzbS93PH9lKJZVS9FafRlS0sWbm+lOmDuWR/Lj7At/tvMBz7byY2P1hi5cr5sjQd4MewA75kLIUKeWGRw9JKcvWHb/O26tPYm9nw8oXAgmsp1YlLm2uDnb8p6cPszaeJvhMNN0bu5k6pCJbe+w6szaepm9Td94f5Kdm1ZdThpYbb39YUlHKJ122nveDTjHp5+M0reXIxomdVFIxodHtvalfowoz14eTmW0eG4Ltioxl8m8nCKzrzJfDWqgCj3LM4MF7IYSPEOKjO/49SwixQgjRxLihKWVFVGIGw7/fz+J9l3i+U11W/qud2hbWxGxtrJg+0I9Lt9JYuOeSqcN5qBNXE3hp+RF83Bz4YUyAWtW6nDN0SRdf4Agw5Y7D1dEG9PcLIeobMTalDMgrJT4Tlcy8kS2ZNsBXLc1SRjzWsAa9fN34OvgsUYkZpg6nUBlZOWwOi2L84sM4V7Flybg2ONpVMHVYSgkz9B3iQyAZbetfAKSUE4AmQFLu/Uo5cHcp8bpXOjKgWU1Th6Xc5d3+vmTrJZ9sOm3qUPJl5+jZfTaWN387QZsPt/LS8iPY2lixdHxb1dK1EIYO3gcCX0kpt995UEoZIYRYALxstMgUk0nOyOLN30LZfCpKlRKXcZ7VK/PiY/X4Ovgco9p5EeBtmkmGUkqOXkkg6Ph1Np68SVyKDvuKNvTyc2NQ85p0bOCiWroWxNB3i6rA/dbuzgDURhtmLjI6mZeWHeFyfBrv9m/C853qqsqdMm5C1/qsPnKN6UGnCHq1U6kNikspOROVTNCJG6w/cYNr/6Rja2NFj8auDGpek26NXdVYioUyNLEcBcYLIeZLKTPzDgohKgLj0MZfFDOVV0pcpaIqJTYnlW1teKd/E15deYyfD1/h2UCvEn2+y7dSCTp+g6ATNzgbk4K1laBjAxf+07Mhvfzc1BiKYnBi+QDYBIQKIRYBN4BawFigPtDXqNEppUKXreejP0+zeN8lAryqMf/ZVripvnCz0t/fg2V1L/P5lgj6+3vgVNm4C4DGJGWwPvQmQSducOJqAgBtvKvxwWA/+vp74GJf0ajPp5g3Yei0FCHEc8BcwAmQaEu7JACTpJTLjB5hCQsICJAhISGmDsNkohIzeGXlUY5c/ofxHesytV9j1Rdupk7fTKL/V7t5rp0XMwY3Lfb1EtJ0bAqLIuj4DQ5cvIWU4OvhyKAWNRnYvCa1nCoZIWrFXAkhjkgpAwq7z+ARWSnlMiHEaqAD4ArEAPuklGnFC1MpbfvP32LiqqOk6XL4ekRLBjZXVV/mrImHI6PaebHswGVGBHrS2N3R4Guk6bL5Ozya9SdusDMylqwcSV2XKkzs7sOg5jVp4GpfApEr5c0jlfrkJpGtRo5FKSVSSr7fdYHPtkTgVb0yq/7VDh83tQ9GefDG4w1Zf+IG09ed4ud/tytS4YUuW8+uyFiCTtzg7/Bo0rNycHe0Y0x7bwa3qEXTWo6qgEMxiMGJRWi/YY3QWit3qgQMkFJONEZgSsm4s5S4b1N3PnuqGQ5qsLXccKpsy+TejfjfH2FsPHnzvnOPcvSSgxduEXTiBpvCokhMz8KpcgWGtKrFoOY1aevtjJVackV5RIYum18XCAJ8H3CaSixl1J2lxP/r14QXOqtS4vJoeBtPVh68wkcbT9O9sSuVbbU/cyklJ64lEnT8BhtCbxCTnEllW2t6+boxuEUtOvmouSaKcRjaYpkNuANvA68Dh4E1wGi0MZcRRo1OMZqgEzeY8nsoVSrasOKFQNqpUuJyy9pKMGOQH08t2M+3O84zsHlNgo7fYH3oDS7fSsPW2oqujWowqEVNejR2o5KtmmuiGJehiaUj8IWUcrYQojrQVkq5RAjxM3AM6A6sNXaQyqNTpcSWKcDbmSda1OTr4HN8HXwOKwEd6rvwStcG9G7qTtVKqvtTKTmGJhZ7ID73+wPAvwGklJlCiOXAK8BrxgtPKY7opAxeXqGVEo/r6M07/Zqorg4L8k6/JugltPJ0ol8zD1wd1AcKpXQYmlgigCeFEAvRZtk7CSF8pZThQCraki9KGXDgwi1eXXmMNF02X41oySBVSmxxXB3t+GpES1OHoVggQxPLl8AyYIWU8hkhRBjwsxBiLfAv4KSxA1QMI6Xkh90X+HRzXilxoGWVEv9zGSo5gZ36jKMopmJQYpFSrhBCWKOVGwNMANYB76K1WKbc77FKyUvOyOKt30PZFGaBpcRSwqEf4K//gWNNeG4tONc1dVRKWZKZAglXoGot9cGjhD3KzPuld3y/VwjhhZZozkspE40ZnFJ0kdHJvLT8CJdvWWApcUYiBL0G4WuhXle4eQIW9oHn1oCbn6mjU0xNlwaHvoe9cyD9H+1YRUeoWvuuW53b3zt4gLWFfCgrAcXeZENKmYq26rFiIutP3GDK6lAq21pgKfHNE/DrGO2T6OMzof1EiIuAZUNgUV949neo09bUUSqmkJ0JRxbDrs8hNQYa9AT/Z7TvE6/l3q7C9SOQdqvgY4WVllzuTjh3fm9XFSzlw5uB1O5NZu73I9eY/NsJyysllhJCFsLmqVC5Ooz7Ezzbafe5NoHxW2DZE7B0MAxbpr2pKJYhJxtOrISdn2mJw6sTPLMUvNrf/zG6VEi8rp2fn3TuSDyngyDnrq2obB0Kb/U41bH4Vo9BqxsLIfYAW6SUH5RcSKXL3Fc3HvLNXtJ1Oayf2MlySokzk2H9fyDsd6jfA4Z+D1Vc7j0vJQaWD4WYM9o5TYeWfqxK6dHnQNga2PERxF+AWq2h+zSte7S4LQu9HlJjc5PNlXuTT+K1h7R67mrxONcH53pgbb6f7Y25urED2v72ShlwMzGdY1cSmNyroeUklehT8Oto7Y2j+zTo9AZY3ee127vCmA2wajj8Pl4biwkYV7rxKiVPSjizAYI/hNjT4NYUhq+CRn2N11VlZQUObtqtduvCz9GlQdJdrZ6Eq7mtnqNwen3BVo91RXBtrMXr6quNB7r5ab+3Zs7QxLIMmCaEqC6lvPXQs5UStSUsCoC+/h4mjqQUSAnHlsGfb2p926ODoG7nhz+ukhOMWgO/jYEN/4H0eC0Zqb5x8yclnNsGwR/AzeNQ3QeeWgi+Q+7/YaMk2VYGFx/tVpj8Vs9ViDsL0WEQEw7ntsLxFbfPq1JDSzCuuYnGzRdqNIYK5rP/zaPMY+kGbBNCPCmlPF8CMSlFtCksioZu9tSvUc73yNClwoY3IPRnqNsFnvzRsE91tpVh+EpYOwG2zYS0eOg1SyUXc3ZpDwTPgiv7wckTBn8DzYaV7a6lAq2eu3qQUuO01nj0KYjJ/RryE2RnaPcLK6jeILdl01RLNm5+UNXTNEn0IQz9X/gBiAW6AOFCiHVAyh33Synl88YKTrm/2ORMDl2KZ2L3+3w6Ki9izmitjdgI6DoVHnsTrB5h0UTrCjDke6hUDfbPg4wEGDC3bL8RKfe6FqIllAvbtfGL/v8HLUeDjXG3Yi51VVygXhftlkefA/EXb7dsok9pLbPwO5ZjtHXQilXyutHc/LTkU8mp9F/DHQz9qxp717+fuuvfElCJpRT8FR6FlNC3qbupQyk5x1fBxjfAtgqMzp2jUhxWVtD3My257PwU0hPgyZ+ggoVU0pmzqJPaGErkJq0KsNeH0OZ5s+oeMpiVNbg00G5+T9w+npmsfeDKa9lEn4JTf8CRRbfPcaxdMNm4+WktnlKqUjN05n3Za3NZqM1hUXhXr0xj93K4XIsuDTa9CceWa6WiT/0EDkZKoEJAt3e05LL5bVj5tNZNVrEc/hzLg7izsP0jOLUGKlaF7u9C4EuW/f9V0QHqtNFueaSEpBu5LZuw3IQTDue3gT5bO8faFlwa3R63cfMD9+ZgX8PoIap+ADOUkKZj//lbvNC5XvmbXR93Vqv6igmHzpO17q+S6K5qN0FLLmtfhiWDtImUVSxoYmlZ988lbR7KiVVgU0n7XejwqvZ/ptxLCG2pmqq1wOfx28ezdXDrbG6iCdOSzcVd2nglQPtXofeHRg9HJRYz9Hd4NNl6ST//ctYNFvobrJ+kdU2NWl3ykxqbD9cqzH4do83Sf+4P7Q9TMZ2km7BrNhxdqg1Yt3sZOv6nRD5VWwQb29tdYTxz+3havPbhrUrJ/FwN3Zo4+CGnSCllj2LEoxTB5rAoajlVwr9WOVlILytD65Y6sgg822vjHqX1Bt+or7am2MrhsLC3tnilS4PSeW7lttQ42PMlHP5R67ppNVor1HBU2z2UiMrO4N2pxC5vaItFjzZAn8cRbQFKR+AicMFIcSn3kZyRxe6zcTzX3qt8dIPdOq9VfUWdhI6TtEmPpb0MhncnGLsBlj+Zm1zWgEfz0o3BUqUnwL6v4cC3kJ0OzYZD1ylQzdvUkSnFYOjg/T19E0KICsAk4C3gP0aKS7mP4DMx6HL0t6vBdGnaootu/uZXOnvqD1g3Uat+GfELNOpjulhqtoDxm2HpE7B4AIz4Gbw7mi6e8i4zBQ5+qyWVjETwGwJd34EaDU0dmWIExljdOAv4XAiRAswG+hU7KuW+Np2MwtWhIq08cwcx/5yszdqtWBXqd9XGJer3KNtjBdmZ8Ne72lLmtdvAU4u0hftMzcUHnt+irYy8fCg8vcS0ya48ykrXFg/d/QWkxaZa5dMAACAASURBVEHDvtD9f+Dub+rIFCMy5kfcdcAnRryecpc0XTY7ImN4unUdrKwExJyG4yuhySBtEPrcNghfp53s6gcNemiJxrMd2FQ0bfB5/rkEv42FG8e0ipQe08vW5LaqtWHcZljxJPw8EoYsgGbPPPxxyoNl67QleXbNhuSb2pyk7tPunYGulAvGTCwjgCwjXk+5y86IWDKy9PTNqwYLnqXVtA+cqw3GSXl77aFzW7V+631fQYUqUPex24nGVDsrnl4Pa1/Rvh+2ApoMME0cD1OlOoxZD6tGwJp/aZtDBb5o6qjM16U9Wll3wmWoEwhDfyjaOm+K2TJWVVhNwAdYcZ/7FSPYFBaFcxVb2no7w9XD2oqu3d7Vkgpotex5pYUdJ2n92Jd2a0nm7N/arGXQZuA26KndvDpqa2mVpGwdbJ0OB76Bmi3h6cVlf3C2ooM2t+X38bDpLS25dJmi1hcz1MnftTXanLy0n2eDnupnaAEMbbHUo2BVWJ4UYD7wbrEjUgqVkZXDttPRDGxeExsrAdtmaDXo7Sbc/0EV7bVy2kZ9tdbMrfO3WzNHFsPBBdrS3d4dbycal4bG/cNPuAK/jYPrIdD2Rej1QdnplnuYCnbaBlHrX4MdH2u1/30+KZOL/pU5UmoD839P0z68DF+hJjdaEEOrwrxLKA7lIfacjSNVl0Ofpu5wPlhrifT5VEseRSHE7XWH2r2kDaJe3qeNy5zbClve0W5VPW93mdV9DOwcHz3oiM3wx4vaYnpPL9Yqf8yNtQ0Mmldw8crB8y12Z8Ai0edov0sHF2j/508sUOuxWRgzq0+1XJvConCws6FDveqwcKaWAIqzaVWFSrkJpAfwkdayyEsyJ3/TJita2UCddto5Po9ry3UXpTWTk6UtT7/vK63a5+klUL3+o8dqalZW2jL7lappe39kJGqJsjwvgPiostJhzb+1rXzbvaL93FQLz+IYnFiEEJ2BCVLKkbn/XgD4A/+TUu4wbngKQFaOnq2no3m8iRu2keu1pbOf+Na4XUpOuYkqYJw2JnLt0O1us20ztJu9W26XWQ+o1+322M6dEq/D7+Pg6kEIGA+9Py4fn1aFgMcma8uRb5wMy4bCyJ+1ajxFkxavFTxcPQi9P4L2r5g6IsVEDB28bw9sA6yBkbmHY4GmwBYhRKCU8rhxQ1T2n79FYnoW/fxqQPAL2m5yzYaV3BPa2Gqz0b07Qc/3ITlK6347txUi/tTmzQgrbU/xBo9ryaZmCzi/XauiytFpy7L4372rQjnQ5gWwc9K6+Bb313anLAdbyRbbP5dhxVNaOfnTi8yz21MxGiFlYWPx9zlZiL+AOkA/KeXFO467ADuBi1LKMlpDWriAgAAZEhJi6jAeaOqakwQdv87xgVFU2DjJtKW6+hxt/+681sz1I4DUuonS/9Hmzzyz5P7bs5YXZ7fCL6O0taxGr9VafJbq5glY8bS22+HwVWrFAgshhDgipSx0IpKhiSUW+EhK+WUh970BTJFSuj1ypCZQ1hNLjl7S9sOtdK7rwJyY8dqueS9sLTslm2nxua2ZbdoueF2nlnz5cllx5QCsfEabJ/TcH+Da2NQRlb5z27RtDuycYNTv2m6GikV4UGIxdFStInC/chhbQI1mGtmhi/HcStXx70rBkHQdek4vO0kFtHEW/6dgyLdaKbGlJBXQVjQY+6e2Gu+ivnDtiKkjKl3HV2qJtZq39mFHJRUll6GJZR/wihCiwCL+uV1hE4C9xgpM0WwOu4mzTQZNzv2gDZjXfczUISl3cm+qrS9W0QGWDIQLO0wdUcmTEnZ9rk189OoI4/4ERw9TR6WUIYYmlmmAG3BGCLFACPGeEOI74DTgigETJIUQTYQQG4UQiUKIBCHEWiGEt4Hx5F3LSwjxqxAiRghxQwixSAhh9iOqer1k86koZrhsR6THQ4/3TB2SUhjnejB+izbOsuJpbema8ionGza+oZVd+z+jzaZXlXHKXQxKLFLKw2irF98C/g28D/wLiEcb0C9SX4AQohawGwhEW7jyc6AbsEsI4WRITEKI2kAI0B5tdeVvgIHAQSGEWf/GH7uaQHZSDH2SV4PvYKjVytQhKffj6KF9cvdoro05HF1m6oiMT5emFSyELIROr8OQ78rWAqJKmWHwPBYpZTDQUAjRAK2VEiOlPGfgZaYD1YFueXNfhBBngZ+B14CZBlxrIuACdJZS7sm91hngN2AcMMfA2MqMzWE3mVhhHTb6TG0lWKVsq+wMo9dpb75Br8LVA/DYW1DNy9SRFV9qHKwcplUB9vsc2v7L1BEpZdgjT4mVUp6TUu4zNKnkbgw2DIi4a0LlaiAReNbAUPJKce5sLR256z6zI6XkSGgoz1pvQ7QYWf7Ld8sL2yraJmHtXoHQX+Hr1rDhdUi8ZurIHl38BfjpcYgOg2HLVVJRHsrgxCKE6CyEWHnHvxcIIfYKIboW8RLN0LYyPnTnQSllNnACrTVUyJTu+4rK/ep+x7Gad91ndsKuJzEidYW270rXt00djmIIm4rQ5yN47Ti0HqN1i33VEv58E5Jumjo6w1w/Aj8+rs1RGh1Udrc6UMoUgxLLHTPv75z2fefM+xZFuIx37tcbhdwXddc5RfENkAosEEI0E0K0Ab7Mjeunwh4ghPi3ECJECBESGxtrwFOVnkOH9jLUejdZrZ7XNp9SzE/VWtD//+C1o9BipDY28VUL2DwVkqNNHd3DRW7Rtmm2rQLP/w2egaaOSDEThrZYZgDngQZ5B6SU04D6wDlgVhGukTfRobBNwXR3nfNQUsoTwNNAF7QWzyG0LrABUsqr93nM91LKACllQI0aNQo7xaSklPic+gqdVSXsur1p6nCU4nLy1DZjezUEmj4FB7+Duc217ZlT40wdXeGOLNHW/XJpqCUV1RWrGMDQxNIS+P7O5VwApJRxaK2DNkW4Rmru18Iqtqredc5DCSGeAP4AtqDtYjkaLcH8ndvCMjtXQnfzWM5+IuuP1XYzVMoH57rwxHx49bBW5bd/PsxpBltnaCsYlAVSwvaPtD1o6neDsRvBwawW01DKAFPMvL+Q+7WweSZ5zYdLRQlGCGEN/AicBYZIKX+WUi4DeqO1fuYX5TpljQieSZx0pGbf/5o6FKUkVK8PQ7+Dlw9qm7Dt+VJLMMEfamMZppKTBetehZ2fQotRWhFCUff7UZQ7mGLmfRha9VeBTa+FEJXQWkSnpZRF/etyRStbPiOl1OcdlFKmoSUn3yJep+w4vx3PxMOsdxxJjeoupo5GKUk1GsJTP8HL+6FBd9j1GcxpDjs+1fZ8KU2ZKbBqOBxfDl3ehsHz1GZmyiMr9Zn3udVfq4CaQojH77hrCFqLaGWhDyxcLJAMBAoh8sdlhBAeQBPg4v0eWCZJScaW6VyTLog2400djVJaXJtoWyC/tAfqdoYdH2ktmN3/B5nJJf/8ydGwuJ+27cHAr6Db1LK1Hp1idkwy8x74IPcaPwshpgoh3gUWAFeBrwGEEA2EEGNzJ2LeL55stAqwOsAeIcTrQoi30Gb1VwE+NeT1mdzpIOxiTjA3eyi9mlnwMuyWyt1f2xv+3zu0BS63zdQG+ffOBV2Rhx0NE3dWm6MSdxZGrNLKoxWlmAyexyKlDJZSNgQaAp2AhlLKRlLK7QZc40buYw8AU4E3gWDgMSllXh9AJ2BR7tcHeR9twD4bbUb/W8B1tKqwxUWNyeRysiF4Fles63DeYyA1ndRC0RarZksY+Qu8EKx9//d7WoLZP1/b+tdYrh6Cn3ppSWvsBmjY23jXViyaQfuxAAghHIEO3Dv4XgntzXygkWIrFWVmP5Zjy2HdK7yo+w+teo/mxS5mvEe8YlxXDmrdYxd2gL07dH4DWo0p3pbPpzfA6ue1jcpGrdYW0lQUAzxoPxZDtyZuAWyiYFIRQF52MixLKZqsDNj+MTGOfmyJacP/mqolyJU7eAZqa5Bd2quVAm96C/bMgcf+Cy2f02b6G+LQD9o1arbSWkZVVJGIYlyGdoV9AqQDw9EGxn9GW5V4BVp5bzejRmcpQhZC0jXmWz2Lr0dVPKtb0GZZStF5d9S6rEYHgVMd2PhfbS2yI0u0UuGH0evh7+nw52Tw6Q1j1qukopQIQxNLa2CelPI3YBngLaXcCYwBQoHnjRxf+ZeZDLs/J9PzMZZEedO3qfvDH6NYLiGgXhdt/5dRa8DeTZvM+HVrOLZCG6srTLYO/ngR9s6B1uO0xSQtabdPpVQZumy+DZCR+30I8F8AKaUUQvyCNhCvGGL/fEi7xd8eL0Ik9PU378SSmZlJfHw8ycnJ5OTkmDqccq4mdJqnDehnJEKGDkJ2gZ0jVKh8u2RY6rWlY2o+BfVe0O6PPGva0JUyxdraGgcHB5ydnalY0cCu1UIYmlhOAuOFEL+hrcllL4RoK6U8hDbzvvgRWZLUONg3D5oMZPnV6vi46mjg6mDqqB5ZZmYmV65coVq1anh7e1OhQgWEmg9ROqTUkktyFGSna3/Z9m5ga68te+9UFZyaQmW1RJBSkJSSrKwskpKSuHLlCp6ensVOLoZ2hX0ENAd+kFLGos20XyOEWAq8AxwsVjSWZvcXkJVKQrspHLoYb/bdYPHx8VSrVg0XFxdsbW1VUilNQkAlJ6jRCKrVBQQkXIaYcMjJ1Kq+VFJRCiGEwNbWFhcXF6pVq0Z8fPHXrTN0guRmoBfanBOAF9EWjBwFJABvFDsiS5FwFQ7/CM1Hsim6KnoJfcy8Giw5ORlHR0dTh2HZ8hNMY6jmDRUdobqP1v2lKA/h6OhIcnLxV3t4lK2JtwPbc78PBxoJIZyllGVkeVYzsfMTQELXt/lz9Q28qlemiYf5doMB5OTkUKGCWl+qTBACKlXTbopSRBUqVDDK2Ogjb018J5VUDBQbCcdXQpsXSLR1Z//5W/Rp6l4uuo7Kw2tQFEtlrL9foyQWxUDBH2hVO53/y9+no8nWS/qaeTeYoihKHpVYStv1o3A6CNq/ClVc2Bx2k5pV7Wheu7B9zxRFUcyPSiylbdtMqOQM7V8hJTObXWfj6NPUQ3UhKYpSbqjEUpou7IQL2+GxyWDnSPCZGHTZerOfFKkoinInlVhKi5SwbQY41oYAbeWbTSdvUsOhIq09VeWOoijlh0ospeXMBrh+BLq+DRXsSNflsCMilt5+blhZqW4wRVHKD5VYSoM+B4JngUtDaD4CgJ2RMaRn5ahqMEVRyh2DJ0gqjyD0F4g9A08vAWvtR74pLIpqlSsQWNfZxMEpiqIYl0osJS07U9ucyaMF+A4GIDM7h+DTMfTz98DG2jIajTPWnyL8RpKpw3gg35qOTB/oZ5Rrbd68mQ8//JCTJ09iY2ND69at+eSTT2jZsmX+OYcPH2batGns3buXihUr0rJlS2bMmEGHDh0KXKso53l7e+Pt7c2OHTsKPLZnz55kZ2cXOC6EYMyYMfz444/MmTOHpUuXYm9vz759+wo8dsWKFXzxxRdERERgb29Px44dmT17NvXqFdxt8q+//uKDDz7gyJEjVK1alcDAQD788EP8/LSfZXBwMD169GD27NlMnjw5/3FSSry8vHB2dub48eOP9HNWyibLeFczpZBFkHgVek7PX8Z877k4kjOz6aOqwcql4OBgBgwYQGpqKrNmzeLtt98mIiKCPn36kJCQkH9Op06dOHXqFO+88w5Tp07lypUrdOnShZ07dxa4VlHOM5Rer2fQoEHMmDEDX19fevcuuN/90qVLGTVqFE5OTsyePZuJEyeyc+dOBg4cWGDJj2XLltGnTx9u3brFzJkzmThxIocOHaJdu3ZEREQA0LVrV2rXrs2qVasKPMf+/fu5evUqY8aMeeTXoZRRUkqLvrVu3VqWmIwkKT+tJ+XiAVLq9fmH//vrcdl0+maZmZVTcs9tAuHh4aYOoUyYM2eOHDBggLx582b+sV9++UUCcuPGjVJKKX18fKSTk1OBc86fPy8BOWzYsPxjRT3Py8tLdunS5Z5YevTocc9xQFauXFn6+/vL6OjoQl/D1KlT5dChQ2VGRkb+sU8//VQC8tSpU1JKKVNSUqSTk5OsX7++TE1NzT9v27ZtEpBTpkzJPzZlyhQJyIiIiPxjkyZNkjY2NjIqKqrQGBTTKOrfMRAi7/O+qrrCStKBbyEtDnrcbq1k5ej5Ozyank3csLVRDcbyaNKkSUyaNImYmBg2btzI4cOHWbZsGQAxMTGcPXuWs2fP8sILL+DufrvVWq9ePbS/V01Rz3sUOp2O33//HVdX10Lv/+ijjwC4evUqISEhHDhwgKVLl+a/Bl9fX/bt20dCQgKTJ0+mcuXbu1F27979nviee+45Pv30U1atWsX06dORUvL777/Tu3dv3NzcivValLJHvbOVlNRbsPcraDwAagfkHz5w4RaJ6Vn0MfO9V5T7Cw8Pp3Pnzri5uTFs2DC2bNlC27Zt8++PiYkBoFatWg+8TlHPe5CkpMLHtQYPHkzDhg3v+7i9e/fSrFkzPD09GTduHAcPHiQgIKDAOYbE5+fnR4sWLfK7w/bs2cP169cZPXp0UV+KYkZUYikpe7RNvOg+rcDhTWFRVLa1pkvDGiYKTClpQ4YM4fz58+zbt4+kpCT279/Piy++mH9/Xivh+vXr9zx22rRp+ecW9bz70el0+eMcd7O3t7/v41JSUhg0aBB6vZ7Q0FASEhLYsWMHTz75ZIHzHhTfhAkT+N///lfg2HPPPUdERATHjh3j119/xcnJiUGDBj3wNSjmSSWWkpB4HQ79AM2Gg2vj/MM5eslfp6Lo1tgVuwrWJgxQKSm3bt0iMjKS7t270759e6ysrNDr9SxcuDD/HB8fH3x8fFizZk3+p36AuLg45s6dS2hoqEHngZYobty4USCWr7766r4tlgeJiIggPj6eJ554An9/f0DbdjqvOy9Phw4dcHJyYvHixWRkZBR4/IIFC7hw4UKB80eOHIm1tTUrVqxg9erVPPPMM9jZ2Rkcn1L2qTGWknDHJl53CrkUT1yKzuy3IFbuz9nZmdq1a7N27Vo+/vhjbG1t+eWXXzhx4gQA6enpACxYsIC+ffsSEBDAhAkTsLW15aeffiI9PZ1Zs2blX6+o53Xr1o158+bxf//3f4wYMYKgoCA+/fRTvL29DX4N9evXp0qVKvzwww+4uLiQlpbGsmXL8hNF3muoUqUKc+fOZezYsQQEBDBu3DgyMzP59ttvsbe3v6fF4u7uTs+ePfn666/R6XSqGqw8u9+ovqXcjF4VFhsp5fvVpPzzrXvumr4uTDb8358yJSPLuM9ZRqiqMM3x48dl9+7dZZUqVaSzs7McNWqU3LFjhxRCyH79+uWfd+jQIdmrVy9ZpUoV6eLiIvv06SNDQkLuuV5RzktISJDPPvusdHZ2lg4ODrJ3794yNDT0vlVhY8aMeeBr2L59u2zbtq2sVKmSdHd3ly+//LJcvXq1BOTLL79c4NzNmzfLjh07Sjs7O+nh4SGfeuopGRkZWeh1ly9fLgHZoEGDBz6/YjrGqAoTspjVJeYuICBAhoSEGO+Cv46Bs3/DpBNgf3scRa+XdPgkGP/aVflhdMADLmC+Tp8+TZMmTUwdhlKGRUVF4eHhwcyZM5k2bdrDH6CUuqL+HQshjkgpC30zU11hxnTjOISvhcfeKpBUAI5fSyAqKYO3mjYyUXCKYjrr16/n6tWr7Nq1C1tbW8aOHWvqkJQSpBKLMeVt4tXh1Xvu2hwWRQVrQY8mqmZfsTyXLl3iv//9L66urixZsoQ6deqYOiSlBKnEYiwXd8P5bdBrFtgV3GZYSsmmsJt0bOBC1UoVTBSgopjOxIkTmThxoqnDUEqJKjc2Bpm7iZdDTWjzwj13n7qRxNX4dFUNpiiKRVCJxRgi/oRrh3M38ap0z92bwm5ibSV43FclFkVRyj+VWIpLnwPbPoDqDaDFs/fcrXWDRRFY1xnnKrYmCFBRFKV0qcRSXKG/Quxp6P5u/iZedzobk8KF2FTVDaYoisVQiaU4snWw4yPwaA5NBhd6yqaTUQgBvf1UYlEUxTKoqrDiOLIYEq7AgDlgVXiO3hR2kwCvarg6qjWRFEWxDKrF8qgyU2DXZ+DdGep3L/SUi3GpnIlKpk9Tj1IOTlEUxXRUi+VRnVoDqbEwfFX+Jl532xR2E0DtvaIoikVRieVRtXwOXH0LbOJ1t81hUTSvXZVaTveWICuKopRXqivsUQnxwKRy7Z80Qq8lqm4wpVDHjh3jo48+4uDBg0U6Pycnh+zsbKPHMXHiRBYsWIBOpwO0JfEnTJhAUFCQ0Z9LsRyqxVJCNodFAagyYwsTHh5OZmYmdnZ2iDu6SD09PQvsCz937lxWrVpFZGQkgYGBD7xmdnY2Q4YMwdHRkeXLlyOEICkpidOnT2Nra4utrS3W1vduHKfX68nKysLR0ZG6devec/+xY8eYP38+devWZfDgwXh4eCCE4PDhwyxfvpwDBw7g5+dXjJ+GYqlUYikhm8OiaOLhiLdLFVOHopSiiRMnsn//fipVqoQQAp1OR3JyMuHh4flLkZ87d45Vq1axYsUKJkyYwJEjR2jduvV9r2ljY0NgYCDTpk3D2dmZr7/+mmPHjtGjR4/8xGJjY0NaWhrp6elUr14dgKysLDIzMxk/fjzffPNNgWvq9XpeeeUVpJR88cUXCCGIitI+DH333Xf8+uuvODg45B+TUpKVlQVoSVJRHuh+G7VYys3oG31JKaMS06XXlA1y7tbCNzsqr9RGX/datWqVBGRiYmL+sQEDBshu3bpJKaWcP3++9PLyktevX3/otV599VXZtm1bmZKSUuj906dPl9bW1kWK6+OPP5aAwbfAwMAiXV8xX8bY6Eu1WErAllOqG8ySrV+/noYNG9KoUSPi4uKws7PD0dERgG+++YYtW7Zw9OhRACZMmEBwcDAdO3Zk2bJldOrU6b7XnTNnDjqdjkqVilcMEhQUlL9t8KJFi+jTp0/+fevWreOll15i//79+dsa6/V60tPTqVy5Mg4ODsV6bsUyqMH7ErDpZBT1a1TBx039EVqioUOHsmPHDgBiY2Nxc9P24AkKCuL1119n6tSpNG3aFAAhBCtWrKB169Z07tyZYcOGsWfPnkKva21tXeyksnHjRoYPH06zZs0AGDduHB4eHvm3l156CYD27dvnH6tVqxYNGjRgxYoV2NvbF+v5FcugEouR3UrJ5ODFW/RV1WAWKycnB3d3rbUaFxeHu7s7GzZsYMiQIQQEBDB//nyEEPk3Ozs7Nm/ezOeff05QUBBLly7Nv1aHDh2oV68ejRs3pnHjxqxYsSL/voSEBEJCQkhISChybFeuXOGpp57ip59+AuCPP/4o0IWxatUqAM6ePZt/LDs7m9jYWLXro1JkqivMyP4Oj0Yvoa+/6gYrYNPbEHXS1FE8mLs/9P2kWJfQ6/VIKfO7vuLi4nBzc2PAgAEsWLCAoUOHkpKSkl/JJYQgJyeHjIwMvL29efrpp6lR4/a21n369CE1NZXMzEzmzp1Lampq/n0HDx6kT58+bNy4kX79+t0Ti06nIyMjA3t7e6xylxyaMGECEyZMICwsDIAhQ4YU+jp8fHzuOdaxY8f7tqYeJCUlhZSUlAeeU7VqVYNbY5s3b+bDDz/k5MmT2NjY0Lp1az755BNatmxZ4LzDhw8zbdo09u7dS8WKFWnZsiUzZsygQ4cOBp/n7e2Nt7d3fos0T8+ePcnOzi5wXAjBmDFj+PHHH5kzZw5Lly7F3t6effv2FXjsihUr+OKLL4iIiMDe3p6OHTsye/Zs6tWrV+C8v/76iw8++IAjR45QtWpVAgMD+fDDD/Mr94KDg+nRowezZ89m8uTJ+Y+TUuLl5YWzszPHjx836Gf8qFRiMbJNYVF4OlfG18PR1KEoJpA3HyTvTTIuLo769esD8K9//Qsgv2qrMHdXXL333nuAtrXv3LlzqVixYv59dnZ2Bb6C1loSd60EcfHixfzxkrsVZYxFSklGRkZ+cjLU559/zowZMx54zqJFiwxqEQUHBzNgwACaNWvGrFmzyMjIYN68efTp04eIiAicnJzyz+vbty+urq6888472Nra8v3339OlSxe2bt1Kly5dDDrPUHq9nkGDBrF792769++fXxmYZ+nSpYwZM4bu3bsze/Zs4uPj+fLLLxk4cCChoaH5ZeTLli1jzJgxNG7cmJkzZ6LT6Zg3bx7t2rUjJCSERo0a0bVrV2rXrs2qVasKJJb9+/dz9epVXn/99Ud6DY9CJRYjSkzLYu+5OJ7vVPeeP26LV8yWgLnIzMwE4Nq1a5w5c4YbN27g4+PD+fPnqV+/PmfOnKFGjRokJiYyatQofvnlF+rUqUNwcDC7d+9m/PjxD9wP/s75Knnf3/mGb21tnd8ayczMJCMjAw+Pe7tlc3JyAEhNTS3QlZbXIkpKSipwPK9VBeDl5WXQz2TUqFG0a9fugef4+/sbdM2TJ0/St29ffvjhh/xuR09PT4YNG8a+ffvyW3AvvfQSlStX5vDhw/nnDRkyhPr16/Ptt9/mJ4yinmeo1atXU79+fc6fP4+rq+s99585c4ahQ4eycuXK/A8NFSpUYMqUKURERODr60tqaiqvvfYa9erVIyQkJH8+VLt27ejRoweLFi3ik08+wcrKimeffZZPP/2UyMhIGjZsCMCvv/6KjY0NI0eOfKTX8EjuVy5mKTdjlhv/HnJVek3ZII9ejjfaNc2JKjeW8vr164WW6TZu3Fjm5ORIV1dXOW/ePJmcnCytrKzk5s2bpZRaKbGjo6NMSkoq9LoXL16UgFy2bFn+sd27d0tAbt++XUppWLnxoUOHJCAdHBxk9erVH3qrVq2atLOzk2+++WbxfkBGFh0dLTds2CCnT58u69WrJwG5aNEiKaWUkZGREpAvvPDCA69R1POklNLLy0t26dLlnuM9evS45zggbWxsZERExEOve+XKFblmd9qkzgAAEzpJREFUzRr51ltvSXd39wL/r3/99ZcE5KxZsx56nbCwMAnI999/X0oppV6vl7Vq1ZL9+/d/6GPzGKPcWA3eG9GmsCg8qtrRvLaTqUNRTKRmzZr3/JGNHTuW6tWrY2VlxaBBg9iyZQv29vbUr1+f06dPA7Bnzx6GDBlSpHLey5cvGyXOL7/8kmPHjhEXF5d/69y5M+3bty9wLC4ujvj4eNLT0/nss8+K/dzGEB4eTufOnXFzc2PYsGFs2bKFtm3bFjgnJiYGgFq1aj3wWkU970GSkpIKPT548OD8lkNh9u7dS7NmzfD09GTcuHEcPHiQgICCS0UZEp+fnx8tWrTIL8LYs2cP169fZ/To0UV9KUahEouRpGRms+tsLL393LGyUt1gli4nJ4eLFy8SGRlJcnJyfp9/v3792L59O1lZWfj6+nLmzBkSExM5efLkfQfS73To0CFatGjBxYsXHymuhIQEjh07RmJiIn369CErK4szZ87k327evMnNmzcLHMu7hYaGcvjwYW7duvVIz21MQ4YM4fz58+zbt4+kpCT279/Piy++WOCcvK6n69ev3/P4adOm5Z9f1PPuR6fTERERUeh9DyrPTklJYdCgQej1ekJDQ0lISGDHjh08+eSTRX4dEyZMyJ+TlOe5554jIiKCY8eO8euvv+Lk5MSgQYMe+BqMTSUWI9l+JgZdtp5+/qrM2FLFx8fTsWNH6tatS6VKlahXrx5LliwhOTmZqlWrAvDYY4+RkpLCnj176NChAx4eHmzatAlbW1t69er10OeYN28e9vb2ha4NVhQHDx4kMDCQwMBAOnXqlH/z9/fH19eXo0ePcuLECZo2bYqvr2+Bczp16kTHjh3vqYgqbbdu3SIyMpLu3bvTvn17rKys0Ov1LFy4sMB5Pj4++Pj4sGbNmvxP/aAVVMydO5fQ0FCDzgMtUdy4caPA83z11Vf3bbE8SEREBPHx8TzxxBP5Y0yZmZksW7aswHkdOnTAycmJxYsX549z5T1+wYIFXLhwocD5I0eOxNra+v/bu/voqqozj+PfXyNBCCl6g0iUARxYCHag1qZqEVEsBXTNOINj6VDtAE6nIyiWKZXxvfWly5fqmlGc+sJYUQTBloIyLuNbZSG6qEIrFgZFsdEUeQkQRAkvQZ75Y58LN+Em5CaHHMx9PmudddbdZ99zn5wF97ln7332ZtasWcybN4/Ro0fXGeDRGrzzPiblKzfQpVN7vt7z2KRDcQlJpVKcdNJJnHPOOQwZMoSBAwdSWlrKoEGD9o8MKykpYerUqaRSKaZOnQqEL4Lhw4c3Otw23SF/7LHHMn/+fLp06UJFRUWj8ZiF+b127dq1f/jziBEj9o9cS5s+fToTJkxg2rRpzJ8/n7179zJ58mRGjx7NmDFjuOOOOygqOnLmvEulUnTv3p0FCxZw++23U1hYyNy5c1mxYgUQZmhOe/DBBzn//PMpKytjwoQJFBYW8sgjj7Bz505uu+22nOsNHTqU+++/n3vuuYcxY8bwzDPPcOeddzY46q4xvXv3pqioiOnTp9OlSxdqamqYOXPm/kSR/juKioq49957GTduHGVlZYwfP57du3fzwAMP0KlTp4PuWLp168awYcOYNm0ae/bsYezYsTnH1mINdb7kyxZH533N7r3W74bn7Lrfvt3ic32Reef9wfbt22c9e/a0a665xiorK23NmjVWWVm5f/vwww+tc+fOdtddd9lHH31k77zzTtZzDBgwwAoKCuyll16yK664Iqf5vbJ16NfW1lp5ebmde+651q5dO5s2bZqZ1e2EXrRokZWWllpJSYlNmTLFlixZYrW1tYf1ejXVW2+9Zeedd54VFRVZKpWySy+91BYtWmSS7IILLqhT94033rDhw4dbUVGRdenSxUaOHGnLli076JxNqbdt2za75JJLLJVKWXFxsY0YMcLefvvtBjvvx44d2+jf8corr9jpp59uHTp0sG7dutnEiRNt3rx5BtjEiRPr1C0vL7ezzjrLjj76aCstLbWLL77Y1qzJPh/hE088YYD16dOn0c/PJo7O+8S/2JPe4kgsz/1pvfX8j/+1V9dUtfhcX2SeWA6YPXu2DRkyxE455RQD7KGHHrKxY8c2KwmYhS+ge++918zMNm3aZJWVlbZhwwarqqpqcNu4caOtW7fO1q5du/88K1eutFGjRlkqlTLAhg0bZm+++eb+44MHD7bTTjtt/+vq6mq79tpr7ZhjjjHAOnXqZPPmzTtMV83FZf369QbYLbfckvN740gsCsfzV1lZmS1btqxF55g8548sWlPFm9cPo11B/nZbrV69+qAHwPLV2rVrufDCC+nfvz9nnnkmV111FTU1NUCYBj+bffv2sWPHjqzPncTFzBg1ahQlJSVMmjSJU089tc7xsrIytmzZctDggJqaGubOnUt5eTmzZs1q8G9wyVq4cCGVlZUsXryY+fPn8/777zf6XFQ2Tf1/LGm5mWVd7dD/dbTQ7r2f8/LqTZw/oFteJxVXV+/evVm1alWdssLCwkO+L90XcrhIYsGCBQ0eb+hHVseOHRk/fjzjx48/XKG5GFRUVDBlyhS6du3KY489lnNSiYsnlhZ6/f0tfLp7r0866ZxL3KRJk5g0aVLSYSQ33FhSf0nPSvpE0jZJCyT1asZ5FkmyBrYZsQdez3Mr11Pc/igG9Wl4/ifnnMsnidyxSDoReDV6eQdQAFwNLJY00MyaPg94eP+MemXdgVuBmhaG2qjaz/fxwv9t5Fv9u9L+qOY9V+Ccc21NUk1hPwVKgKFmtghA0nvAHOAq4JamnsjMyuuXSUpP4/lCiyNtxO8/2Mq2mlpGejPYfmbmE3A69wUV12CuVm8Kk9QO+C7wbjqpROYBnwCXxPAxFxHuVp6P4VwNem7lejq0K+CcvscdunIeKCgooLa2NukwnHPNVFtb2+xZHTIl0ccyEPgy8EZmoZntBVYAfSWlmntySccDg4DnzWznoeo31+f7jOdXbeS8fl3pUOjNYADFxcXNmtrCOXdk2L59e5MmQj2UJBJLr2j/cZZjG+rVaY5RhL9rfkMVJP1Q0jJJy6qqqpr1Ics/rGbzZ7sZ+Te+UmRaKpWiurqazZs3s2fPnthuq51zh4+ZsWfPHjZv3kx1dTWpVLN/1++XRB9Lx2ifrc1kT706zXFRdO6FDVUws4eBhyE8INmcD/mSYEjf4xja7+DFe/JV+/bt6dGjB1u3bqWiomL/YlLOuSNbQUEBxcXF9OjRo84qpc2VRGJJL9rdOcuxzvXq5ETSscC5wKIcR5blrKxXiscvO/3QFfNM+/btKS0tPaxPjzvnjmxJNIWl53jO9lM/3Qte0cxz/x3QjkaawZxzzh1eSSSWlYTRX2dnFkrqAHwNWG1m1c089z8SJvNreM4K55xzh1WrJ5Zo9NeTwAmSvp1xaBTQHpjdnPNKKgKGA0vNbH2LA3XOOdcsST0geSvwHWCOpLsJT95PBSqBaQCS+gCDgSVm9n4TznkBcDTeDOacc4lKZK4wM/uYkDSWAtcSpnP5HTDEzD6Jqg0GHo32TXFRtPfE4pxzCfL1WGJYj8U55/JNY+ux+AIizjnnYuWJxTnnXKzyvilMUhXwYTPf3gXYHGM4X3R+Pery63GAX4u62sL16GlmWWfgzfvE0hKSljXUxpiP/HrU5dfjAL8WdbX16+FNYc4552LlicU551ysPLG0zMNJB3CE8etRl1+PA/xa1NWmr4f3sTjnnIuV37E455yLlScW55xzsfLE0gyS+kt6VtInkrZJWiCpV9JxJUVSv+h6fCpph6QXJA1IOq6kSbpakkmakXQsSZF0paR3JdVI+oOkkUnHlCRJPSU9JWmTpI8lPSqpzS1D630sOZJ0IrAienkPYWbmqwlrzAw83CtXHmkklQCrgCLgFxyYqfoToK+ZbU8wvMRIOhl4izDj9mNmNi7ZiFqfpBuBmwkd1SuBy4H+wFlmtjTJ2JIgqTvwR2AXcB9hmZDJwKfAqRkT8H7hJTVt/hfZT4ESYKiZLQKQ9B4wB7gKuCW50BLxL8DxwPfM7EkASbWEpRG+D/x3grElQtKXCDNz7yEklrwTfYneANxhZtdFZS8A7wITCDOb55tJhCfuzzazJQCS3gF+DYwH/ivB2GLlTWE5kNQO+C7wbjqpROYRfqFfkkRcCTsj2j+fUbY82vdr5ViOFP8OfBP4SdKBJOifgELgoXSBma0hLD8+KamgEpb+/7A8o6xN/l/xxJKbgcCXgTcyC6NVMVcAfSWlkggsQb8FbgQyl5M+Idpvbf1wkiWpL+Fu7X+AFxMOJ0lnEZp4iiUtlbQr+nV+br42jwIbon23jLIT6h1rEzyx5KZXtP84y7EN9erkBTObZWa3WdRZJ+koQlOHAU8nGlwri5rAfkWYXHBKwuEk7SSgFniG8EPseqAj8JSkMxp7Yxv2S2AH8KCkgZK+AfwnUAU8kmhkMfM+ltx0jPa1WY7tqVcn70RfrA8AXwfuMrM/JBxSa/sR4Zf6SDPbnod3r5k6ASngdjO7G0DSUmAJIclcmGBsiTCzFZK+Q1jlNj0A6FNgmJlVJhdZ/PyOJTc7on3nLMc616uTVyS1B2YDPyC0q1+TbEStS1If4OeEpsEVkroR+hMAOkjqJqkwsQBbX/rH15PpAjN7DagBvpZIRAmT9A+EpPI8MAb4Z0KCeVHSN5OMLW6eWHLzQbTPNu48/SVS0TqhHDkkdQTKCQMbbjWzyy3/xrEPBjoAFwHroy3dFzc6ej0omdASsSXa76pXvpUwMiqvSCog9Lu9B4wyszlmNhMYQWjtaFOjJ70pLDcrCaO/zs4slNSB8CtstZlVZ3tjWxU1f80DzgEmmtkDCYeUlBeB8+uVHQ/MAF4iPPP0divHlKRVhGbBXhxIMgDHAH9JIqCEdSU8pvCKme1LF5pZjaQKoE09UOyJJQdmtlfSk8Dlkr5tZulRP6MIDzvNTi66xFwFjARuyuOkgpmtA9ZllmXMxrDOzMpbO6aELQR+SHg+YzmApDMJfS+/TzCupFQR+lPOkNTRzGoAJJUSHhr9c5LBxc2fvM+RpBMIvzwF3M2BJ823AQPa0tOzhxLdqX1E+IHyE+DzelU+M7PftHpgR4gosfyZPHzyXpKAl4GhhBFPq4EfE365n2Zmf0owvERIuhm4ifD0/UygHSH59gbGm9mM5KKLlyeWZpDUj9C0cTZhWO0rwGQzq0gyrtaW8cXZkA/NrFerBHMEyufEAiCpGLid0MfUmdCUfH0e3r0B+5PtpYQHRPsCewlNhneZ2bNJxhY3TyzOOedi5aPCnHPOxcoTi3POuVh5YnHOORcrTyzOOedi5YnFOedcrDyxOOeci5UnFueaSFJFtIZ9tm1OwrGZpBlJxuBcmk/p4lxutpB9Zcg2NSWHcy3hicW53HzWlqbecO5w8KYw55xzsfLE4lxMJPWK+jrulPSwpGpJn0r6jaQeWeqfI2mRpBpJWyTNlNQ9S73hkl6N6q2XtEDSVxqI4WRJ5dHn/kXSTVnqXCJpeVSnStJCSafEcxWc87nCnGuyaN2MAuAbWQ5vBroT+lp2ERZ8mw6cSJh08GPgq+nZryVdDMwB1hIWgEpF9bYDZ5rZR1G97wOPAe8AvwIKgSuBYqDMzN6N6qUnQ/0K8CxhBt0JhCnZR5nZgqjeKMIqly9H+w7AvxGms++Tns7duRYxM998860JGyFZWAPbqYRFrYywSmIq431XROU/jl53IKzPUQF0yqg3CNgH/Dp6XQRUA+8DHTPqnRed746MsnQcN2SUfTUqm5ZRNi0q65xRVgY8CPRI+hr71jY277x3LjebgLFZytcSVggEeNrMtmYcmwvcD6TXNT+LsDzvfWb2WbqSmb0uaSnwt9FStoMIKy7ebRl3Emb2O8J6QPVtAH6R8Tq95klxRtmrhDue+yQ9BrxlZsuAZQ3/yc7lxhOLc7nZaQ2sJyIpnVg21Du0hXAnckz0+vhon22J3kpCAupMWBQL6q1M2YjlZrY7/cLM9oUlQA4ws6ckdQF+AJQD7SStJKwJMrOJn+Nco7zz3rn41e+A70r4v5a+i9nYQD2AvyL00XxCuDuC0E9Th6QHJP28XvHmpgRnZr80s9MITW1lwAfA45IGN+X9zh2KJxbn4vf3ko7LeP29aL8k2r9GuIu5TFJRulK0JvwZwLNm9jnwOmHJ63GSjs6odzJwOfDXuQYm6SVJiwHMrNbMlgM/iw6fnuv5nMvGm8Kcy00nSeMaOJZOHAYskfQQ4Q7kSsJosRkAZrZT0kTgSWC5pMxRYVVET/ab2Q5JP4ret0zSo0B7wmivz4D6dyxNsRi4WdLTwAuEUW6XAZ9Hx5xrMU8szuWmBHi0gWMDov39hGamG4B2wNPAZDP7NF0x6uvYTLhbuBXYCTwDXGvRUOOo3uOSNgI3ArcRRom9BlxnZu81I/5bCU1s/0pITF8idPJfGHXiO9di/hyLczGR1ItwZ3Kzmf0s0WCcS5D3sTjnnIuVJxbnnHOx8qYw55xzsfI7Fuecc7HyxOKccy5Wnlicc87FyhOLc865WHlicc45F6v/B5AzHK0TQ+o6AAAAAElFTkSuQmCC\n", 208 | "text/plain": [ 209 | "
" 210 | ] 211 | }, 212 | "metadata": { 213 | "needs_background": "light" 214 | }, 215 | "output_type": "display_data" 216 | } 217 | ], 218 | "source": [ 219 | "plot_graphs(history, 'accuracy')" 220 | ] 221 | } 222 | ], 223 | "metadata": { 224 | "kernelspec": { 225 | "display_name": "Python 3", 226 | "language": "python", 227 | "name": "python3" 228 | }, 229 | "language_info": { 230 | "codemirror_mode": { 231 | "name": "ipython", 232 | "version": 3 233 | }, 234 | "file_extension": ".py", 235 | "mimetype": "text/x-python", 236 | "name": "python", 237 | "nbconvert_exporter": "python", 238 | "pygments_lexer": "ipython3", 239 | "version": "3.7.0" 240 | } 241 | }, 242 | "nbformat": 4, 243 | "nbformat_minor": 2 244 | } 245 | -------------------------------------------------------------------------------- /code/save_restore_model.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "## 准备数据" 8 | ] 9 | }, 10 | { 11 | "cell_type": "code", 12 | "execution_count": 1, 13 | "metadata": {}, 14 | "outputs": [], 15 | "source": [ 16 | "import warnings\n", 17 | "warnings.simplefilter(\"ignore\")" 18 | ] 19 | }, 20 | { 21 | "cell_type": "code", 22 | "execution_count": 2, 23 | "metadata": {}, 24 | "outputs": [], 25 | "source": [ 26 | "import tensorflow as tf\n", 27 | "from tensorflow import keras\n", 28 | "from tensorflow.keras import datasets, layers, models, callbacks\n", 29 | "from tensorflow.keras.datasets import mnist\n", 30 | "\n", 31 | "import os\n", 32 | "file_path = os.path.abspath('./mnist.npz')\n", 33 | "\n", 34 | "(train_x, train_y), (test_x, test_y) = datasets.mnist.load_data(path=file_path)\n", 35 | "train_y, test_y = train_y[:1000], test_y[:1000]\n", 36 | "train_x = train_x[:1000].reshape(-1, 28 * 28) / 255.0\n", 37 | "test_x = test_x[:1000].reshape(-1, 28 * 28) / 255.0" 38 | ] 39 | }, 40 | { 41 | "cell_type": "markdown", 42 | "metadata": {}, 43 | "source": [ 44 | "## 搭建模型" 45 | ] 46 | }, 47 | { 48 | "cell_type": "code", 49 | "execution_count": 3, 50 | "metadata": {}, 51 | "outputs": [], 52 | "source": [ 53 | "def create_model():\n", 54 | " model = models.Sequential([\n", 55 | " layers.Dense(512, activation='relu', input_shape=(784,)),\n", 56 | " layers.Dropout(0.2),\n", 57 | " layers.Dense(10, activation='softmax')\n", 58 | " ])\n", 59 | "\n", 60 | " model.compile(optimizer='adam', metrics=['accuracy'],\n", 61 | " loss='sparse_categorical_crossentropy')\n", 62 | "\n", 63 | " return model\n", 64 | "\n", 65 | "def evaluate(target_model):\n", 66 | " _, acc = target_model.evaluate(test_x, test_y)\n", 67 | " print(\"Restore model, accuracy: {:5.2f}%\".format(100*acc))" 68 | ] 69 | }, 70 | { 71 | "cell_type": "markdown", 72 | "metadata": {}, 73 | "source": [ 74 | "## 自动保存 checkpoints" 75 | ] 76 | }, 77 | { 78 | "cell_type": "code", 79 | "execution_count": 4, 80 | "metadata": {}, 81 | "outputs": [ 82 | { 83 | "name": "stderr", 84 | "output_type": "stream", 85 | "text": [ 86 | "WARNING: Logging before flag parsing goes to stderr.\n", 87 | "W0713 00:06:20.997914 140735530943360 callbacks.py:859] `period` argument is deprecated. Please use `save_freq` to specify the frequency in number of samples seen.\n", 88 | "W0713 00:06:21.173481 140735530943360 deprecation.py:323] From /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/tensorflow/python/ops/math_grad.py:1250: add_dispatch_support..wrapper (from tensorflow.python.ops.array_ops) is deprecated and will be removed in a future version.\n", 89 | "Instructions for updating:\n", 90 | "Use tf.where in 2.0, which has the same broadcast rule as np.where\n" 91 | ] 92 | }, 93 | { 94 | "name": "stdout", 95 | "output_type": "stream", 96 | "text": [ 97 | "\n", 98 | "Epoch 00010: saving model to training_2/cp-0010.ckpt\n", 99 | "\n", 100 | "Epoch 00020: saving model to training_2/cp-0020.ckpt\n", 101 | "\n", 102 | "Epoch 00030: saving model to training_2/cp-0030.ckpt\n", 103 | "\n", 104 | "Epoch 00040: saving model to training_2/cp-0040.ckpt\n", 105 | "\n", 106 | "Epoch 00050: saving model to training_2/cp-0050.ckpt\n" 107 | ] 108 | }, 109 | { 110 | "data": { 111 | "text/plain": [ 112 | "" 113 | ] 114 | }, 115 | "execution_count": 4, 116 | "metadata": {}, 117 | "output_type": "execute_result" 118 | } 119 | ], 120 | "source": [ 121 | "# 存储模型的文件名，语法与 str.format 一致\n", 122 | "# period=10：每 10 epochs 保存一次\n", 123 | "checkpoint_path = \"training_2/cp-{epoch:04d}.ckpt\"\n", 124 | "checkpoint_dir = os.path.dirname(checkpoint_path)\n", 125 | "cp_callback = callbacks.ModelCheckpoint(\n", 126 | " checkpoint_path, verbose=1, save_weights_only=True, period=10)\n", 127 | "\n", 128 | "model = create_model()\n", 129 | "model.save_weights(checkpoint_path.format(epoch=0))\n", 130 | "model.fit(train_x, train_y, epochs=50, callbacks=[cp_callback],\n", 131 | " validation_data=(test_x, test_y), verbose=0)" 132 | ] 133 | }, 134 | { 135 | "cell_type": "code", 136 | "execution_count": 5, 137 | "metadata": {}, 138 | "outputs": [ 139 | { 140 | "name": "stdout", 141 | "output_type": "stream", 142 | "text": [ 143 | "1000/1000 [==============================] - 0s 94us/sample - loss: 0.5232 - accuracy: 0.8720\n", 144 | "Restore model, accuracy: 87.20%\n" 145 | ] 146 | } 147 | ], 148 | "source": [ 149 | "latest = tf.train.latest_checkpoint(checkpoint_dir)\n", 150 | "# 'training_2/cp-0050.ckpt'\n", 151 | "model = create_model()\n", 152 | "model.load_weights(latest)\n", 153 | "evaluate(model)" 154 | ] 155 | }, 156 | { 157 | "cell_type": "markdown", 158 | "metadata": {}, 159 | "source": [ 160 | "## 手动保存权重" 161 | ] 162 | }, 163 | { 164 | "cell_type": "code", 165 | "execution_count": 6, 166 | "metadata": {}, 167 | "outputs": [ 168 | { 169 | "name": "stdout", 170 | "output_type": "stream", 171 | "text": [ 172 | "1000/1000 [==============================] - 0s 90us/sample - loss: 0.5232 - accuracy: 0.8720\n", 173 | "Restore model, accuracy: 87.20%\n" 174 | ] 175 | } 176 | ], 177 | "source": [ 178 | "# 手动保存权重\n", 179 | "model.save_weights('./checkpoints/mannul_checkpoint')\n", 180 | "model = create_model()\n", 181 | "model.load_weights('./checkpoints/mannul_checkpoint')\n", 182 | "evaluate(model)" 183 | ] 184 | }, 185 | { 186 | "cell_type": "markdown", 187 | "metadata": {}, 188 | "source": [ 189 | "## 保存整个模型" 190 | ] 191 | }, 192 | { 193 | "cell_type": "markdown", 194 | "metadata": {}, 195 | "source": [ 196 | "### HDF5" 197 | ] 198 | }, 199 | { 200 | "cell_type": "code", 201 | "execution_count": 7, 202 | "metadata": {}, 203 | "outputs": [], 204 | "source": [ 205 | "model.save('my_model.h5')" 206 | ] 207 | }, 208 | { 209 | "cell_type": "code", 210 | "execution_count": 8, 211 | "metadata": {}, 212 | "outputs": [ 213 | { 214 | "name": "stderr", 215 | "output_type": "stream", 216 | "text": [ 217 | "W0713 00:06:28.440529 140735530943360 hdf5_format.py:192] Error in loading the saved optimizer state. As a result, your model is starting with a freshly initialized optimizer.\n" 218 | ] 219 | }, 220 | { 221 | "name": "stdout", 222 | "output_type": "stream", 223 | "text": [ 224 | "1000/1000 [==============================] - 0s 91us/sample - loss: 0.5232 - accuracy: 0.8720\n", 225 | "Restore model, accuracy: 87.20%\n" 226 | ] 227 | } 228 | ], 229 | "source": [ 230 | "new_model = models.load_model('my_model.h5')\n", 231 | "evaluate(new_model)" 232 | ] 233 | }, 234 | { 235 | "cell_type": "markdown", 236 | "metadata": {}, 237 | "source": [ 238 | "### saved_model" 239 | ] 240 | }, 241 | { 242 | "cell_type": "code", 243 | "execution_count": 9, 244 | "metadata": {}, 245 | "outputs": [ 246 | { 247 | "name": "stderr", 248 | "output_type": "stream", 249 | "text": [ 250 | "W0713 00:06:29.094913 140735530943360 deprecation.py:323] From /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/tensorflow/python/saved_model/signature_def_utils_impl.py:253: build_tensor_info (from tensorflow.python.saved_model.utils_impl) is deprecated and will be removed in a future version.\n", 251 | "Instructions for updating:\n", 252 | "This function will only be available through the v1 compatibility library as tf.compat.v1.saved_model.utils.build_tensor_info or tf.compat.v1.saved_model.build_tensor_info.\n", 253 | "W0713 00:06:29.096133 140735530943360 export_utils.py:182] Export includes no default signature!\n", 254 | "W0713 00:06:29.277875 140735530943360 util.py:244] Unresolved object in checkpoint: (root).optimizer.iter\n", 255 | "W0713 00:06:29.278687 140735530943360 util.py:244] Unresolved object in checkpoint: (root).optimizer.beta_1\n", 256 | "W0713 00:06:29.279356 140735530943360 util.py:244] Unresolved object in checkpoint: (root).optimizer.beta_2\n", 257 | "W0713 00:06:29.280627 140735530943360 util.py:244] Unresolved object in checkpoint: (root).optimizer.decay\n", 258 | "W0713 00:06:29.281178 140735530943360 util.py:244] Unresolved object in checkpoint: (root).optimizer.learning_rate\n", 259 | "W0713 00:06:29.281950 140735530943360 util.py:252] A checkpoint was restored (e.g. tf.train.Checkpoint.restore or tf.keras.Model.load_weights) but not all checkpointed values were used. See above for specific issues. Use expect_partial() on the load status object, e.g. tf.train.Checkpoint.restore(...).expect_partial(), to silence these warnings, or use assert_consumed() to make the check explicit. See https://www.tensorflow.org/alpha/guide/checkpoints#loading_mechanics for details.\n", 260 | "W0713 00:06:29.402854 140735530943360 export_utils.py:182] Export includes no default signature!\n" 261 | ] 262 | }, 263 | { 264 | "data": { 265 | "text/plain": [ 266 | "(1000, 10)" 267 | ] 268 | }, 269 | "execution_count": 9, 270 | "metadata": {}, 271 | "output_type": "execute_result" 272 | } 273 | ], 274 | "source": [ 275 | "import time\n", 276 | "saved_model_path = \"./saved_models/{}\".format(int(time.time()))\n", 277 | "tf.keras.experimental.export_saved_model(model, saved_model_path)\n", 278 | "new_model = tf.keras.experimental.load_from_saved_model(saved_model_path)\n", 279 | "model.predict(test_x).shape" 280 | ] 281 | }, 282 | { 283 | "cell_type": "code", 284 | "execution_count": 10, 285 | "metadata": {}, 286 | "outputs": [ 287 | { 288 | "name": "stdout", 289 | "output_type": "stream", 290 | "text": [ 291 | "1000/1000 [==============================] - 0s 95us/sample - loss: 0.5232 - accuracy: 0.8720\n", 292 | "Restore model, accuracy: 87.20%\n" 293 | ] 294 | } 295 | ], 296 | "source": [ 297 | "new_model.compile(optimizer=model.optimizer,\n", 298 | " loss='sparse_categorical_crossentropy',\n", 299 | " metrics=['accuracy'])\n", 300 | "evaluate(new_model)" 301 | ] 302 | } 303 | ], 304 | "metadata": { 305 | "kernelspec": { 306 | "display_name": "Python 3", 307 | "language": "python", 308 | "name": "python3" 309 | }, 310 | "language_info": { 311 | "codemirror_mode": { 312 | "name": "ipython", 313 | "version": 3 314 | }, 315 | "file_extension": ".py", 316 | "mimetype": "text/x-python", 317 | "name": "python", 318 | "nbconvert_exporter": "python", 319 | "pygments_lexer": "ipython3", 320 | "version": "3.7.0" 321 | } 322 | }, 323 | "nbformat": 4, 324 | "nbformat_minor": 2 325 | } 326 | -------------------------------------------------------------------------------- /tf2doc.md: -------------------------------------------------------------------------------- 1 | --- 2 | title: TensorFlow 2 / 2.0 中文文档 3 | date: 2019-07-09 00:10:10 4 | description: TensorFlow2文档，TensorFlow2.0文档，TensorFlow2.0 TF2.0 TensorFlow 2 / 2.0 官方文档中文版，有删改，序言，介绍整个文档的构成。 5 | tags: 6 | - TensorFlow 2 7 | - 官方文档 8 | keywords: 9 | - TensorFlow2文档 10 | - TensorFlow2.0文档 11 | - TensorFlow2.0 12 | - TF2.0 13 | nav: 简明教程 14 | categories: 15 | - TensorFlow2 文档 16 | top: 3 17 | image: post/tf2doc/tf.jpg 18 | github: https://github.com/geektutu/tensorflow2-docs-zh 19 | --- 20 | 21 | ![TensorFlow 2.0](tf2doc/tf.jpg) 22 | 23 | 如果你对 Python 还不熟悉，推荐先阅读 [一篇文章入门 Python](https://geektutu.com/post/quick-python.html) 24 | 25 | ## 文档地址 26 | 27 | - 文档地址：[TensorFlow 2 / 2.0 中文文档](https://geektutu.com/post/tf2doc.html) 28 | - Github：[Github - tensorflow2-docs](https://github.com/geektutu/tensorflow2-docs-zh) 29 | - 知乎专栏：[Zhihu - Tensorflow2-docs](https://zhuanlan.zhihu.com/geektutu) 30 | 31 | ## 目录(持续更新) 32 | 33 | ### 基础 - 机器学习基础 ML basics 34 | 35 | 1. [图像分类 Classify images](https://geektutu.com/post/tf2doc-ml-basic-image.html) 36 | 2. [文本分类 Classify text](https://geektutu.com/post/tf2doc-ml-basic-text.html) 37 | 3. [结构化数据分类 Classify structured data](https://geektutu.com/post/tf2doc-ml-basic-structured-data.html) 38 | 4. [回归 Regression](https://geektutu.com/post/tf2doc-ml-basic-regression.html) 39 | 5. [过拟合与欠拟合 Overfitting and underfitting](https://geektutu.com/post/tf2doc-ml-basic-overfit.html) 40 | 6. [保存和恢复模型 Save and restore models](https://geektutu.com/post/tf2doc-ml-basic-save-model.html) 41 | 42 | ### 基础 - 图像分类 43 | 44 | 1. [卷积神经网络 Convolutional Neural Networks](https://geektutu.com/post/tf2doc-cnn-cifar10.html) 45 | 2. [使用TFHub进行迁移学习 TensorFlow Hub with Keras](https://geektutu.com/post/tf2doc-tfhub-image-tl.html) 46 | 3. 使用预训练CNN进行迁移学习 Transfer Learning Using Pretrained ConvNets 47 | 48 | ### 基础 - 文本分类 49 | 50 | 1. [使用RNN对文本分类进行分类 Text classification with an RNN](https://geektutu.com/post/tf2doc-rnn-lstm-text.html) 51 | 52 | ### 进阶 - 自定义 53 | 54 | 1. 张量和操作 Tensors and operations 55 | 2. 自定义层 Custom layers 56 | 3. 自动微分 Automatic differentiation 57 | 4. 自定义训练：攻略 Custom training：walkthrough 58 | 5. 动态图机制 TF function and AutoGraph 59 | 60 | ## 极客兔兔实战 61 | 62 | ### 监督学习 63 | 64 | 1. [mnist手写数字识别(CNN卷积神经网络)](https://geektutu.com/post/tensorflow2-mnist-cnn.html) 65 | 2. [监督学习玩转 OpenAI gym game](https://geektutu.com/post/tensorflow2-gym-nn.html) 66 | 67 | ### 强化学习 68 | 69 | 1. [强化学习 Q-Learning 玩转 OpenAI gym](https://geektutu.com/post/tensorflow2-gym-q-learning.html) 70 | 2. [强化学习 DQN 玩转 gym Mountain Car](https://geektutu.com/post/tensorflow2-gym-dqn.html) 71 | 3. [强化学习 70行代码实战 Policy Gradient](https://geektutu.com/post/tensorflow2-gym-pg.html) 72 | 73 | ## 声明 74 | 75 | **TensorFlow 2 中文文档**主要参考 [TensorFlow官网](https://www.tensorflow.org/beta/tutorials/keras)，书写而成。选取了一些有价值的章节作总结，内容目录基本与官方文档一致，但在内容上作了大量的简化，以代码实践为主。TensorFlow 是机器学习的高阶框架，功能强大，接口很多，TensorFlow 2 废弃了大量重复的接口，将 Keras 作为搭建网络的主力接口，也添加了很多新的特性，极大地改进了可用性，能有效地减少代码量。 76 | 77 | **TensorFlow 2 中文文档**的目的是选取官方文档中有代表性的内容，帮助大家快速入门，一览TensorFlow 在图像识别、文本分类、结构化数据等方面的风采。介绍 TensorFlow 1.x 的文档已经很多，所以这份文档侧重于总结 TensorFlow 2 的新特性。 78 | 79 | TensorFlow官网的文档遵循[署名 4.0 国际 (CC BY 4.0)](https://creativecommons.org/licenses/by/4.0/deed.zh)协议，代码遵循[Apache 2.0 协议](https://www.apache.org/licenses/LICENSE-2.0)，本文档完全遵守上述协议。将在显著地方注明来源。 80 | 81 | 代码基于**Python3**和**TensorFlow 2.0 beta**实现。 82 | 83 | 力求简洁，部分代码删改过，例如兼容Python 2.x的代码均被删除。 -------------------------------------------------------------------------------- /tf2doc/tf.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/geektutu/tensorflow2-docs-zh/3418ce1c0e721f549d0cff7eeff65ca030c55f82/tf2doc/tf.jpg --------------------------------------------------------------------------------