├── .DS_Store ├── .gitignore ├── 1.Introduction ├── Codlab-exercise │ ├── .DS_Store │ ├── Course_1_Part_6_Lesson_2_Improving_Computer_Vision_Accuracy.ipynb │ ├── Course_1_Part_6_Lesson_3_different_weightingNotebook.ipynb │ ├── Course_1_Part_8_Lesson_2_imageGenerator.ipynb │ ├── Course_1_Part_8_Lesson_3_imageGenerator_validating.ipynb │ └── Fahion-Minist.ipynb ├── Excercise-3-Question-mproving Computer Vision Accuracy using Convolutions .ipynb ├── Exercise4-Question-ImageDataGenerator.ipynb ├── Exercise_1_House_Prices_Question.ipynb └── Exercise_2-Question-Fashion MNIST.ipynb ├── 2.Convolutional Neural Networks in Tensorflow ├── .DS_Store ├── Exercise_1_Cats_vs_Dogs_Question-FINAL.ipynb ├── Exercise_2_Cats_vs_Dogs_using_augmentation_Question-FINAL.ipynb ├── Exercise_3_Horses_vs_humans_using_Transfer_Learning_Question-FINAL.ipynb ├── Exercise_4_Multi_class_classifier_Question-FINAL.ipynb └── codlab │ ├── Course_2_Part_2_Lesson_2_cat_vs_dog.ipynb │ ├── Course_2_Part_4_Lesson_2_Notebook_(Cats_v_Dogs_Augmentation).ipynb │ ├── Course_2_Part_6_Lesson_3_Notebook.ipynb │ └── Course_2_Part_8_Lesson_2_Notebook_(RockPaperScissors).ipynb ├── 3.Natural Language Processing in TensorFlow ├── .DS_Store ├── week1 │ ├── Course_3_Week_1_Exercise_question_BBC News.ipynb │ └── bbc-text.csv ├── week2 │ ├── Course_3_Week_2_Exercise.ipynb │ ├── Course_3_Week_2_Lesson_1.ipynb │ ├── Course_3_Week_2_Lesson_2.ipynb │ ├── Course_3_Week_2_Lesson_3.ipynb │ ├── meta.tsv │ └── vecs.tsv ├── week3 │ ├── Course_3_Week_3_Lesson_1a.ipynb │ ├── Course_3_Week_3_Lesson_1b.ipynb │ ├── Course_3_Week_3_Lesson_1c.ipynb │ ├── Course_3_Week_3_Lesson_2.ipynb │ ├── Course_3_Week_3_Lesson_2c.ipynb │ ├── Course_3_Week_3_Lesson_2d.ipynb │ ├── NLP_Course_Week_3_Exercise.ipynb │ └── NLP_Course_Week_3_Exercise_Question.ipynb └── week4 │ ├── Course_3_Week_4_Lesson_1_Notebook.ipynb │ ├── Course_3_Week_4_Lesson_2_Notebook.ipynb │ └── NLP_Week4_Exercise_Shakespeare_Answer.ipynb ├── 4.Sequence, Time Series and Prediction ├── .DS_Store ├── week1 │ ├── S+P_Week_1_Lesson_2.ipynb │ ├── S+P_Week_1_Lesson_3_Notebook.ipynb │ ├── Week_1_Exercise_Answer.ipynb │ └── Week_1_Exercise_Question.ipynb ├── week2 │ ├── S+P_Week_2_Exercise_Answer.ipynb │ ├── S+P_Week_2_Exercise_Question (1).ipynb │ ├── S+P_Week_2_Exercise_Question.ipynb │ ├── S+P_Week_2_Lesson_1.ipynb │ ├── S+P_Week_2_Lesson_2.ipynb │ └── S+P_Week_2_Lesson_3.ipynb ├── week3 │ ├── S+P_Week_3_Exercise_Answer.ipynb │ ├── S+P_Week_3_Exercise_Question.ipynb │ └── S+P_Week_3_Lesson_2_RNN.ipynb └── week4 │ ├── NLP_Week4_Exercise_Shakespeare_Question.ipynb │ ├── S+P_Week_4_Exercise_Answer.ipynb │ ├── S+P_Week_4_Exercise_Question.ipynb │ ├── S+P_Week_4_Lesson_1.ipynb │ ├── S+P_Week_4_Lesson_3.ipynb │ └── S+P_Week_4_Lesson_5.ipynb ├── Practise ├── CNN.py ├── CNN_imageGenerator.py ├── CNN_preTrain.py ├── Natural_language_processing.py ├── Natural_language_processing_1.py ├── Natural_language_processing_lstm.py ├── Natural_language_processing_lstm_1.py ├── Natural_language_processing_ngram.py ├── Natural_language_processing_text_generation.py ├── tensorflow_datasets.ipynb └── time_series_2.py ├── README.md └── tensorflow_datasets.ipynb /.DS_Store: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/catwang42/TensorFlow-developer/2169b0b71c3038748ae8c37ed0ceeb8f413ea4b2/.DS_Store -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | .DS_Store -------------------------------------------------------------------------------- /1.Introduction /Codlab-exercise/.DS_Store: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/catwang42/TensorFlow-developer/2169b0b71c3038748ae8c37ed0ceeb8f413ea4b2/1.Introduction /Codlab-exercise/.DS_Store -------------------------------------------------------------------------------- /1.Introduction /Excercise-3-Question-mproving Computer Vision Accuracy using Convolutions .ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": { 6 | "colab_type": "text", 7 | "id": "iQjHqsmTAVLU" 8 | }, 9 | "source": [ 10 | "## Exercise 3\n", 11 | "In the videos you looked at how you would improve Fashion MNIST using Convolutions. For your exercise see if you can improve MNIST to 99.8% accuracy or more using only a single convolutional layer and a single MaxPooling 2D. You should stop training once the accuracy goes above this amount. It should happen in less than 20 epochs, so it's ok to hard code the number of epochs for training, but your training must end once it hits the above metric. If it doesn't, then you'll need to redesign your layers.\n", 12 | "\n", 13 | "I've started the code for you -- you need to finish it!\n", 14 | "\n", 15 | "When 99.8% accuracy has been hit, you should print out the string \"Reached 99.8% accuracy so cancelling training!\"\n" 16 | ] 17 | }, 18 | { 19 | "cell_type": "code", 20 | "execution_count": 1, 21 | "metadata": {}, 22 | "outputs": [], 23 | "source": [ 24 | "import tensorflow as tf\n", 25 | "from os import path, getcwd, chdir\n", 26 | "\n", 27 | "# DO NOT CHANGE THE LINE BELOW. If you are developing in a local\n", 28 | "# environment, then grab mnist.npz from the Coursera Jupyter Notebook\n", 29 | "# and place it inside a local folder and edit the path to that location\n", 30 | "path = f\"{getcwd()}/../tmp2/mnist.npz\"" 31 | ] 32 | }, 33 | { 34 | "cell_type": "code", 35 | "execution_count": 2, 36 | "metadata": {}, 37 | "outputs": [], 38 | "source": [ 39 | "config = tf.ConfigProto()\n", 40 | "config.gpu_options.allow_growth = True\n", 41 | "sess = tf.Session(config=config)" 42 | ] 43 | }, 44 | { 45 | "cell_type": "code", 46 | "execution_count": 16, 47 | "metadata": {}, 48 | "outputs": [], 49 | "source": [ 50 | "# GRADED FUNCTION: train_mnist_conv\n", 51 | "def train_mnist_conv():\n", 52 | " # Please write your code only where you are indicated.\n", 53 | " # please do not remove model fitting inline comments.\n", 54 | "\n", 55 | " # YOUR CODE STARTS HERE\n", 56 | " print(tf.__version__)\n", 57 | " \n", 58 | " # YOUR CODE ENDS HERE\n", 59 | "\n", 60 | " mnist = tf.keras.datasets.mnist\n", 61 | " (training_images, training_labels), (test_images, test_labels) = mnist.load_data(path=path)\n", 62 | " # YOUR CODE STARTS HERE\n", 63 | " training_images = training_images.reshape(60000,28,28,1)\n", 64 | " training_images = training_images /255.0\n", 65 | " test_images = test_images.reshape(10000, 28, 28, 1)\n", 66 | " test_labels = test_labels /255.0\n", 67 | "\n", 68 | " # YOUR CODE ENDS HERE\n", 69 | "\n", 70 | " model = tf.keras.models.Sequential([\n", 71 | " # YOUR CODE STARTS HERE\n", 72 | " tf.keras.layers.Conv2D(64,(3,3),activation='relu',input_shape=(28,28,1)),\n", 73 | " tf.keras.layers.MaxPooling2D(2,2),\n", 74 | " tf.keras.layers.Conv2D(64,(3,3),activation='relu'),\n", 75 | " tf.keras.layers.MaxPooling2D(2,2),\n", 76 | " tf.keras.layers.Flatten(),\n", 77 | " tf.keras.layers.Dense(128,activation='relu'),\n", 78 | " tf.keras.layers.Dense(10,activation='softmax')\n", 79 | "\n", 80 | " # YOUR CODE ENDS HERE\n", 81 | " ])\n", 82 | "\n", 83 | " model.compile(optimizer='adam', loss='sparse_categorical_crossentropy', metrics=['accuracy'])\n", 84 | " # model fitting\n", 85 | " history = model.fit(\n", 86 | " # YOUR CODE STARTS HERE\n", 87 | " training_images,training_labels,epochs=10\n", 88 | "\n", 89 | " # YOUR CODE ENDS HERE\n", 90 | " )\n", 91 | " # model fitting\n", 92 | " return history.epoch, history.history['acc'][-1]\n", 93 | "\n" 94 | ] 95 | }, 96 | { 97 | "cell_type": "code", 98 | "execution_count": 17, 99 | "metadata": {}, 100 | "outputs": [ 101 | { 102 | "name": "stdout", 103 | "output_type": "stream", 104 | "text": [ 105 | "1.14.0\n", 106 | "Epoch 1/10\n", 107 | "60000/60000 [==============================] - 17s 290us/sample - loss: 0.1274 - acc: 0.9605\n", 108 | "Epoch 2/10\n", 109 | "60000/60000 [==============================] - 18s 292us/sample - loss: 0.0412 - acc: 0.9870\n", 110 | "Epoch 3/10\n", 111 | "60000/60000 [==============================] - 18s 295us/sample - loss: 0.0285 - acc: 0.9912\n", 112 | "Epoch 4/10\n", 113 | "60000/60000 [==============================] - 17s 287us/sample - loss: 0.0195 - acc: 0.9940\n", 114 | "Epoch 5/10\n", 115 | "60000/60000 [==============================] - 18s 295us/sample - loss: 0.0150 - acc: 0.9949\n", 116 | "Epoch 6/10\n", 117 | "60000/60000 [==============================] - 19s 309us/sample - loss: 0.0122 - acc: 0.9959\n", 118 | "Epoch 7/10\n", 119 | "60000/60000 [==============================] - 18s 296us/sample - loss: 0.0086 - acc: 0.9972\n", 120 | "Epoch 8/10\n", 121 | "60000/60000 [==============================] - 19s 313us/sample - loss: 0.0089 - acc: 0.9969\n", 122 | "Epoch 9/10\n", 123 | "60000/60000 [==============================] - 19s 310us/sample - loss: 0.0077 - acc: 0.9973\n", 124 | "Epoch 10/10\n", 125 | "60000/60000 [==============================] - 19s 317us/sample - loss: 0.0061 - acc: 0.9980\n" 126 | ] 127 | } 128 | ], 129 | "source": [ 130 | "_, _ = train_mnist_conv()" 131 | ] 132 | }, 133 | { 134 | "cell_type": "code", 135 | "execution_count": null, 136 | "metadata": {}, 137 | "outputs": [], 138 | "source": [ 139 | "# Now click the 'Submit Assignment' button above.\n", 140 | "# Once that is complete, please run the following two cells to save your work and close the notebook" 141 | ] 142 | }, 143 | { 144 | "cell_type": "code", 145 | "execution_count": null, 146 | "metadata": {}, 147 | "outputs": [], 148 | "source": [ 149 | "%%javascript\n", 150 | "\n", 151 | "IPython.notebook.save_checkpoint();" 152 | ] 153 | }, 154 | { 155 | "cell_type": "code", 156 | "execution_count": null, 157 | "metadata": {}, 158 | "outputs": [], 159 | "source": [ 160 | "%%javascript\n", 161 | "IPython.notebook.session.delete();\n", 162 | "window.onbeforeunload = null\n", 163 | "setTimeout(function() { window.close(); }, 1000);" 164 | ] 165 | } 166 | ], 167 | "metadata": { 168 | "coursera": { 169 | "course_slug": "introduction-tensorflow", 170 | "graded_item_id": "ml06H", 171 | "launcher_item_id": "hQF8A" 172 | }, 173 | "kernelspec": { 174 | "display_name": "Python 3", 175 | "language": "python", 176 | "name": "python3" 177 | }, 178 | "language_info": { 179 | "codemirror_mode": { 180 | "name": "ipython", 181 | "version": 3 182 | }, 183 | "file_extension": ".py", 184 | "mimetype": "text/x-python", 185 | "name": "python", 186 | "nbconvert_exporter": "python", 187 | "pygments_lexer": "ipython3", 188 | "version": "3.6.8" 189 | } 190 | }, 191 | "nbformat": 4, 192 | "nbformat_minor": 1 193 | } 194 | -------------------------------------------------------------------------------- /1.Introduction /Exercise4-Question-ImageDataGenerator.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": { 6 | "colab_type": "text", 7 | "id": "UncprnB0ymAE" 8 | }, 9 | "source": [ 10 | "Below is code with a link to a happy or sad dataset which contains 80 images, 40 happy and 40 sad. \n", 11 | "Create a convolutional neural network that trains to 100% accuracy on these images, which cancels training upon hitting training accuracy of >.999\n", 12 | "\n", 13 | "Hint -- it will work best with 3 convolutional layers." 14 | ] 15 | }, 16 | { 17 | "cell_type": "code", 18 | "execution_count": 1, 19 | "metadata": {}, 20 | "outputs": [], 21 | "source": [ 22 | "import tensorflow as tf\n", 23 | "import os\n", 24 | "import zipfile\n", 25 | "from os import path, getcwd, chdir\n", 26 | "\n", 27 | "# DO NOT CHANGE THE LINE BELOW. If you are developing in a local\n", 28 | "# environment, then grab happy-or-sad.zip from the Coursera Jupyter Notebook\n", 29 | "# and place it inside a local folder and edit the path to that location\n", 30 | "path = f\"{getcwd()}/../tmp2/happy-or-sad.zip\"\n", 31 | "\n", 32 | "zip_ref = zipfile.ZipFile(path, 'r')\n", 33 | "zip_ref.extractall(\"/tmp/h-or-s\")\n", 34 | "zip_ref.close()" 35 | ] 36 | }, 37 | { 38 | "cell_type": "code", 39 | "execution_count": 6, 40 | "metadata": {}, 41 | "outputs": [], 42 | "source": [ 43 | "train_horse_dir = os.path.join('/tmp/h-or-s')" 44 | ] 45 | }, 46 | { 47 | "cell_type": "code", 48 | "execution_count": 7, 49 | "metadata": {}, 50 | "outputs": [ 51 | { 52 | "data": { 53 | "text/plain": [ 54 | "11" 55 | ] 56 | }, 57 | "execution_count": 7, 58 | "metadata": {}, 59 | "output_type": "execute_result" 60 | } 61 | ], 62 | "source": [ 63 | "len(train_horse_dir)" 64 | ] 65 | }, 66 | { 67 | "cell_type": "code", 68 | "execution_count": 4, 69 | "metadata": {}, 70 | "outputs": [], 71 | "source": [ 72 | "validation_horse_dir = os.path.join('/tmp/validation-horse-or-human/horses')" 73 | ] 74 | }, 75 | { 76 | "cell_type": "code", 77 | "execution_count": 5, 78 | "metadata": {}, 79 | "outputs": [ 80 | { 81 | "data": { 82 | "text/plain": [ 83 | "37" 84 | ] 85 | }, 86 | "execution_count": 5, 87 | "metadata": {}, 88 | "output_type": "execute_result" 89 | } 90 | ], 91 | "source": [ 92 | "len(validation_horse_dir)" 93 | ] 94 | }, 95 | { 96 | "cell_type": "code", 97 | "execution_count": 13, 98 | "metadata": {}, 99 | "outputs": [], 100 | "source": [ 101 | "# GRADED FUNCTION: train_happy_sad_model\n", 102 | "def train_happy_sad_model():\n", 103 | " # Please write your code only where you are indicated.\n", 104 | " # please do not remove # model fitting inline comments.\n", 105 | "\n", 106 | " DESIRED_ACCURACY = 0.999\n", 107 | "\n", 108 | " class myCallback(tf.keras.callbacks.Callback):\n", 109 | " def on_epoch_end(self, epoch, logs={}):\n", 110 | " if logs.get('acc') > DESIRED_ACCURACY:\n", 111 | " print(\"\\n Reached training accuracy of >.999, stop training process ... \")\n", 112 | " self.model.stop_training = True\n", 113 | "\n", 114 | " callbacks = myCallback()\n", 115 | " \n", 116 | " # This Code Block should Define and Compile the Model. Please assume the images are 150 X 150 in your implementation.\n", 117 | " model = tf.keras.models.Sequential([\n", 118 | " tf.keras.layers.Conv2D(16,(3,3),activation='relu',input_shape=(150,150,3)),\n", 119 | " tf.keras.layers.MaxPooling2D(2,2),\n", 120 | " tf.keras.layers.Conv2D(32,(3,3),activation='relu'),\n", 121 | " tf.keras.layers.MaxPooling2D(2,2),\n", 122 | " tf.keras.layers.Conv2D(64,(3,3),activation='relu'),\n", 123 | " tf.keras.layers.MaxPooling2D(2,2),\n", 124 | " tf.keras.layers.Flatten(),\n", 125 | " tf.keras.layers.Dense(512,activation='relu'),\n", 126 | " tf.keras.layers.Dense(1, activation='sigmoid')\n", 127 | " \n", 128 | " ])\n", 129 | "\n", 130 | " from tensorflow.keras.optimizers import RMSprop\n", 131 | "\n", 132 | " model.compile(loss='binary_crossentropy',\n", 133 | " optimizer= RMSprop(lr=0.001),\n", 134 | " metrics=['acc'])\n", 135 | " \n", 136 | "\n", 137 | " # This code block should create an instance of an ImageDataGenerator called train_datagen \n", 138 | " # And a train_generator by calling train_datagen.flow_from_directory\n", 139 | "\n", 140 | " from tensorflow.keras.preprocessing.image import ImageDataGenerator\n", 141 | "\n", 142 | " train_datagen = ImageDataGenerator(rescale=1./255)\n", 143 | "\n", 144 | " # Please use a target_size of 150 X 150.\n", 145 | " train_generator = train_datagen.flow_from_directory(\n", 146 | " '/tmp/h-or-s',\n", 147 | " target_size=(150,150),\n", 148 | " batch_size=10,\n", 149 | " class_mode='binary')\n", 150 | " # Expected output: 'Found 80 images belonging to 2 classes'\n", 151 | "\n", 152 | " # This code block should call model.fit_generator and train for\n", 153 | " # a number of epochs.\n", 154 | " # model fitting\n", 155 | " history = model.fit_generator(\n", 156 | " train_generator,\n", 157 | " steps_per_epoch=8,\n", 158 | " epochs = 15,\n", 159 | " verbose = 1,\n", 160 | " callbacks=[callbacks])\n", 161 | " # model fitting\n", 162 | " return history.history['acc'][-1]" 163 | ] 164 | }, 165 | { 166 | "cell_type": "code", 167 | "execution_count": 14, 168 | "metadata": {}, 169 | "outputs": [ 170 | { 171 | "name": "stdout", 172 | "output_type": "stream", 173 | "text": [ 174 | "Found 80 images belonging to 2 classes.\n", 175 | "Epoch 1/15\n", 176 | "8/8 [==============================] - 5s 624ms/step - loss: 1.0053 - acc: 0.6500\n", 177 | "Epoch 2/15\n", 178 | "8/8 [==============================] - 0s 52ms/step - loss: 0.5288 - acc: 0.7625\n", 179 | "Epoch 3/15\n", 180 | "8/8 [==============================] - 1s 63ms/step - loss: 0.2428 - acc: 0.9125\n", 181 | "Epoch 4/15\n", 182 | "8/8 [==============================] - 1s 63ms/step - loss: 0.1061 - acc: 0.9750\n", 183 | "Epoch 5/15\n", 184 | "8/8 [==============================] - 0s 62ms/step - loss: 0.0930 - acc: 0.9375\n", 185 | "Epoch 6/15\n", 186 | "8/8 [==============================] - 0s 53ms/step - loss: 0.0320 - acc: 0.9750\n", 187 | "Epoch 7/15\n", 188 | "6/8 [=====================>........] - ETA: 0s - loss: 0.0366 - acc: 1.0000\n", 189 | " Reached training accuracy of >.999, stop training process ... \n", 190 | "8/8 [==============================] - 1s 72ms/step - loss: 0.0320 - acc: 1.0000\n" 191 | ] 192 | }, 193 | { 194 | "data": { 195 | "text/plain": [ 196 | "1.0" 197 | ] 198 | }, 199 | "execution_count": 14, 200 | "metadata": {}, 201 | "output_type": "execute_result" 202 | } 203 | ], 204 | "source": [ 205 | "# The Expected output: \"Reached 99.9% accuracy so cancelling training!\"\"\n", 206 | "train_happy_sad_model()" 207 | ] 208 | }, 209 | { 210 | "cell_type": "code", 211 | "execution_count": 4, 212 | "metadata": {}, 213 | "outputs": [], 214 | "source": [ 215 | "# Now click the 'Submit Assignment' button above.\n", 216 | "# Once that is complete, please run the following two cells to save your work and close the notebook" 217 | ] 218 | }, 219 | { 220 | "cell_type": "code", 221 | "execution_count": null, 222 | "metadata": {}, 223 | "outputs": [], 224 | "source": [ 225 | "%%javascript\n", 226 | "\n", 227 | "IPython.notebook.save_checkpoint();" 228 | ] 229 | }, 230 | { 231 | "cell_type": "code", 232 | "execution_count": null, 233 | "metadata": {}, 234 | "outputs": [], 235 | "source": [ 236 | "%%javascript\n", 237 | "IPython.notebook.session.delete();\n", 238 | "window.onbeforeunload = null\n", 239 | "setTimeout(function() { window.close(); }, 1000);" 240 | ] 241 | } 242 | ], 243 | "metadata": { 244 | "coursera": { 245 | "course_slug": "introduction-tensorflow", 246 | "graded_item_id": "1kAlw", 247 | "launcher_item_id": "PNLYD" 248 | }, 249 | "kernelspec": { 250 | "display_name": "Python 3", 251 | "language": "python", 252 | "name": "python3" 253 | }, 254 | "language_info": { 255 | "codemirror_mode": { 256 | "name": "ipython", 257 | "version": 3 258 | }, 259 | "file_extension": ".py", 260 | "mimetype": "text/x-python", 261 | "name": "python", 262 | "nbconvert_exporter": "python", 263 | "pygments_lexer": "ipython3", 264 | "version": "3.6.8" 265 | } 266 | }, 267 | "nbformat": 4, 268 | "nbformat_minor": 1 269 | } 270 | -------------------------------------------------------------------------------- /1.Introduction /Exercise_2-Question-Fashion MNIST.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": { 6 | "colab_type": "text", 7 | "id": "tOoyQ70H00_s" 8 | }, 9 | "source": [ 10 | "## Exercise 2\n", 11 | "In the course you learned how to do classificaiton using Fashion MNIST, a data set containing items of clothing. There's another, similar dataset called MNIST which has items of handwriting -- the digits 0 through 9.\n", 12 | "\n", 13 | "Write an MNIST classifier that trains to 99% accuracy or above, and does it without a fixed number of epochs -- i.e. you should stop training once you reach that level of accuracy.\n", 14 | "\n", 15 | "Some notes:\n", 16 | "1. It should succeed in less than 10 epochs, so it is okay to change epochs= to 10, but nothing larger\n", 17 | "2. When it reaches 99% or greater it should print out the string \"Reached 99% accuracy so cancelling training!\"\n", 18 | "3. If you add any additional variables, make sure you use the same names as the ones used in the class\n", 19 | "\n", 20 | "I've started the code for you below -- how would you finish it? " 21 | ] 22 | }, 23 | { 24 | "cell_type": "code", 25 | "execution_count": 1, 26 | "metadata": {}, 27 | "outputs": [], 28 | "source": [ 29 | "import tensorflow as tf\n", 30 | "from os import path, getcwd, chdir\n", 31 | "\n", 32 | "# DO NOT CHANGE THE LINE BELOW. If you are developing in a local\n", 33 | "# environment, then grab mnist.npz from the Coursera Jupyter Notebook\n", 34 | "# and place it inside a local folder and edit the path to that location\n", 35 | "path = f\"{getcwd()}/../tmp2/mnist.npz\"" 36 | ] 37 | }, 38 | { 39 | "cell_type": "code", 40 | "execution_count": 6, 41 | "metadata": { 42 | "colab": {}, 43 | "colab_type": "code", 44 | "id": "9rvXQGAA0ssC" 45 | }, 46 | "outputs": [], 47 | "source": [ 48 | "# GRADED FUNCTION: train_mnist\n", 49 | "def train_mnist():\n", 50 | " # Please write your code only where you are indicated.\n", 51 | " # please do not remove # model fitting inline comments.\n", 52 | "\n", 53 | " # YOUR CODE SHOULD START HERE\n", 54 | " mnist = tf.keras.datasets.fashion_mnist\n", 55 | "\n", 56 | " # YOUR CODE SHOULD END HERE\n", 57 | "\n", 58 | " mnist = tf.keras.datasets.mnist\n", 59 | "\n", 60 | " (x_train, y_train),(x_test, y_test) = mnist.load_data(path=path)\n", 61 | " # YOUR CODE SHOULD START HERE\n", 62 | " x_train = x_train / 255.0\n", 63 | " x_test = x_test / 255.0\n", 64 | " \n", 65 | " # YOUR CODE SHOULD END HERE\n", 66 | " model = tf.keras.models.Sequential([\n", 67 | " # YOUR CODE SHOULD START HERE\n", 68 | " tf.keras.layers.Flatten(),\n", 69 | " tf.keras.layers.Dense(128, activation = tf.nn.relu),\n", 70 | " tf.keras.layers.Dense(10, activation= tf.nn.softmax)\n", 71 | " # YOUR CODE SHOULD END HERE\n", 72 | " ])\n", 73 | "\n", 74 | " model.compile(optimizer='adam',\n", 75 | " loss='sparse_categorical_crossentropy',\n", 76 | " metrics=['accuracy'])\n", 77 | " \n", 78 | " # model fitting\n", 79 | " history = model.fit(# YOUR CODE SHOULD START HERE\n", 80 | " x_train,y_train,epochs=5\n", 81 | " # YOUR CODE SHOULD END HERE\n", 82 | " )\n", 83 | " # model fitting\n", 84 | " return history.epoch, history.history['acc'][-1]" 85 | ] 86 | }, 87 | { 88 | "cell_type": "code", 89 | "execution_count": 7, 90 | "metadata": { 91 | "colab": {}, 92 | "colab_type": "code", 93 | "id": "9rvXQGAA0ssC" 94 | }, 95 | "outputs": [ 96 | { 97 | "name": "stderr", 98 | "output_type": "stream", 99 | "text": [ 100 | "WARNING: Logging before flag parsing goes to stderr.\n", 101 | "W0808 13:42:14.624055 139923221956416 deprecation.py:506] From /usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/init_ops.py:1251: calling VarianceScaling.__init__ (from tensorflow.python.ops.init_ops) with dtype is deprecated and will be removed in a future version.\n", 102 | "Instructions for updating:\n", 103 | "Call initializer instance with the dtype argument instead of passing it to the constructor\n" 104 | ] 105 | }, 106 | { 107 | "name": "stdout", 108 | "output_type": "stream", 109 | "text": [ 110 | "Epoch 1/5\n", 111 | "60000/60000 [==============================] - 13s 215us/sample - loss: 0.2525 - acc: 0.9279\n", 112 | "Epoch 2/5\n", 113 | "60000/60000 [==============================] - 13s 208us/sample - loss: 0.1117 - acc: 0.9663\n", 114 | "Epoch 3/5\n", 115 | "60000/60000 [==============================] - 12s 200us/sample - loss: 0.0767 - acc: 0.9772\n", 116 | "Epoch 4/5\n", 117 | "60000/60000 [==============================] - 12s 200us/sample - loss: 0.0578 - acc: 0.9823\n", 118 | "Epoch 5/5\n", 119 | "60000/60000 [==============================] - 12s 195us/sample - loss: 0.0450 - acc: 0.9859\n" 120 | ] 121 | }, 122 | { 123 | "data": { 124 | "text/plain": [ 125 | "([0, 1, 2, 3, 4], 0.98588336)" 126 | ] 127 | }, 128 | "execution_count": 7, 129 | "metadata": {}, 130 | "output_type": "execute_result" 131 | } 132 | ], 133 | "source": [ 134 | "train_mnist()" 135 | ] 136 | }, 137 | { 138 | "cell_type": "code", 139 | "execution_count": 4, 140 | "metadata": {}, 141 | "outputs": [], 142 | "source": [ 143 | "# Now click the 'Submit Assignment' button above.\n", 144 | "# Once that is complete, please run the following two cells to save your work and close the notebook" 145 | ] 146 | }, 147 | { 148 | "cell_type": "code", 149 | "execution_count": null, 150 | "metadata": {}, 151 | "outputs": [], 152 | "source": [ 153 | "%%javascript\n", 154 | "\n", 155 | "IPython.notebook.save_checkpoint();" 156 | ] 157 | }, 158 | { 159 | "cell_type": "code", 160 | "execution_count": null, 161 | "metadata": {}, 162 | "outputs": [], 163 | "source": [ 164 | "%%javascript\n", 165 | "IPython.notebook.session.delete();\n", 166 | "window.onbeforeunload = null\n", 167 | "setTimeout(function() { window.close(); }, 1000);" 168 | ] 169 | } 170 | ], 171 | "metadata": { 172 | "coursera": { 173 | "course_slug": "introduction-tensorflow", 174 | "graded_item_id": "d6dew", 175 | "launcher_item_id": "FExZ4" 176 | }, 177 | "kernelspec": { 178 | "display_name": "Python 3", 179 | "language": "python", 180 | "name": "python3" 181 | }, 182 | "language_info": { 183 | "codemirror_mode": { 184 | "name": "ipython", 185 | "version": 3 186 | }, 187 | "file_extension": ".py", 188 | "mimetype": "text/x-python", 189 | "name": "python", 190 | "nbconvert_exporter": "python", 191 | "pygments_lexer": "ipython3", 192 | "version": "3.6.8" 193 | } 194 | }, 195 | "nbformat": 4, 196 | "nbformat_minor": 1 197 | } 198 | -------------------------------------------------------------------------------- /2.Convolutional Neural Networks in Tensorflow/.DS_Store: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/catwang42/TensorFlow-developer/2169b0b71c3038748ae8c37ed0ceeb8f413ea4b2/2.Convolutional Neural Networks in Tensorflow/.DS_Store -------------------------------------------------------------------------------- /2.Convolutional Neural Networks in Tensorflow/Exercise_4_Multi_class_classifier_Question-FINAL.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": 1, 6 | "metadata": { 7 | "colab": {}, 8 | "colab_type": "code", 9 | "id": "wYtuKeK0dImp" 10 | }, 11 | "outputs": [], 12 | "source": [ 13 | "# ATTENTION: Please do not alter any of the provided code in the exercise. Only add your own code where indicated\n", 14 | "# ATTENTION: Please do not add or remove any cells in the exercise. The grader will check specific cells based on the cell position.\n", 15 | "# ATTENTION: Please use the provided epoch values when training.\n", 16 | "\n", 17 | "import csv\n", 18 | "import numpy as np\n", 19 | "import tensorflow as tf\n", 20 | "from tensorflow.keras.preprocessing.image import ImageDataGenerator\n", 21 | "from os import getcwd" 22 | ] 23 | }, 24 | { 25 | "cell_type": "code", 26 | "execution_count": 2, 27 | "metadata": { 28 | "colab": {}, 29 | "colab_type": "code", 30 | "id": "4kxw-_rmcnVu" 31 | }, 32 | "outputs": [ 33 | { 34 | "name": "stdout", 35 | "output_type": "stream", 36 | "text": [ 37 | "(27455, 28, 28)\n", 38 | "(27455,)\n", 39 | "(7172, 28, 28)\n", 40 | "(7172,)\n" 41 | ] 42 | } 43 | ], 44 | "source": [ 45 | "def get_data(filename):\n", 46 | " # You will need to write code that will read the file passed\n", 47 | " # into this function. The first line contains the column headers\n", 48 | " # so you should ignore it\n", 49 | " # Each successive line contians 785 comma separated values between 0 and 255\n", 50 | " # The first value is the label\n", 51 | " # The rest are the pixel values for that picture\n", 52 | " # The function will return 2 np.array types. One with all the labels\n", 53 | " # One with all the images\n", 54 | " #\n", 55 | " # Tips: \n", 56 | " # If you read a full line (as 'row') then row[0] has the label\n", 57 | " # and row[1:785] has the 784 pixel values\n", 58 | " # Take a look at np.array_split to turn the 784 pixels into 28x28\n", 59 | " # You are reading in strings, but need the values to be floats\n", 60 | " # Check out np.array().astype for a conversion\n", 61 | " with open(filename) as training_file:\n", 62 | " csv_load = csv.reader(training_file, delimiter=',')\n", 63 | " temp_img = []\n", 64 | " temp_label =[]\n", 65 | " next(csv_load)\n", 66 | " for row in csv_load:\n", 67 | " temp_label.append(row[0])\n", 68 | " img = row[1:785]\n", 69 | " temp_img.append(np.array_split(img,28))\n", 70 | " images = np.array(temp_img).astype('float')\n", 71 | " labels = np.array(temp_label).astype('float')\n", 72 | " \n", 73 | " return images, labels\n", 74 | "\n", 75 | "path_sign_mnist_train = f\"{getcwd()}/../tmp2/sign_mnist_train.csv\"\n", 76 | "path_sign_mnist_test = f\"{getcwd()}/../tmp2/sign_mnist_test.csv\"\n", 77 | "training_images, training_labels = get_data(path_sign_mnist_train)\n", 78 | "testing_images, testing_labels = get_data(path_sign_mnist_test)\n", 79 | "\n", 80 | "# Keep these\n", 81 | "print(training_images.shape)\n", 82 | "print(training_labels.shape)\n", 83 | "print(testing_images.shape)\n", 84 | "print(testing_labels.shape)\n", 85 | "\n", 86 | "# Their output should be:\n", 87 | "# (27455, 28, 28)\n", 88 | "# (27455,)\n", 89 | "# (7172, 28, 28)\n", 90 | "# (7172,)" 91 | ] 92 | }, 93 | { 94 | "cell_type": "code", 95 | "execution_count": 3, 96 | "metadata": { 97 | "colab": {}, 98 | "colab_type": "code", 99 | "id": "awoqRpyZdQkD" 100 | }, 101 | "outputs": [ 102 | { 103 | "name": "stdout", 104 | "output_type": "stream", 105 | "text": [ 106 | "(27455, 28, 28, 1)\n", 107 | "(7172, 28, 28, 1)\n" 108 | ] 109 | } 110 | ], 111 | "source": [ 112 | "# In this section you will have to add another dimension to the data\n", 113 | "# So, for example, if your array is (10000, 28, 28)\n", 114 | "# You will need to make it (10000, 28, 28, 1)\n", 115 | "# Hint: np.expand_dims\n", 116 | "\n", 117 | "training_images = np.expand_dims(training_images, axis=3)\n", 118 | "testing_images = np.expand_dims(testing_images, axis=3)\n", 119 | "\n", 120 | "# Create an ImageDataGenerator and do Image Augmentation\n", 121 | "train_datagen = ImageDataGenerator(\n", 122 | " rescale = 1.0 / 255.0,\n", 123 | " rotation_range = 40,\n", 124 | " height_shift_range = 0.2,\n", 125 | " width_shift_range = 0.2,\n", 126 | " shear_range = 0.2,\n", 127 | " zoom_range = 0.2,\n", 128 | " horizontal_flip=True,\n", 129 | " fill_mode = 'nearest'\n", 130 | " )\n", 131 | "\n", 132 | "validation_datagen = ImageDataGenerator(\n", 133 | " rescale = 1. / 255.)\n", 134 | " \n", 135 | "# Keep These\n", 136 | "print(training_images.shape)\n", 137 | "print(testing_images.shape)\n", 138 | " \n", 139 | "# Their output should be:\n", 140 | "# (27455, 28, 28, 1)\n", 141 | "# (7172, 28, 28, 1)" 142 | ] 143 | }, 144 | { 145 | "cell_type": "code", 146 | "execution_count": 4, 147 | "metadata": { 148 | "colab": {}, 149 | "colab_type": "code", 150 | "id": "Rmb7S32cgRqS" 151 | }, 152 | "outputs": [ 153 | { 154 | "name": "stdout", 155 | "output_type": "stream", 156 | "text": [ 157 | "Epoch 1/2\n", 158 | "858/857 [==============================] - 67s 78ms/step - loss: 2.6057 - acc: 0.2024 - val_loss: 1.7982 - val_acc: 0.3694\n", 159 | "Epoch 2/2\n", 160 | "858/857 [==============================] - 63s 74ms/step - loss: 1.7569 - acc: 0.4344 - val_loss: 1.0756 - val_acc: 0.6311\n" 161 | ] 162 | }, 163 | { 164 | "data": { 165 | "text/plain": [ 166 | "[181.0985336516466, 0.4488288]" 167 | ] 168 | }, 169 | "execution_count": 4, 170 | "metadata": {}, 171 | "output_type": "execute_result" 172 | } 173 | ], 174 | "source": [ 175 | "# Define the model\n", 176 | "# Use no more than 2 Conv2D and 2 MaxPooling2D\n", 177 | "model = tf.keras.models.Sequential([\n", 178 | " tf.keras.layers.Conv2D(32,(3,3), activation = 'relu', input_shape=(28,28,1)),\n", 179 | " tf.keras.layers.MaxPooling2D(2,2),\n", 180 | " tf.keras.layers.Conv2D(64,(3,3),activation='relu'),\n", 181 | " tf.keras.layers.MaxPooling2D(2,2),\n", 182 | " tf.keras.layers.Flatten(),\n", 183 | " tf.keras.layers.Dense(512, activation='relu'),\n", 184 | " tf.keras.layers.Dense(26, activation='softmax')\n", 185 | " ])\n", 186 | "\n", 187 | "# Compile Model. \n", 188 | "model.compile(optimizer='adam',\n", 189 | " loss = 'sparse_categorical_crossentropy',\n", 190 | " metrics=['acc'])\n", 191 | "\n", 192 | "# Train the Model #\n", 193 | "history = model.fit_generator(train_datagen.flow(training_images, training_labels,batch_size=32),\n", 194 | " steps_per_epoch = len(training_images)/32,\n", 195 | " epochs=2,\n", 196 | " validation_data= validation_datagen.flow(testing_images, testing_labels,batch_size=32),\n", 197 | " validation_steps=len(testing_images)/32)\n", 198 | "\n", 199 | "model.evaluate(testing_images, testing_labels, verbose=0)" 200 | ] 201 | }, 202 | { 203 | "cell_type": "code", 204 | "execution_count": 5, 205 | "metadata": { 206 | "colab": {}, 207 | "colab_type": "code", 208 | "id": "_Q3Zpr46dsij" 209 | }, 210 | "outputs": [ 211 | { 212 | "ename": "KeyError", 213 | "evalue": "'accuracy'", 214 | "output_type": "error", 215 | "traceback": [ 216 | "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m", 217 | "\u001b[0;31mKeyError\u001b[0m Traceback (most recent call last)", 218 | "\u001b[0;32m\u001b[0m in \u001b[0;36m\u001b[0;34m\u001b[0m\n\u001b[1;32m 2\u001b[0m \u001b[0mget_ipython\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mrun_line_magic\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m'matplotlib'\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m'inline'\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 3\u001b[0m \u001b[0;32mimport\u001b[0m \u001b[0mmatplotlib\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mpyplot\u001b[0m \u001b[0;32mas\u001b[0m \u001b[0mplt\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 4\u001b[0;31m \u001b[0macc\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mhistory\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mhistory\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m'accuracy'\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 5\u001b[0m \u001b[0mval_acc\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mhistory\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mhistory\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m'val_accuracy'\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 6\u001b[0m \u001b[0mloss\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mhistory\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mhistory\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m'loss'\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n", 219 | "\u001b[0;31mKeyError\u001b[0m: 'accuracy'" 220 | ] 221 | } 222 | ], 223 | "source": [ 224 | "# Plot the chart for accuracy and loss on both training and validation\n", 225 | "%matplotlib inline\n", 226 | "import matplotlib.pyplot as plt\n", 227 | "acc = history.history['acc']\n", 228 | "val_acc = history.history['val_acc']\n", 229 | "loss = history.history['loss']\n", 230 | "val_loss = history.history['val_loss']\n", 231 | "\n", 232 | "\n", 233 | "epochs = range(len(acc))\n", 234 | "\n", 235 | "plt.plot(epochs, acc, 'r', label='Training accuracy')\n", 236 | "plt.plot(epochs, val_acc, 'b', label='Validation accuracy')\n", 237 | "plt.title('Training and validation accuracy')\n", 238 | "plt.legend()\n", 239 | "plt.figure()\n", 240 | "\n", 241 | "plt.plot(epochs, loss, 'r', label='Training Loss')\n", 242 | "plt.plot(epochs, val_loss, 'b', label='Validation Loss')\n", 243 | "plt.title('Training and validation loss')\n", 244 | "plt.legend()\n", 245 | "\n", 246 | "plt.show()" 247 | ] 248 | }, 249 | { 250 | "cell_type": "markdown", 251 | "metadata": {}, 252 | "source": [ 253 | "# Submission Instructions" 254 | ] 255 | }, 256 | { 257 | "cell_type": "code", 258 | "execution_count": null, 259 | "metadata": {}, 260 | "outputs": [], 261 | "source": [ 262 | "# Now click the 'Submit Assignment' button above." 263 | ] 264 | }, 265 | { 266 | "cell_type": "markdown", 267 | "metadata": {}, 268 | "source": [ 269 | "# When you're done or would like to take a break, please run the two cells below to save your work and close the Notebook. This will free up resources for your fellow learners. " 270 | ] 271 | }, 272 | { 273 | "cell_type": "code", 274 | "execution_count": null, 275 | "metadata": {}, 276 | "outputs": [], 277 | "source": [ 278 | "%%javascript\n", 279 | "\n", 280 | "IPython.notebook.save_checkpoint();" 281 | ] 282 | }, 283 | { 284 | "cell_type": "code", 285 | "execution_count": null, 286 | "metadata": {}, 287 | "outputs": [], 288 | "source": [ 289 | "%%javascript\n", 290 | "IPython.notebook.session.delete();\n", 291 | "window.onbeforeunload = null\n", 292 | "setTimeout(function() { window.close(); }, 1000);" 293 | ] 294 | } 295 | ], 296 | "metadata": { 297 | "colab": { 298 | "name": "Exercise 8 - Question.ipynb", 299 | "provenance": [] 300 | }, 301 | "coursera": { 302 | "course_slug": "convolutional-neural-networks-tensorflow", 303 | "graded_item_id": "8mIh8", 304 | "launcher_item_id": "gg95t" 305 | }, 306 | "kernelspec": { 307 | "display_name": "Python 3", 308 | "language": "python", 309 | "name": "python3" 310 | }, 311 | "language_info": { 312 | "codemirror_mode": { 313 | "name": "ipython", 314 | "version": 3 315 | }, 316 | "file_extension": ".py", 317 | "mimetype": "text/x-python", 318 | "name": "python", 319 | "nbconvert_exporter": "python", 320 | "pygments_lexer": "ipython3", 321 | "version": "3.6.8" 322 | } 323 | }, 324 | "nbformat": 4, 325 | "nbformat_minor": 1 326 | } 327 | -------------------------------------------------------------------------------- /3.Natural Language Processing in TensorFlow/.DS_Store: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/catwang42/TensorFlow-developer/2169b0b71c3038748ae8c37ed0ceeb8f413ea4b2/3.Natural Language Processing in TensorFlow/.DS_Store -------------------------------------------------------------------------------- /3.Natural Language Processing in TensorFlow/week2/Course_3_Week_2_Lesson_1.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "nbformat": 4, 3 | "nbformat_minor": 0, 4 | "metadata": { 5 | "accelerator": "GPU", 6 | "colab": { 7 | "name": "Course 3 - Week 2 - Lesson 1.ipynb", 8 | "provenance": [], 9 | "collapsed_sections": [], 10 | "toc_visible": true 11 | }, 12 | "kernelspec": { 13 | "display_name": "Python 3", 14 | "language": "python", 15 | "name": "python3" 16 | }, 17 | "language_info": { 18 | "codemirror_mode": { 19 | "name": "ipython", 20 | "version": 3 21 | }, 22 | "file_extension": ".py", 23 | "mimetype": "text/x-python", 24 | "name": "python", 25 | "nbconvert_exporter": "python", 26 | "pygments_lexer": "ipython3", 27 | "version": "3.6.9" 28 | } 29 | }, 30 | "cells": [ 31 | { 32 | "cell_type": "code", 33 | "metadata": { 34 | "colab_type": "code", 35 | "id": "zX4Kg8DUTKWO", 36 | "colab": {} 37 | }, 38 | "source": [ 39 | "#@title Licensed under the Apache License, Version 2.0 (the \"License\");\n", 40 | "# you may not use this file except in compliance with the License.\n", 41 | "# You may obtain a copy of the License at\n", 42 | "#\n", 43 | "# https://www.apache.org/licenses/LICENSE-2.0\n", 44 | "#\n", 45 | "# Unless required by applicable law or agreed to in writing, software\n", 46 | "# distributed under the License is distributed on an \"AS IS\" BASIS,\n", 47 | "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n", 48 | "# See the License for the specific language governing permissions and\n", 49 | "# limitations under the License." 50 | ], 51 | "execution_count": null, 52 | "outputs": [] 53 | }, 54 | { 55 | "cell_type": "markdown", 56 | "metadata": { 57 | "colab_type": "text", 58 | "id": "view-in-github" 59 | }, 60 | "source": [ 61 | "\"Open" 62 | ] 63 | }, 64 | { 65 | "cell_type": "code", 66 | "metadata": { 67 | "colab_type": "code", 68 | "id": "P-AhVYeBWgQ3", 69 | "colab": { 70 | "base_uri": "https://localhost:8080/", 71 | "height": 34 72 | }, 73 | "outputId": "4fdaaf1a-6e00-4e13-bc22-ada2c735fd47" 74 | }, 75 | "source": [ 76 | "import tensorflow as tf\n", 77 | "print(tf.__version__)\n", 78 | "\n", 79 | "# !pip install -q tensorflow-datasets" 80 | ], 81 | "execution_count": 1, 82 | "outputs": [ 83 | { 84 | "output_type": "stream", 85 | "text": [ 86 | "2.3.0\n" 87 | ], 88 | "name": "stdout" 89 | } 90 | ] 91 | }, 92 | { 93 | "cell_type": "code", 94 | "metadata": { 95 | "colab_type": "code", 96 | "id": "_IoM4VFxWpMR", 97 | "colab": {} 98 | }, 99 | "source": [ 100 | "import tensorflow_datasets as tfds\n", 101 | "imdb, info = tfds.load(\"imdb_reviews\", with_info=True, as_supervised=True)\n" 102 | ], 103 | "execution_count": null, 104 | "outputs": [] 105 | }, 106 | { 107 | "cell_type": "code", 108 | "metadata": { 109 | "colab_type": "code", 110 | "id": "wHQ2Ko0zl7M4", 111 | "colab": {} 112 | }, 113 | "source": [ 114 | "import numpy as np\n", 115 | "\n", 116 | "train_data, test_data = imdb['train'], imdb['test']\n", 117 | "\n", 118 | "training_sentences = []\n", 119 | "training_labels = []\n", 120 | "\n", 121 | "testing_sentences = []\n", 122 | "testing_labels = []\n", 123 | "\n", 124 | "# str(s.tonumpy()) is needed in Python3 instead of just s.numpy()\n", 125 | "for s,l in train_data:\n", 126 | " training_sentences.append(s.numpy().decode('utf8'))\n", 127 | " training_labels.append(l.numpy())\n", 128 | " \n", 129 | "for s,l in test_data:\n", 130 | " testing_sentences.append(s.numpy().decode('utf8'))\n", 131 | " testing_labels.append(l.numpy())\n", 132 | " \n", 133 | "training_labels_final = np.array(training_labels)\n", 134 | "testing_labels_final = np.array(testing_labels)\n" 135 | ], 136 | "execution_count": null, 137 | "outputs": [] 138 | }, 139 | { 140 | "cell_type": "code", 141 | "metadata": { 142 | "colab_type": "code", 143 | "id": "7n15yyMdmoH1", 144 | "colab": {} 145 | }, 146 | "source": [ 147 | "vocab_size = 10000\n", 148 | "embedding_dim = 16\n", 149 | "max_length = 120\n", 150 | "trunc_type='post'\n", 151 | "oov_tok = \"\"\n", 152 | "\n", 153 | "\n", 154 | "from tensorflow.keras.preprocessing.text import Tokenizer\n", 155 | "from tensorflow.keras.preprocessing.sequence import pad_sequences\n", 156 | "\n", 157 | "tokenizer = Tokenizer(num_words = vocab_size, oov_token=oov_tok)\n", 158 | "tokenizer.fit_on_texts(training_sentences)\n", 159 | "word_index = tokenizer.word_index\n", 160 | "sequences = tokenizer.texts_to_sequences(training_sentences)\n", 161 | "padded = pad_sequences(sequences,maxlen=max_length, truncating=trunc_type)\n", 162 | "\n", 163 | "testing_sequences = tokenizer.texts_to_sequences(testing_sentences)\n", 164 | "testing_padded = pad_sequences(testing_sequences,maxlen=max_length)\n" 165 | ], 166 | "execution_count": null, 167 | "outputs": [] 168 | }, 169 | { 170 | "cell_type": "code", 171 | "metadata": { 172 | "colab_type": "code", 173 | "id": "9axf0uIXVMhO", 174 | "colab": {} 175 | }, 176 | "source": [ 177 | "reverse_word_index = dict([(value, key) for (key, value) in word_index.items()])\n", 178 | "\n", 179 | "def decode_review(text):\n", 180 | " return ' '.join([reverse_word_index.get(i, '?') for i in text])\n", 181 | "\n", 182 | "print(decode_review(padded[3]))\n", 183 | "print(training_sentences[3])" 184 | ], 185 | "execution_count": null, 186 | "outputs": [] 187 | }, 188 | { 189 | "cell_type": "code", 190 | "metadata": { 191 | "colab_type": "code", 192 | "id": "5NEpdhb8AxID", 193 | "colab": {} 194 | }, 195 | "source": [ 196 | "model = tf.keras.Sequential([\n", 197 | " tf.keras.layers.Embedding(vocab_size, embedding_dim, input_length=max_length),\n", 198 | " tf.keras.layers.Flatten(),\n", 199 | " tf.keras.layers.Dense(6, activation='relu'),\n", 200 | " tf.keras.layers.Dense(1, activation='sigmoid')\n", 201 | "])\n", 202 | "model.compile(loss='binary_crossentropy',optimizer='adam',metrics=['accuracy'])\n", 203 | "model.summary()\n" 204 | ], 205 | "execution_count": null, 206 | "outputs": [] 207 | }, 208 | { 209 | "cell_type": "code", 210 | "metadata": { 211 | "colab_type": "code", 212 | "id": "V5LLrXC-uNX6", 213 | "colab": {} 214 | }, 215 | "source": [ 216 | "num_epochs = 10\n", 217 | "model.fit(padded, training_labels_final, epochs=num_epochs, validation_data=(testing_padded, testing_labels_final))" 218 | ], 219 | "execution_count": null, 220 | "outputs": [] 221 | }, 222 | { 223 | "cell_type": "code", 224 | "metadata": { 225 | "colab_type": "code", 226 | "id": "yAmjJqEyCOF_", 227 | "colab": {} 228 | }, 229 | "source": [ 230 | "e = model.layers[0]\n", 231 | "weights = e.get_weights()[0]\n", 232 | "print(weights.shape) # shape: (vocab_size, embedding_dim)" 233 | ], 234 | "execution_count": null, 235 | "outputs": [] 236 | }, 237 | { 238 | "cell_type": "code", 239 | "metadata": { 240 | "colab_type": "code", 241 | "id": "jmB0Uxk0ycP6", 242 | "colab": {} 243 | }, 244 | "source": [ 245 | "import io\n", 246 | "\n", 247 | "out_v = io.open('vecs.tsv', 'w', encoding='utf-8')\n", 248 | "out_m = io.open('meta.tsv', 'w', encoding='utf-8')\n", 249 | "for word_num in range(1, vocab_size):\n", 250 | " word = reverse_word_index[word_num]\n", 251 | " embeddings = weights[word_num]\n", 252 | " out_m.write(word + \"\\n\")\n", 253 | " out_v.write('\\t'.join([str(x) for x in embeddings]) + \"\\n\")\n", 254 | "out_v.close()\n", 255 | "out_m.close()" 256 | ], 257 | "execution_count": null, 258 | "outputs": [] 259 | }, 260 | { 261 | "cell_type": "code", 262 | "metadata": { 263 | "colab_type": "code", 264 | "id": "VDeqpOCVydtq", 265 | "colab": {} 266 | }, 267 | "source": [ 268 | "try:\n", 269 | " from google.colab import files\n", 270 | "except ImportError:\n", 271 | " pass\n", 272 | "else:\n", 273 | " files.download('vecs.tsv')\n", 274 | " files.download('meta.tsv')" 275 | ], 276 | "execution_count": null, 277 | "outputs": [] 278 | }, 279 | { 280 | "cell_type": "code", 281 | "metadata": { 282 | "colab_type": "code", 283 | "id": "YRxoxc2apscY", 284 | "colab": {} 285 | }, 286 | "source": [ 287 | "sentence = \"I really think this is amazing. honest.\"\n", 288 | "sequence = tokenizer.texts_to_sequences([sentence])\n", 289 | "print(sequence)" 290 | ], 291 | "execution_count": null, 292 | "outputs": [] 293 | } 294 | ] 295 | } -------------------------------------------------------------------------------- /3.Natural Language Processing in TensorFlow/week2/Course_3_Week_2_Lesson_3.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "nbformat": 4, 3 | "nbformat_minor": 0, 4 | "metadata": { 5 | "accelerator": "GPU", 6 | "colab": { 7 | "name": "Course 3 - Week 2 - Lesson 3.ipynb", 8 | "provenance": [], 9 | "collapsed_sections": [], 10 | "toc_visible": true 11 | }, 12 | "kernelspec": { 13 | "display_name": "Python 3", 14 | "language": "python", 15 | "name": "python3" 16 | }, 17 | "language_info": { 18 | "codemirror_mode": { 19 | "name": "ipython", 20 | "version": 3 21 | }, 22 | "file_extension": ".py", 23 | "mimetype": "text/x-python", 24 | "name": "python", 25 | "nbconvert_exporter": "python", 26 | "pygments_lexer": "ipython3", 27 | "version": "3.7.6" 28 | } 29 | }, 30 | "cells": [ 31 | { 32 | "cell_type": "code", 33 | "metadata": { 34 | "colab_type": "code", 35 | "id": "zX4Kg8DUTKWO", 36 | "colab": {} 37 | }, 38 | "source": [ 39 | "#@title Licensed under the Apache License, Version 2.0 (the \"License\");\n", 40 | "# you may not use this file except in compliance with the License.\n", 41 | "# You may obtain a copy of the License at\n", 42 | "#\n", 43 | "# https://www.apache.org/licenses/LICENSE-2.0\n", 44 | "#\n", 45 | "# Unless required by applicable law or agreed to in writing, software\n", 46 | "# distributed under the License is distributed on an \"AS IS\" BASIS,\n", 47 | "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n", 48 | "# See the License for the specific language governing permissions and\n", 49 | "# limitations under the License." 50 | ], 51 | "execution_count": null, 52 | "outputs": [] 53 | }, 54 | { 55 | "cell_type": "markdown", 56 | "metadata": { 57 | "id": "OqQqxaDXhcY3", 58 | "colab_type": "text" 59 | }, 60 | "source": [ 61 | "\"Open" 62 | ] 63 | }, 64 | { 65 | "cell_type": "code", 66 | "metadata": { 67 | "colab_type": "code", 68 | "id": "P-AhVYeBWgQ3", 69 | "colab": {} 70 | }, 71 | "source": [ 72 | "# NOTE: PLEASE MAKE SURE YOU ARE RUNNING THIS IN A PYTHON3 ENVIRONMENT\n", 73 | "\n", 74 | "import tensorflow as tf\n", 75 | "print(tf.__version__)\n" 76 | ], 77 | "execution_count": null, 78 | "outputs": [] 79 | }, 80 | { 81 | "cell_type": "code", 82 | "metadata": { 83 | "colab_type": "code", 84 | "id": "wm9S3T8-9H4q", 85 | "colab": {} 86 | }, 87 | "source": [ 88 | "# Double check TF 2.0x is installed. If you ran the above block, there was a \n", 89 | "# 'reset all runtimes' button at the bottom that you needed to press\n", 90 | "import tensorflow as tf\n", 91 | "print(tf.__version__)" 92 | ], 93 | "execution_count": null, 94 | "outputs": [] 95 | }, 96 | { 97 | "cell_type": "code", 98 | "metadata": { 99 | "colab_type": "code", 100 | "id": "_IoM4VFxWpMR", 101 | "colab": {} 102 | }, 103 | "source": [ 104 | "# If the import fails, run this\n", 105 | "# !pip install -q tensorflow-datasets\n", 106 | "\n", 107 | "import tensorflow_datasets as tfds\n", 108 | "imdb, info = tfds.load(\"imdb_reviews/subwords8k\", with_info=True, as_supervised=True)\n" 109 | ], 110 | "execution_count": null, 111 | "outputs": [] 112 | }, 113 | { 114 | "cell_type": "code", 115 | "metadata": { 116 | "colab_type": "code", 117 | "id": "wHQ2Ko0zl7M4", 118 | "colab": {} 119 | }, 120 | "source": [ 121 | "train_data, test_data = imdb['train'], imdb['test']" 122 | ], 123 | "execution_count": null, 124 | "outputs": [] 125 | }, 126 | { 127 | "cell_type": "code", 128 | "metadata": { 129 | "colab_type": "code", 130 | "id": "fqGRSe_eCdOz", 131 | "colab": {} 132 | }, 133 | "source": [ 134 | "tokenizer = info.features['text'].encoder" 135 | ], 136 | "execution_count": null, 137 | "outputs": [] 138 | }, 139 | { 140 | "cell_type": "code", 141 | "metadata": { 142 | "colab_type": "code", 143 | "id": "F6b_EraCDLOh", 144 | "colab": {} 145 | }, 146 | "source": [ 147 | "print(tokenizer.subwords)" 148 | ], 149 | "execution_count": null, 150 | "outputs": [] 151 | }, 152 | { 153 | "cell_type": "code", 154 | "metadata": { 155 | "colab_type": "code", 156 | "id": "fPl2BXhYEHRP", 157 | "colab": {} 158 | }, 159 | "source": [ 160 | "sample_string = 'TensorFlow, from basics to mastery'\n", 161 | "\n", 162 | "tokenized_string = tokenizer.encode(sample_string)\n", 163 | "print ('Tokenized string is {}'.format(tokenized_string))\n", 164 | "\n", 165 | "original_string = tokenizer.decode(tokenized_string)\n", 166 | "print ('The original string: {}'.format(original_string))\n" 167 | ], 168 | "execution_count": null, 169 | "outputs": [] 170 | }, 171 | { 172 | "cell_type": "code", 173 | "metadata": { 174 | "colab_type": "code", 175 | "id": "_3t7vvNLEZml", 176 | "colab": {} 177 | }, 178 | "source": [ 179 | "for ts in tokenized_string:\n", 180 | " print ('{} ----> {}'.format(ts, tokenizer.decode([ts])))" 181 | ], 182 | "execution_count": null, 183 | "outputs": [] 184 | }, 185 | { 186 | "cell_type": "code", 187 | "metadata": { 188 | "id": "L2y0CqdrhcZS", 189 | "colab_type": "code", 190 | "colab": {} 191 | }, 192 | "source": [ 193 | "BUFFER_SIZE = 10000\n", 194 | "BATCH_SIZE = 64\n", 195 | "\n", 196 | "train_dataset = train_data.shuffle(BUFFER_SIZE)\n", 197 | "train_dataset = train_dataset.padded_batch(BATCH_SIZE, tf.compat.v1.data.get_output_shapes(train_dataset))\n", 198 | "test_dataset = test_data.padded_batch(BATCH_SIZE, tf.compat.v1.data.get_output_shapes(test_data))" 199 | ], 200 | "execution_count": null, 201 | "outputs": [] 202 | }, 203 | { 204 | "cell_type": "code", 205 | "metadata": { 206 | "colab_type": "code", 207 | "id": "5NEpdhb8AxID", 208 | "colab": {} 209 | }, 210 | "source": [ 211 | "embedding_dim = 64\n", 212 | "model = tf.keras.Sequential([\n", 213 | " tf.keras.layers.Embedding(tokenizer.vocab_size, embedding_dim),\n", 214 | " tf.keras.layers.GlobalAveragePooling1D(),\n", 215 | " tf.keras.layers.Dense(6, activation='relu'),\n", 216 | " tf.keras.layers.Dense(1, activation='sigmoid')\n", 217 | "])\n", 218 | "\n", 219 | "model.summary()" 220 | ], 221 | "execution_count": null, 222 | "outputs": [] 223 | }, 224 | { 225 | "cell_type": "code", 226 | "metadata": { 227 | "colab_type": "code", 228 | "id": "fkt8c5dNuUlT", 229 | "colab": {} 230 | }, 231 | "source": [ 232 | "num_epochs = 10\n", 233 | "\n", 234 | "model.compile(loss='binary_crossentropy',optimizer='adam',metrics=['accuracy'])\n", 235 | "\n", 236 | "history = model.fit(train_data, epochs=num_epochs, validation_data=test_data)" 237 | ], 238 | "execution_count": null, 239 | "outputs": [] 240 | }, 241 | { 242 | "cell_type": "code", 243 | "metadata": { 244 | "colab_type": "code", 245 | "id": "-_rMnm7WxQGT", 246 | "colab": {} 247 | }, 248 | "source": [ 249 | "import matplotlib.pyplot as plt\n", 250 | "\n", 251 | "\n", 252 | "def plot_graphs(history, string):\n", 253 | " plt.plot(history.history[string])\n", 254 | " plt.plot(history.history['val_'+string])\n", 255 | " plt.xlabel(\"Epochs\")\n", 256 | " plt.ylabel(string)\n", 257 | " plt.legend([string, 'val_'+string])\n", 258 | " plt.show()\n", 259 | " \n", 260 | "plot_graphs(history, \"accuracy\")\n", 261 | "plot_graphs(history, \"loss\")" 262 | ], 263 | "execution_count": null, 264 | "outputs": [] 265 | }, 266 | { 267 | "cell_type": "code", 268 | "metadata": { 269 | "colab_type": "code", 270 | "id": "qACq5FLzTW4A", 271 | "colab": {} 272 | }, 273 | "source": [ 274 | "e = model.layers[0]\n", 275 | "weights = e.get_weights()[0]\n", 276 | "print(weights.shape) # shape: (vocab_size, embedding_dim)\n", 277 | "\n", 278 | "import io\n", 279 | "\n", 280 | "out_v = io.open('vecs.tsv', 'w', encoding='utf-8')\n", 281 | "out_m = io.open('meta.tsv', 'w', encoding='utf-8')\n", 282 | "for word_num in range(1, tokenizer.vocab_size):\n", 283 | " word = tokenizer.decode([word_num])\n", 284 | " embeddings = weights[word_num]\n", 285 | " out_m.write(word + \"\\n\")\n", 286 | " out_v.write('\\t'.join([str(x) for x in embeddings]) + \"\\n\")\n", 287 | "out_v.close()\n", 288 | "out_m.close()\n", 289 | "\n", 290 | "\n", 291 | "try:\n", 292 | " from google.colab import files\n", 293 | "except ImportError:\n", 294 | " pass\n", 295 | "else:\n", 296 | " files.download('vecs.tsv')\n", 297 | " files.download('meta.tsv')" 298 | ], 299 | "execution_count": null, 300 | "outputs": [] 301 | } 302 | ] 303 | } -------------------------------------------------------------------------------- /3.Natural Language Processing in TensorFlow/week3/Course_3_Week_3_Lesson_1c.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "nbformat": 4, 3 | "nbformat_minor": 0, 4 | "metadata": { 5 | "accelerator": "GPU", 6 | "colab": { 7 | "name": "Course 3 - Week 3 - Lesson 1c.ipynb", 8 | "provenance": [], 9 | "collapsed_sections": [], 10 | "toc_visible": true 11 | }, 12 | "kernelspec": { 13 | "display_name": "Python 3", 14 | "name": "python3" 15 | } 16 | }, 17 | "cells": [ 18 | { 19 | "cell_type": "code", 20 | "metadata": { 21 | "id": "zX4Kg8DUTKWO", 22 | "colab_type": "code", 23 | "colab": {} 24 | }, 25 | "source": [ 26 | "#@title Licensed under the Apache License, Version 2.0 (the \"License\");\n", 27 | "# you may not use this file except in compliance with the License.\n", 28 | "# You may obtain a copy of the License at\n", 29 | "#\n", 30 | "# https://www.apache.org/licenses/LICENSE-2.0\n", 31 | "#\n", 32 | "# Unless required by applicable law or agreed to in writing, software\n", 33 | "# distributed under the License is distributed on an \"AS IS\" BASIS,\n", 34 | "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n", 35 | "# See the License for the specific language governing permissions and\n", 36 | "# limitations under the License." 37 | ], 38 | "execution_count": null, 39 | "outputs": [] 40 | }, 41 | { 42 | "cell_type": "markdown", 43 | "metadata": { 44 | "colab_type": "text", 45 | "id": "rFiCyWQ-NC5D" 46 | }, 47 | "source": [ 48 | "# Multiple Layer GRU" 49 | ] 50 | }, 51 | { 52 | "cell_type": "code", 53 | "metadata": { 54 | "colab_type": "code", 55 | "id": "Y20Lud2ZMBhW", 56 | "colab": {} 57 | }, 58 | "source": [ 59 | "from __future__ import absolute_import, division, print_function, unicode_literals\n", 60 | "\n", 61 | "\n", 62 | "import tensorflow_datasets as tfds\n", 63 | "import tensorflow as tf\n", 64 | "print(tf.__version__)" 65 | ], 66 | "execution_count": null, 67 | "outputs": [] 68 | }, 69 | { 70 | "cell_type": "code", 71 | "metadata": { 72 | "colab_type": "code", 73 | "id": "uAU8g7C0MPZE", 74 | "colab": {} 75 | }, 76 | "source": [ 77 | "import tensorflow_datasets as tfds\n", 78 | "import tensorflow as tf\n", 79 | "print(tf.__version__)" 80 | ], 81 | "execution_count": null, 82 | "outputs": [] 83 | }, 84 | { 85 | "cell_type": "code", 86 | "metadata": { 87 | "colab_type": "code", 88 | "id": "AW-4Vo4TMUHb", 89 | "colab": {} 90 | }, 91 | "source": [ 92 | "# Get the data\n", 93 | "dataset, info = tfds.load('imdb_reviews/subwords8k', with_info=True, as_supervised=True)\n", 94 | "train_dataset, test_dataset = dataset['train'], dataset['test']\n" 95 | ], 96 | "execution_count": null, 97 | "outputs": [] 98 | }, 99 | { 100 | "cell_type": "code", 101 | "metadata": { 102 | "colab_type": "code", 103 | "id": "L11bIR6-PKvs", 104 | "colab": {} 105 | }, 106 | "source": [ 107 | "tokenizer = info.features['text'].encoder" 108 | ], 109 | "execution_count": null, 110 | "outputs": [] 111 | }, 112 | { 113 | "cell_type": "code", 114 | "metadata": { 115 | "colab_type": "code", 116 | "id": "ffvRUI0_McDS", 117 | "colab": {} 118 | }, 119 | "source": [ 120 | "BUFFER_SIZE = 10000\n", 121 | "BATCH_SIZE = 64\n", 122 | "\n", 123 | "train_dataset = train_dataset.shuffle(BUFFER_SIZE)\n", 124 | "train_dataset = train_dataset.padded_batch(BATCH_SIZE, train_dataset.output_shapes)\n", 125 | "test_dataset = test_dataset.padded_batch(BATCH_SIZE, test_dataset.output_shapes)" 126 | ], 127 | "execution_count": null, 128 | "outputs": [] 129 | }, 130 | { 131 | "cell_type": "code", 132 | "metadata": { 133 | "colab_type": "code", 134 | "id": "jo1jjO3vn0jo", 135 | "colab": {} 136 | }, 137 | "source": [ 138 | "model = tf.keras.Sequential([\n", 139 | " tf.keras.layers.Embedding(tokenizer.vocab_size, 64),\n", 140 | " tf.keras.layers.Conv1D(128, 5, activation='relu'),\n", 141 | " tf.keras.layers.GlobalAveragePooling1D(),\n", 142 | " tf.keras.layers.Dense(64, activation='relu'),\n", 143 | " tf.keras.layers.Dense(1, activation='sigmoid')\n", 144 | "])" 145 | ], 146 | "execution_count": null, 147 | "outputs": [] 148 | }, 149 | { 150 | "cell_type": "code", 151 | "metadata": { 152 | "colab_type": "code", 153 | "id": "QKI5dfPgMioL", 154 | "colab": {} 155 | }, 156 | "source": [ 157 | "model.summary()" 158 | ], 159 | "execution_count": null, 160 | "outputs": [] 161 | }, 162 | { 163 | "cell_type": "code", 164 | "metadata": { 165 | "colab_type": "code", 166 | "id": "Uip7QOVzMoMq", 167 | "colab": {} 168 | }, 169 | "source": [ 170 | "model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])" 171 | ], 172 | "execution_count": null, 173 | "outputs": [] 174 | }, 175 | { 176 | "cell_type": "code", 177 | "metadata": { 178 | "colab_type": "code", 179 | "id": "7mlgzaRDMtF6", 180 | "colab": {} 181 | }, 182 | "source": [ 183 | "NUM_EPOCHS = 10\n", 184 | "history = model.fit(train_dataset, epochs=NUM_EPOCHS, validation_data=test_dataset)" 185 | ], 186 | "execution_count": null, 187 | "outputs": [] 188 | }, 189 | { 190 | "cell_type": "code", 191 | "metadata": { 192 | "colab_type": "code", 193 | "id": "Mp1Z7P9pYRSK", 194 | "colab": {} 195 | }, 196 | "source": [ 197 | "import matplotlib.pyplot as plt\n", 198 | "\n", 199 | "\n", 200 | "def plot_graphs(history, string):\n", 201 | " plt.plot(history.history[string])\n", 202 | " plt.plot(history.history['val_'+string])\n", 203 | " plt.xlabel(\"Epochs\")\n", 204 | " plt.ylabel(string)\n", 205 | " plt.legend([string, 'val_'+string])\n", 206 | " plt.show()" 207 | ], 208 | "execution_count": null, 209 | "outputs": [] 210 | }, 211 | { 212 | "cell_type": "code", 213 | "metadata": { 214 | "colab_type": "code", 215 | "id": "R_sX6ilIM515", 216 | "colab": {} 217 | }, 218 | "source": [ 219 | "plot_graphs(history, 'accuracy')" 220 | ], 221 | "execution_count": null, 222 | "outputs": [] 223 | }, 224 | { 225 | "cell_type": "code", 226 | "metadata": { 227 | "colab_type": "code", 228 | "id": "RFEXtKtqNARB", 229 | "colab": {} 230 | }, 231 | "source": [ 232 | "plot_graphs(history, 'loss')" 233 | ], 234 | "execution_count": null, 235 | "outputs": [] 236 | } 237 | ] 238 | } -------------------------------------------------------------------------------- /3.Natural Language Processing in TensorFlow/week3/Course_3_Week_3_Lesson_2.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "nbformat": 4, 3 | "nbformat_minor": 0, 4 | "metadata": { 5 | "accelerator": "GPU", 6 | "colab": { 7 | "name": "Course 3 - Week 3 - Lesson 2.ipynb", 8 | "provenance": [], 9 | "collapsed_sections": [], 10 | "toc_visible": true 11 | }, 12 | "kernelspec": { 13 | "display_name": "Python 3", 14 | "language": "python", 15 | "name": "python3" 16 | }, 17 | "language_info": { 18 | "codemirror_mode": { 19 | "name": "ipython", 20 | "version": 3 21 | }, 22 | "file_extension": ".py", 23 | "mimetype": "text/x-python", 24 | "name": "python", 25 | "nbconvert_exporter": "python", 26 | "pygments_lexer": "ipython3", 27 | "version": "3.7.6" 28 | } 29 | }, 30 | "cells": [ 31 | { 32 | "cell_type": "code", 33 | "metadata": { 34 | "colab_type": "code", 35 | "id": "zX4Kg8DUTKWO", 36 | "colab": {} 37 | }, 38 | "source": [ 39 | "#@title Licensed under the Apache License, Version 2.0 (the \"License\");\n", 40 | "# you may not use this file except in compliance with the License.\n", 41 | "# You may obtain a copy of the License at\n", 42 | "#\n", 43 | "# https://www.apache.org/licenses/LICENSE-2.0\n", 44 | "#\n", 45 | "# Unless required by applicable law or agreed to in writing, software\n", 46 | "# distributed under the License is distributed on an \"AS IS\" BASIS,\n", 47 | "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n", 48 | "# See the License for the specific language governing permissions and\n", 49 | "# limitations under the License." 50 | ], 51 | "execution_count": null, 52 | "outputs": [] 53 | }, 54 | { 55 | "cell_type": "markdown", 56 | "metadata": { 57 | "id": "_8w9IPRVpIdH", 58 | "colab_type": "text" 59 | }, 60 | "source": [ 61 | "\"Open" 62 | ] 63 | }, 64 | { 65 | "cell_type": "code", 66 | "metadata": { 67 | "colab_type": "code", 68 | "id": "jGwXGIXvFhXW", 69 | "colab": {} 70 | }, 71 | "source": [ 72 | "import numpy as np\n", 73 | "\n", 74 | "import json\n", 75 | "import tensorflow as tf\n", 76 | "\n", 77 | "from tensorflow.keras.preprocessing.text import Tokenizer\n", 78 | "from tensorflow.keras.preprocessing.sequence import pad_sequences\n", 79 | "\n", 80 | "!wget --no-check-certificate \\\n", 81 | " https://storage.googleapis.com/laurencemoroney-blog.appspot.com/sarcasm.json \\\n", 82 | " -O /tmp/sarcasm.json\n", 83 | "\n", 84 | "vocab_size = 1000\n", 85 | "embedding_dim = 16\n", 86 | "max_length = 120\n", 87 | "trunc_type='post'\n", 88 | "padding_type='post'\n", 89 | "oov_tok = \"\"\n", 90 | "training_size = 20000\n", 91 | "\n", 92 | "\n", 93 | "with open(\"/tmp/sarcasm.json\", 'r') as f:\n", 94 | " datastore = json.load(f)\n", 95 | "\n", 96 | "\n", 97 | "sentences = []\n", 98 | "labels = []\n", 99 | "urls = []\n", 100 | "for item in datastore:\n", 101 | " sentences.append(item['headline'])\n", 102 | " labels.append(item['is_sarcastic'])\n", 103 | "\n", 104 | "training_sentences = sentences[0:training_size]\n", 105 | "testing_sentences = sentences[training_size:]\n", 106 | "training_labels = labels[0:training_size]\n", 107 | "testing_labels = labels[training_size:]\n", 108 | "\n", 109 | "tokenizer = Tokenizer(num_words=vocab_size, oov_token=oov_tok)\n", 110 | "tokenizer.fit_on_texts(training_sentences)\n", 111 | "\n", 112 | "word_index = tokenizer.word_index\n", 113 | "\n", 114 | "training_sequences = tokenizer.texts_to_sequences(training_sentences)\n", 115 | "training_padded = pad_sequences(training_sequences, maxlen=max_length, padding=padding_type, truncating=trunc_type)\n", 116 | "\n", 117 | "testing_sequences = tokenizer.texts_to_sequences(testing_sentences)\n", 118 | "testing_padded = pad_sequences(testing_sequences, maxlen=max_length, padding=padding_type, truncating=trunc_type)\n", 119 | "\n", 120 | "model = tf.keras.Sequential([\n", 121 | " tf.keras.layers.Embedding(vocab_size, embedding_dim, input_length=max_length),\n", 122 | " tf.keras.layers.Bidirectional(tf.keras.layers.LSTM(32)),\n", 123 | " tf.keras.layers.Dense(24, activation='relu'),\n", 124 | " tf.keras.layers.Dense(1, activation='sigmoid')\n", 125 | "])\n", 126 | "model.compile(loss='binary_crossentropy',optimizer='adam',metrics=['accuracy'])\n", 127 | "model.summary()\n", 128 | "\n", 129 | "num_epochs = 50\n", 130 | "training_padded = np.array(training_padded)\n", 131 | "training_labels = np.array(training_labels)\n", 132 | "testing_padded = np.array(testing_padded)\n", 133 | "testing_labels = np.array(testing_labels)\n", 134 | "history = model.fit(training_padded, training_labels, epochs=num_epochs, validation_data=(testing_padded, testing_labels), verbose=1)\n" 135 | ], 136 | "execution_count": null, 137 | "outputs": [] 138 | }, 139 | { 140 | "cell_type": "code", 141 | "metadata": { 142 | "colab_type": "code", 143 | "id": "g9DC6dmLF8DC", 144 | "colab": {} 145 | }, 146 | "source": [ 147 | "import matplotlib.pyplot as plt\n", 148 | "\n", 149 | "\n", 150 | "def plot_graphs(history, string):\n", 151 | " plt.plot(history.history[string])\n", 152 | " plt.plot(history.history['val_'+string])\n", 153 | " plt.xlabel(\"Epochs\")\n", 154 | " plt.ylabel(string)\n", 155 | " plt.legend([string, 'val_'+string])\n", 156 | " plt.show()\n", 157 | "\n", 158 | "plot_graphs(history, 'accuracy')\n", 159 | "plot_graphs(history, 'loss')" 160 | ], 161 | "execution_count": null, 162 | "outputs": [] 163 | }, 164 | { 165 | "cell_type": "code", 166 | "metadata": { 167 | "colab_type": "code", 168 | "id": "7ZEZIUppGhdi", 169 | "colab": {} 170 | }, 171 | "source": [ 172 | "model.save(\"test.h5\")" 173 | ], 174 | "execution_count": null, 175 | "outputs": [] 176 | } 177 | ] 178 | } -------------------------------------------------------------------------------- /3.Natural Language Processing in TensorFlow/week3/Course_3_Week_3_Lesson_2c.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "nbformat": 4, 3 | "nbformat_minor": 0, 4 | "metadata": { 5 | "accelerator": "GPU", 6 | "colab": { 7 | "name": "Course 3 - Week 3 - Lesson 2c.ipynb", 8 | "provenance": [], 9 | "collapsed_sections": [], 10 | "toc_visible": true 11 | }, 12 | "kernelspec": { 13 | "display_name": "Python 3", 14 | "language": "python", 15 | "name": "python3" 16 | }, 17 | "language_info": { 18 | "codemirror_mode": { 19 | "name": "ipython", 20 | "version": 3 21 | }, 22 | "file_extension": ".py", 23 | "mimetype": "text/x-python", 24 | "name": "python", 25 | "nbconvert_exporter": "python", 26 | "pygments_lexer": "ipython3", 27 | "version": "3.7.6" 28 | } 29 | }, 30 | "cells": [ 31 | { 32 | "cell_type": "code", 33 | "metadata": { 34 | "colab_type": "code", 35 | "id": "zX4Kg8DUTKWO", 36 | "colab": {} 37 | }, 38 | "source": [ 39 | "#@title Licensed under the Apache License, Version 2.0 (the \"License\");\n", 40 | "# you may not use this file except in compliance with the License.\n", 41 | "# You may obtain a copy of the License at\n", 42 | "#\n", 43 | "# https://www.apache.org/licenses/LICENSE-2.0\n", 44 | "#\n", 45 | "# Unless required by applicable law or agreed to in writing, software\n", 46 | "# distributed under the License is distributed on an \"AS IS\" BASIS,\n", 47 | "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n", 48 | "# See the License for the specific language governing permissions and\n", 49 | "# limitations under the License." 50 | ], 51 | "execution_count": null, 52 | "outputs": [] 53 | }, 54 | { 55 | "cell_type": "markdown", 56 | "metadata": { 57 | "id": "3pxMw6kJpI58", 58 | "colab_type": "text" 59 | }, 60 | "source": [ 61 | "\"Open" 62 | ] 63 | }, 64 | { 65 | "cell_type": "code", 66 | "metadata": { 67 | "colab_type": "code", 68 | "id": "jGwXGIXvFhXW", 69 | "colab": {} 70 | }, 71 | "source": [ 72 | "import numpy as np\n", 73 | "\n", 74 | "import json\n", 75 | "import tensorflow as tf\n", 76 | "\n", 77 | "from tensorflow.keras.preprocessing.text import Tokenizer\n", 78 | "from tensorflow.keras.preprocessing.sequence import pad_sequences\n", 79 | "\n", 80 | "!wget --no-check-certificate \\\n", 81 | " https://storage.googleapis.com/laurencemoroney-blog.appspot.com/sarcasm.json \\\n", 82 | " -O /tmp/sarcasm.json\n", 83 | "\n", 84 | "vocab_size = 1000\n", 85 | "embedding_dim = 16\n", 86 | "max_length = 120\n", 87 | "trunc_type='post'\n", 88 | "padding_type='post'\n", 89 | "oov_tok = \"\"\n", 90 | "training_size = 20000\n", 91 | "\n", 92 | "\n", 93 | "with open(\"/tmp/sarcasm.json\", 'r') as f:\n", 94 | " datastore = json.load(f)\n", 95 | "\n", 96 | "\n", 97 | "sentences = []\n", 98 | "labels = []\n", 99 | "urls = []\n", 100 | "for item in datastore:\n", 101 | " sentences.append(item['headline'])\n", 102 | " labels.append(item['is_sarcastic'])\n", 103 | "\n", 104 | "training_sentences = sentences[0:training_size]\n", 105 | "testing_sentences = sentences[training_size:]\n", 106 | "training_labels = labels[0:training_size]\n", 107 | "testing_labels = labels[training_size:]\n", 108 | "\n", 109 | "tokenizer = Tokenizer(num_words=vocab_size, oov_token=oov_tok)\n", 110 | "tokenizer.fit_on_texts(training_sentences)\n", 111 | "\n", 112 | "word_index = tokenizer.word_index\n", 113 | "\n", 114 | "training_sequences = tokenizer.texts_to_sequences(training_sentences)\n", 115 | "training_padded = pad_sequences(training_sequences, maxlen=max_length, padding=padding_type, truncating=trunc_type)\n", 116 | "\n", 117 | "testing_sequences = tokenizer.texts_to_sequences(testing_sentences)\n", 118 | "testing_padded = pad_sequences(testing_sequences, maxlen=max_length, padding=padding_type, truncating=trunc_type)\n", 119 | "\n", 120 | "model = tf.keras.Sequential([\n", 121 | " tf.keras.layers.Embedding(vocab_size, embedding_dim, input_length=max_length),\n", 122 | " tf.keras.layers.Conv1D(128, 5, activation='relu'),\n", 123 | " tf.keras.layers.GlobalMaxPooling1D(),\n", 124 | " tf.keras.layers.Dense(24, activation='relu'),\n", 125 | " tf.keras.layers.Dense(1, activation='sigmoid')\n", 126 | "])\n", 127 | "model.compile(loss='binary_crossentropy',optimizer='adam',metrics=['accuracy'])\n", 128 | "model.summary()\n", 129 | "\n", 130 | "num_epochs = 50\n", 131 | "\n", 132 | "training_padded = np.array(training_padded)\n", 133 | "training_labels = np.array(training_labels)\n", 134 | "testing_padded = np.array(testing_padded)\n", 135 | "testing_labels = np.array(testing_labels)\n", 136 | "\n", 137 | "history = model.fit(training_padded, training_labels, epochs=num_epochs, validation_data=(testing_padded, testing_labels), verbose=1)\n" 138 | ], 139 | "execution_count": null, 140 | "outputs": [] 141 | }, 142 | { 143 | "cell_type": "code", 144 | "metadata": { 145 | "colab_type": "code", 146 | "id": "g9DC6dmLF8DC", 147 | "colab": {} 148 | }, 149 | "source": [ 150 | "import matplotlib.pyplot as plt\n", 151 | "\n", 152 | "\n", 153 | "def plot_graphs(history, string):\n", 154 | " plt.plot(history.history[string])\n", 155 | " plt.plot(history.history['val_'+string])\n", 156 | " plt.xlabel(\"Epochs\")\n", 157 | " plt.ylabel(string)\n", 158 | " plt.legend([string, 'val_'+string])\n", 159 | " plt.show()\n", 160 | "\n", 161 | "plot_graphs(history, 'accuracy')\n", 162 | "plot_graphs(history, 'loss')" 163 | ], 164 | "execution_count": null, 165 | "outputs": [] 166 | }, 167 | { 168 | "cell_type": "code", 169 | "metadata": { 170 | "colab_type": "code", 171 | "id": "7ZEZIUppGhdi", 172 | "colab": {} 173 | }, 174 | "source": [ 175 | "model.save(\"test.h5\")" 176 | ], 177 | "execution_count": null, 178 | "outputs": [] 179 | } 180 | ] 181 | } -------------------------------------------------------------------------------- /3.Natural Language Processing in TensorFlow/week3/Course_3_Week_3_Lesson_2d.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "nbformat": 4, 3 | "nbformat_minor": 0, 4 | "metadata": { 5 | "accelerator": "GPU", 6 | "colab": { 7 | "name": "Course 3 - Week 3 - Lesson 2d.ipynb", 8 | "provenance": [], 9 | "collapsed_sections": [], 10 | "toc_visible": true 11 | }, 12 | "kernelspec": { 13 | "display_name": "Python 3", 14 | "name": "python3" 15 | } 16 | }, 17 | "cells": [ 18 | { 19 | "cell_type": "code", 20 | "metadata": { 21 | "id": "zX4Kg8DUTKWO", 22 | "colab_type": "code", 23 | "colab": {} 24 | }, 25 | "source": [ 26 | "#@title Licensed under the Apache License, Version 2.0 (the \"License\");\n", 27 | "# you may not use this file except in compliance with the License.\n", 28 | "# You may obtain a copy of the License at\n", 29 | "#\n", 30 | "# https://www.apache.org/licenses/LICENSE-2.0\n", 31 | "#\n", 32 | "# Unless required by applicable law or agreed to in writing, software\n", 33 | "# distributed under the License is distributed on an \"AS IS\" BASIS,\n", 34 | "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n", 35 | "# See the License for the specific language governing permissions and\n", 36 | "# limitations under the License." 37 | ], 38 | "execution_count": null, 39 | "outputs": [] 40 | }, 41 | { 42 | "cell_type": "code", 43 | "metadata": { 44 | "colab_type": "code", 45 | "id": "P-AhVYeBWgQ3", 46 | "colab": {} 47 | }, 48 | "source": [ 49 | "import tensorflow as tf\n", 50 | "print(tf.__version__)\n", 51 | "\n", 52 | "# !pip install -q tensorflow-datasets" 53 | ], 54 | "execution_count": null, 55 | "outputs": [] 56 | }, 57 | { 58 | "cell_type": "code", 59 | "metadata": { 60 | "colab_type": "code", 61 | "id": "_IoM4VFxWpMR", 62 | "colab": {} 63 | }, 64 | "source": [ 65 | "import tensorflow_datasets as tfds\n", 66 | "imdb, info = tfds.load(\"imdb_reviews\", with_info=True, as_supervised=True)\n" 67 | ], 68 | "execution_count": null, 69 | "outputs": [] 70 | }, 71 | { 72 | "cell_type": "code", 73 | "metadata": { 74 | "colab_type": "code", 75 | "id": "wHQ2Ko0zl7M4", 76 | "colab": {} 77 | }, 78 | "source": [ 79 | "import numpy as np\n", 80 | "\n", 81 | "train_data, test_data = imdb['train'], imdb['test']\n", 82 | "\n", 83 | "training_sentences = []\n", 84 | "training_labels = []\n", 85 | "\n", 86 | "testing_sentences = []\n", 87 | "testing_labels = []\n", 88 | "\n", 89 | "# str(s.tonumpy()) is needed in Python3 instead of just s.numpy()\n", 90 | "for s,l in train_data:\n", 91 | " training_sentences.append(str(s.numpy()))\n", 92 | " training_labels.append(l.numpy())\n", 93 | " \n", 94 | "for s,l in test_data:\n", 95 | " testing_sentences.append(str(s.numpy()))\n", 96 | " testing_labels.append(l.numpy())\n", 97 | " \n", 98 | "training_labels_final = np.array(training_labels)\n", 99 | "testing_labels_final = np.array(testing_labels)\n" 100 | ], 101 | "execution_count": null, 102 | "outputs": [] 103 | }, 104 | { 105 | "cell_type": "code", 106 | "metadata": { 107 | "colab_type": "code", 108 | "id": "7n15yyMdmoH1", 109 | "colab": {} 110 | }, 111 | "source": [ 112 | "vocab_size = 10000\n", 113 | "embedding_dim = 16\n", 114 | "max_length = 120\n", 115 | "trunc_type='post'\n", 116 | "oov_tok = \"\"\n", 117 | "\n", 118 | "\n", 119 | "from tensorflow.keras.preprocessing.text import Tokenizer\n", 120 | "from tensorflow.keras.preprocessing.sequence import pad_sequences\n", 121 | "\n", 122 | "tokenizer = Tokenizer(num_words = vocab_size, oov_token=oov_tok)\n", 123 | "tokenizer.fit_on_texts(training_sentences)\n", 124 | "word_index = tokenizer.word_index\n", 125 | "sequences = tokenizer.texts_to_sequences(training_sentences)\n", 126 | "padded = pad_sequences(sequences,maxlen=max_length, truncating=trunc_type)\n", 127 | "\n", 128 | "testing_sequences = tokenizer.texts_to_sequences(testing_sentences)\n", 129 | "testing_padded = pad_sequences(testing_sequences,maxlen=max_length)\n" 130 | ], 131 | "execution_count": null, 132 | "outputs": [] 133 | }, 134 | { 135 | "cell_type": "code", 136 | "metadata": { 137 | "colab_type": "code", 138 | "id": "9axf0uIXVMhO", 139 | "colab": {} 140 | }, 141 | "source": [ 142 | "reverse_word_index = dict([(value, key) for (key, value) in word_index.items()])\n", 143 | "\n", 144 | "def decode_review(text):\n", 145 | " return ' '.join([reverse_word_index.get(i, '?') for i in text])\n", 146 | "\n", 147 | "print(decode_review(padded[1]))\n", 148 | "print(training_sentences[1])" 149 | ], 150 | "execution_count": null, 151 | "outputs": [] 152 | }, 153 | { 154 | "cell_type": "code", 155 | "metadata": { 156 | "colab_type": "code", 157 | "id": "5NEpdhb8AxID", 158 | "colab": {} 159 | }, 160 | "source": [ 161 | "model = tf.keras.Sequential([\n", 162 | " tf.keras.layers.Embedding(vocab_size, embedding_dim, input_length=max_length),\n", 163 | " tf.keras.layers.Bidirectional(tf.keras.layers.GRU(32)),\n", 164 | " tf.keras.layers.Dense(6, activation='relu'),\n", 165 | " tf.keras.layers.Dense(1, activation='sigmoid')\n", 166 | "])\n", 167 | "model.compile(loss='binary_crossentropy',optimizer='adam',metrics=['accuracy'])\n", 168 | "model.summary()\n" 169 | ], 170 | "execution_count": null, 171 | "outputs": [] 172 | }, 173 | { 174 | "cell_type": "code", 175 | "metadata": { 176 | "colab_type": "code", 177 | "id": "V5LLrXC-uNX6", 178 | "colab": {} 179 | }, 180 | "source": [ 181 | "num_epochs = 50\n", 182 | "history = model.fit(padded, training_labels_final, epochs=num_epochs, validation_data=(testing_padded, testing_labels_final))" 183 | ], 184 | "execution_count": null, 185 | "outputs": [] 186 | }, 187 | { 188 | "cell_type": "code", 189 | "metadata": { 190 | "colab_type": "code", 191 | "id": "nHGYuU4jPYaj", 192 | "colab": {} 193 | }, 194 | "source": [ 195 | "import matplotlib.pyplot as plt\n", 196 | "\n", 197 | "\n", 198 | "def plot_graphs(history, string):\n", 199 | " plt.plot(history.history[string])\n", 200 | " plt.plot(history.history['val_'+string])\n", 201 | " plt.xlabel(\"Epochs\")\n", 202 | " plt.ylabel(string)\n", 203 | " plt.legend([string, 'val_'+string])\n", 204 | " plt.show()\n", 205 | "\n", 206 | "plot_graphs(history, 'accuracy')\n", 207 | "plot_graphs(history, 'loss')" 208 | ], 209 | "execution_count": null, 210 | "outputs": [] 211 | }, 212 | { 213 | "cell_type": "code", 214 | "metadata": { 215 | "colab_type": "code", 216 | "id": "wSualgGPPK0S", 217 | "colab": {} 218 | }, 219 | "source": [ 220 | "# Model Definition with LSTM\n", 221 | "model = tf.keras.Sequential([\n", 222 | " tf.keras.layers.Embedding(vocab_size, embedding_dim, input_length=max_length),\n", 223 | " tf.keras.layers.Bidirectional(tf.keras.layers.LSTM(32)),\n", 224 | " tf.keras.layers.Dense(6, activation='relu'),\n", 225 | " tf.keras.layers.Dense(1, activation='sigmoid')\n", 226 | "])\n", 227 | "model.compile(loss='binary_crossentropy',optimizer='adam',metrics=['accuracy'])\n", 228 | "model.summary()\n" 229 | ], 230 | "execution_count": null, 231 | "outputs": [] 232 | }, 233 | { 234 | "cell_type": "code", 235 | "metadata": { 236 | "colab_type": "code", 237 | "id": "K_Jc7cY3Qxke", 238 | "colab": {} 239 | }, 240 | "source": [ 241 | "# Model Definition with Conv1D\n", 242 | "model = tf.keras.Sequential([\n", 243 | " tf.keras.layers.Embedding(vocab_size, embedding_dim, input_length=max_length),\n", 244 | " tf.keras.layers.Conv1D(128, 5, activation='relu'),\n", 245 | " tf.keras.layers.GlobalAveragePooling1D(),\n", 246 | " tf.keras.layers.Dense(6, activation='relu'),\n", 247 | " tf.keras.layers.Dense(1, activation='sigmoid')\n", 248 | "])\n", 249 | "model.compile(loss='binary_crossentropy',optimizer='adam',metrics=['accuracy'])\n", 250 | "model.summary()\n" 251 | ], 252 | "execution_count": null, 253 | "outputs": [] 254 | } 255 | ] 256 | } -------------------------------------------------------------------------------- /3.Natural Language Processing in TensorFlow/week4/Course_3_Week_4_Lesson_1_Notebook.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "nbformat": 4, 3 | "nbformat_minor": 0, 4 | "metadata": { 5 | "accelerator": "GPU", 6 | "colab": { 7 | "name": "Course 3 - Week 4 - Lesson 1 - Notebook.ipynb", 8 | "provenance": [], 9 | "collapsed_sections": [], 10 | "toc_visible": true 11 | }, 12 | "kernelspec": { 13 | "display_name": "Python 3", 14 | "name": "python3" 15 | } 16 | }, 17 | "cells": [ 18 | { 19 | "cell_type": "code", 20 | "metadata": { 21 | "id": "zX4Kg8DUTKWO", 22 | "colab_type": "code", 23 | "colab": {} 24 | }, 25 | "source": [ 26 | "#@title Licensed under the Apache License, Version 2.0 (the \"License\");\n", 27 | "# you may not use this file except in compliance with the License.\n", 28 | "# You may obtain a copy of the License at\n", 29 | "#\n", 30 | "# https://www.apache.org/licenses/LICENSE-2.0\n", 31 | "#\n", 32 | "# Unless required by applicable law or agreed to in writing, software\n", 33 | "# distributed under the License is distributed on an \"AS IS\" BASIS,\n", 34 | "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n", 35 | "# See the License for the specific language governing permissions and\n", 36 | "# limitations under the License." 37 | ], 38 | "execution_count": null, 39 | "outputs": [] 40 | }, 41 | { 42 | "cell_type": "code", 43 | "metadata": { 44 | "colab_type": "code", 45 | "id": "BOwsuGQQY9OL", 46 | "colab": {} 47 | }, 48 | "source": [ 49 | "import tensorflow as tf\n", 50 | "\n", 51 | "from tensorflow.keras.preprocessing.sequence import pad_sequences\n", 52 | "from tensorflow.keras.layers import Embedding, LSTM, Dense, Bidirectional\n", 53 | "from tensorflow.keras.preprocessing.text import Tokenizer\n", 54 | "from tensorflow.keras.models import Sequential\n", 55 | "from tensorflow.keras.optimizers import Adam\n", 56 | "import numpy as np " 57 | ], 58 | "execution_count": null, 59 | "outputs": [] 60 | }, 61 | { 62 | "cell_type": "code", 63 | "metadata": { 64 | "colab_type": "code", 65 | "id": "PRnDnCW-Z7qv", 66 | "colab": {} 67 | }, 68 | "source": [ 69 | "tokenizer = Tokenizer()\n", 70 | "\n", 71 | "data=\"In the town of Athy one Jeremy Lanigan \\n Battered away til he hadnt a pound. \\nHis father died and made him a man again \\n Left him a farm and ten acres of ground. \\nHe gave a grand party for friends and relations \\nWho didnt forget him when come to the wall, \\nAnd if youll but listen Ill make your eyes glisten \\nOf the rows and the ructions of Lanigans Ball. \\nMyself to be sure got free invitation, \\nFor all the nice girls and boys I might ask, \\nAnd just in a minute both friends and relations \\nWere dancing round merry as bees round a cask. \\nJudy ODaly, that nice little milliner, \\nShe tipped me a wink for to give her a call, \\nAnd I soon arrived with Peggy McGilligan \\nJust in time for Lanigans Ball. \\nThere were lashings of punch and wine for the ladies, \\nPotatoes and cakes; there was bacon and tea, \\nThere were the Nolans, Dolans, OGradys \\nCourting the girls and dancing away. \\nSongs they went round as plenty as water, \\nThe harp that once sounded in Taras old hall,\\nSweet Nelly Gray and The Rat Catchers Daughter,\\nAll singing together at Lanigans Ball. \\nThey were doing all kinds of nonsensical polkas \\nAll round the room in a whirligig. \\nJulia and I, we banished their nonsense \\nAnd tipped them the twist of a reel and a jig. \\nAch mavrone, how the girls got all mad at me \\nDanced til youd think the ceiling would fall. \\nFor I spent three weeks at Brooks Academy \\nLearning new steps for Lanigans Ball. \\nThree long weeks I spent up in Dublin, \\nThree long weeks to learn nothing at all,\\n Three long weeks I spent up in Dublin, \\nLearning new steps for Lanigans Ball. \\nShe stepped out and I stepped in again, \\nI stepped out and she stepped in again, \\nShe stepped out and I stepped in again, \\nLearning new steps for Lanigans Ball. \\nBoys were all merry and the girls they were hearty \\nAnd danced all around in couples and groups, \\nTil an accident happened, young Terrance McCarthy \\nPut his right leg through miss Finnertys hoops. \\nPoor creature fainted and cried Meelia murther, \\nCalled for her brothers and gathered them all. \\nCarmody swore that hed go no further \\nTil he had satisfaction at Lanigans Ball. \\nIn the midst of the row miss Kerrigan fainted, \\nHer cheeks at the same time as red as a rose. \\nSome of the lads declared she was painted, \\nShe took a small drop too much, I suppose. \\nHer sweetheart, Ned Morgan, so powerful and able, \\nWhen he saw his fair colleen stretched out by the wall, \\nTore the left leg from under the table \\nAnd smashed all the Chaneys at Lanigans Ball. \\nBoys, oh boys, twas then there were runctions. \\nMyself got a lick from big Phelim McHugh. \\nI soon replied to his introduction \\nAnd kicked up a terrible hullabaloo. \\nOld Casey, the piper, was near being strangled. \\nThey squeezed up his pipes, bellows, chanters and all. \\nThe girls, in their ribbons, they got all entangled \\nAnd that put an end to Lanigans Ball.\"\n", 72 | "\n", 73 | "corpus = data.lower().split(\"\\n\")\n", 74 | "\n", 75 | "tokenizer.fit_on_texts(corpus)\n", 76 | "total_words = len(tokenizer.word_index) + 1\n", 77 | "\n", 78 | "print(tokenizer.word_index)\n", 79 | "print(total_words)\n" 80 | ], 81 | "execution_count": null, 82 | "outputs": [] 83 | }, 84 | { 85 | "cell_type": "code", 86 | "metadata": { 87 | "colab_type": "code", 88 | "id": "soPGVheskaQP", 89 | "colab": {} 90 | }, 91 | "source": [ 92 | "input_sequences = []\n", 93 | "for line in corpus:\n", 94 | "\ttoken_list = tokenizer.texts_to_sequences([line])[0]\n", 95 | "\tfor i in range(1, len(token_list)):\n", 96 | "\t\tn_gram_sequence = token_list[:i+1]\n", 97 | "\t\tinput_sequences.append(n_gram_sequence)\n", 98 | "\n", 99 | "# pad sequences \n", 100 | "max_sequence_len = max([len(x) for x in input_sequences])\n", 101 | "input_sequences = np.array(pad_sequences(input_sequences, maxlen=max_sequence_len, padding='pre'))\n", 102 | "\n", 103 | "# create predictors and label\n", 104 | "xs, labels = input_sequences[:,:-1],input_sequences[:,-1]\n", 105 | "\n", 106 | "ys = tf.keras.utils.to_categorical(labels, num_classes=total_words)" 107 | ], 108 | "execution_count": null, 109 | "outputs": [] 110 | }, 111 | { 112 | "cell_type": "code", 113 | "metadata": { 114 | "colab_type": "code", 115 | "id": "pJtwVB2NbOAP", 116 | "colab": {} 117 | }, 118 | "source": [ 119 | "print(tokenizer.word_index['in'])\n", 120 | "print(tokenizer.word_index['the'])\n", 121 | "print(tokenizer.word_index['town'])\n", 122 | "print(tokenizer.word_index['of'])\n", 123 | "print(tokenizer.word_index['athy'])\n", 124 | "print(tokenizer.word_index['one'])\n", 125 | "print(tokenizer.word_index['jeremy'])\n", 126 | "print(tokenizer.word_index['lanigan'])" 127 | ], 128 | "execution_count": null, 129 | "outputs": [] 130 | }, 131 | { 132 | "cell_type": "code", 133 | "metadata": { 134 | "colab_type": "code", 135 | "id": "49Cv68JOakwv", 136 | "colab": {} 137 | }, 138 | "source": [ 139 | "print(xs[6])" 140 | ], 141 | "execution_count": null, 142 | "outputs": [] 143 | }, 144 | { 145 | "cell_type": "code", 146 | "metadata": { 147 | "colab_type": "code", 148 | "id": "iY-jwvfgbEF8", 149 | "colab": {} 150 | }, 151 | "source": [ 152 | "print(ys[6])" 153 | ], 154 | "execution_count": null, 155 | "outputs": [] 156 | }, 157 | { 158 | "cell_type": "code", 159 | "metadata": { 160 | "colab_type": "code", 161 | "id": "wtzlUMYadhKt", 162 | "colab": {} 163 | }, 164 | "source": [ 165 | "print(xs[5])\n", 166 | "print(ys[5])" 167 | ], 168 | "execution_count": null, 169 | "outputs": [] 170 | }, 171 | { 172 | "cell_type": "code", 173 | "metadata": { 174 | "colab_type": "code", 175 | "id": "H4myRpB1c4Gg", 176 | "colab": {} 177 | }, 178 | "source": [ 179 | "print(tokenizer.word_index)" 180 | ], 181 | "execution_count": null, 182 | "outputs": [] 183 | }, 184 | { 185 | "cell_type": "code", 186 | "metadata": { 187 | "colab_type": "code", 188 | "id": "w9vH8Y59ajYL", 189 | "colab": {} 190 | }, 191 | "source": [ 192 | " model = Sequential()\n", 193 | " model.add(Embedding(total_words, 64, input_length=max_sequence_len-1))\n", 194 | " model.add(Bidirectional(LSTM(20)))\n", 195 | " model.add(Dense(total_words, activation='softmax'))\n", 196 | " model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])\n", 197 | " history = model.fit(xs, ys, epochs=500, verbose=1)\n" 198 | ], 199 | "execution_count": null, 200 | "outputs": [] 201 | }, 202 | { 203 | "cell_type": "code", 204 | "metadata": { 205 | "colab_type": "code", 206 | "id": "3YXGelKThoTT", 207 | "colab": {} 208 | }, 209 | "source": [ 210 | "import matplotlib.pyplot as plt\n", 211 | "\n", 212 | "\n", 213 | "def plot_graphs(history, string):\n", 214 | " plt.plot(history.history[string])\n", 215 | " plt.xlabel(\"Epochs\")\n", 216 | " plt.ylabel(string)\n", 217 | " plt.show()" 218 | ], 219 | "execution_count": null, 220 | "outputs": [] 221 | }, 222 | { 223 | "cell_type": "code", 224 | "metadata": { 225 | "colab_type": "code", 226 | "id": "poeprYK8h-c7", 227 | "colab": {} 228 | }, 229 | "source": [ 230 | "plot_graphs(history, 'accuracy')\n" 231 | ], 232 | "execution_count": null, 233 | "outputs": [] 234 | }, 235 | { 236 | "cell_type": "code", 237 | "metadata": { 238 | "colab_type": "code", 239 | "id": "6Vc6PHgxa6Hm", 240 | "colab": {} 241 | }, 242 | "source": [ 243 | "seed_text = \"Laurence went to dublin\"\n", 244 | "next_words = 100\n", 245 | " \n", 246 | "for _ in range(next_words):\n", 247 | "\ttoken_list = tokenizer.texts_to_sequences([seed_text])[0]\n", 248 | "\ttoken_list = pad_sequences([token_list], maxlen=max_sequence_len-1, padding='pre')\n", 249 | "\tpredicted = model.predict_classes(token_list, verbose=0)\n", 250 | "\toutput_word = \"\"\n", 251 | "\tfor word, index in tokenizer.word_index.items():\n", 252 | "\t\tif index == predicted:\n", 253 | "\t\t\toutput_word = word\n", 254 | "\t\t\tbreak\n", 255 | "\tseed_text += \" \" + output_word\n", 256 | "print(seed_text)" 257 | ], 258 | "execution_count": null, 259 | "outputs": [] 260 | } 261 | ] 262 | } -------------------------------------------------------------------------------- /3.Natural Language Processing in TensorFlow/week4/NLP_Week4_Exercise_Shakespeare_Answer.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "nbformat": 4, 3 | "nbformat_minor": 0, 4 | "metadata": { 5 | "accelerator": "GPU", 6 | "colab": { 7 | "name": "NLP_Week4_Exercise_Shakespeare_Answer.ipynb", 8 | "provenance": [], 9 | "toc_visible": true 10 | }, 11 | "kernelspec": { 12 | "display_name": "Python 3", 13 | "name": "python3" 14 | } 15 | }, 16 | "cells": [ 17 | { 18 | "cell_type": "code", 19 | "metadata": { 20 | "id": "zX4Kg8DUTKWO", 21 | "colab_type": "code", 22 | "colab": {} 23 | }, 24 | "source": [ 25 | "#@title Licensed under the Apache License, Version 2.0 (the \"License\");\n", 26 | "# you may not use this file except in compliance with the License.\n", 27 | "# You may obtain a copy of the License at\n", 28 | "#\n", 29 | "# https://www.apache.org/licenses/LICENSE-2.0\n", 30 | "#\n", 31 | "# Unless required by applicable law or agreed to in writing, software\n", 32 | "# distributed under the License is distributed on an \"AS IS\" BASIS,\n", 33 | "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n", 34 | "# See the License for the specific language governing permissions and\n", 35 | "# limitations under the License." 36 | ], 37 | "execution_count": null, 38 | "outputs": [] 39 | }, 40 | { 41 | "cell_type": "code", 42 | "metadata": { 43 | "colab_type": "code", 44 | "id": "BOwsuGQQY9OL", 45 | "colab": {} 46 | }, 47 | "source": [ 48 | "from tensorflow.keras.preprocessing.sequence import pad_sequences\n", 49 | "from tensorflow.keras.layers import Embedding, LSTM, Dense, Dropout, Bidirectional\n", 50 | "from tensorflow.keras.preprocessing.text import Tokenizer\n", 51 | "from tensorflow.keras.models import Sequential\n", 52 | "from tensorflow.keras.optimizers import Adam\n", 53 | "from tensorflow.keras import regularizers\n", 54 | "import tensorflow.keras.utils as ku \n", 55 | "import numpy as np " 56 | ], 57 | "execution_count": null, 58 | "outputs": [] 59 | }, 60 | { 61 | "cell_type": "code", 62 | "metadata": { 63 | "colab_type": "code", 64 | "id": "PRnDnCW-Z7qv", 65 | "colab": {} 66 | }, 67 | "source": [ 68 | "tokenizer = Tokenizer()\n", 69 | "!wget --no-check-certificate \\\n", 70 | " https://storage.googleapis.com/laurencemoroney-blog.appspot.com/sonnets.txt \\\n", 71 | " -O /tmp/sonnets.txt\n", 72 | "data = open('/tmp/sonnets.txt').read()\n", 73 | "\n", 74 | "corpus = data.lower().split(\"\\n\")\n", 75 | "\n", 76 | "\n", 77 | "tokenizer.fit_on_texts(corpus)\n", 78 | "total_words = len(tokenizer.word_index) + 1\n", 79 | "\n", 80 | "# create input sequences using list of tokens\n", 81 | "input_sequences = []\n", 82 | "for line in corpus:\n", 83 | "\ttoken_list = tokenizer.texts_to_sequences([line])[0]\n", 84 | "\tfor i in range(1, len(token_list)):\n", 85 | "\t\tn_gram_sequence = token_list[:i+1]\n", 86 | "\t\tinput_sequences.append(n_gram_sequence)\n", 87 | "\n", 88 | "\n", 89 | "# pad sequences \n", 90 | "max_sequence_len = max([len(x) for x in input_sequences])\n", 91 | "input_sequences = np.array(pad_sequences(input_sequences, maxlen=max_sequence_len, padding='pre'))\n", 92 | "\n", 93 | "# create predictors and label\n", 94 | "predictors, label = input_sequences[:,:-1],input_sequences[:,-1]\n", 95 | "\n", 96 | "label = ku.to_categorical(label, num_classes=total_words)" 97 | ], 98 | "execution_count": null, 99 | "outputs": [] 100 | }, 101 | { 102 | "cell_type": "code", 103 | "metadata": { 104 | "colab_type": "code", 105 | "id": "w9vH8Y59ajYL", 106 | "colab": {} 107 | }, 108 | "source": [ 109 | "model = Sequential()\n", 110 | "model.add(Embedding(total_words, 100, input_length=max_sequence_len-1))\n", 111 | "model.add(Bidirectional(LSTM(150, return_sequences = True)))\n", 112 | "model.add(Dropout(0.2))\n", 113 | "model.add(LSTM(100))\n", 114 | "model.add(Dense(total_words/2, activation='relu', kernel_regularizer=regularizers.l2(0.01)))\n", 115 | "model.add(Dense(total_words, activation='softmax'))\n", 116 | "model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])\n", 117 | "print(model.summary())\n" 118 | ], 119 | "execution_count": null, 120 | "outputs": [] 121 | }, 122 | { 123 | "cell_type": "code", 124 | "metadata": { 125 | "colab_type": "code", 126 | "id": "AIg2f1HBxqof", 127 | "colab": {} 128 | }, 129 | "source": [ 130 | " history = model.fit(predictors, label, epochs=100, verbose=1)" 131 | ], 132 | "execution_count": null, 133 | "outputs": [] 134 | }, 135 | { 136 | "cell_type": "code", 137 | "metadata": { 138 | "colab_type": "code", 139 | "id": "1fXTEO3GJ282", 140 | "colab": {} 141 | }, 142 | "source": [ 143 | "import matplotlib.pyplot as plt\n", 144 | "acc = history.history['accuracy']\n", 145 | "loss = history.history['loss']\n", 146 | "\n", 147 | "epochs = range(len(acc))\n", 148 | "\n", 149 | "plt.plot(epochs, acc, 'b', label='Training accuracy')\n", 150 | "plt.title('Training accuracy')\n", 151 | "\n", 152 | "plt.figure()\n", 153 | "\n", 154 | "plt.plot(epochs, loss, 'b', label='Training Loss')\n", 155 | "plt.title('Training loss')\n", 156 | "plt.legend()\n", 157 | "\n", 158 | "plt.show()" 159 | ], 160 | "execution_count": null, 161 | "outputs": [] 162 | }, 163 | { 164 | "cell_type": "code", 165 | "metadata": { 166 | "colab_type": "code", 167 | "id": "6Vc6PHgxa6Hm", 168 | "colab": {} 169 | }, 170 | "source": [ 171 | "seed_text = \"Help me Obi Wan Kenobi, you're my only hope\"\n", 172 | "next_words = 100\n", 173 | " \n", 174 | "for _ in range(next_words):\n", 175 | "\ttoken_list = tokenizer.texts_to_sequences([seed_text])[0]\n", 176 | "\ttoken_list = pad_sequences([token_list], maxlen=max_sequence_len-1, padding='pre')\n", 177 | "\tpredicted = model.predict_classes(token_list, verbose=0)\n", 178 | "\toutput_word = \"\"\n", 179 | "\tfor word, index in tokenizer.word_index.items():\n", 180 | "\t\tif index == predicted:\n", 181 | "\t\t\toutput_word = word\n", 182 | "\t\t\tbreak\n", 183 | "\tseed_text += \" \" + output_word\n", 184 | "print(seed_text)" 185 | ], 186 | "execution_count": null, 187 | "outputs": [] 188 | } 189 | ] 190 | } -------------------------------------------------------------------------------- /4.Sequence, Time Series and Prediction/.DS_Store: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/catwang42/TensorFlow-developer/2169b0b71c3038748ae8c37ed0ceeb8f413ea4b2/4.Sequence, Time Series and Prediction/.DS_Store -------------------------------------------------------------------------------- /4.Sequence, Time Series and Prediction/week1/Week_1_Exercise_Answer.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "nbformat": 4, 3 | "nbformat_minor": 0, 4 | "metadata": { 5 | "accelerator": "GPU", 6 | "colab": { 7 | "name": "Week 1 Exercise Answer.ipynb", 8 | "provenance": [], 9 | "collapsed_sections": [], 10 | "toc_visible": true 11 | }, 12 | "kernelspec": { 13 | "display_name": "Python 3", 14 | "name": "python3" 15 | } 16 | }, 17 | "cells": [ 18 | { 19 | "cell_type": "code", 20 | "metadata": { 21 | "id": "zX4Kg8DUTKWO", 22 | "colab_type": "code", 23 | "colab": {} 24 | }, 25 | "source": [ 26 | "#@title Licensed under the Apache License, Version 2.0 (the \"License\");\n", 27 | "# you may not use this file except in compliance with the License.\n", 28 | "# You may obtain a copy of the License at\n", 29 | "#\n", 30 | "# https://www.apache.org/licenses/LICENSE-2.0\n", 31 | "#\n", 32 | "# Unless required by applicable law or agreed to in writing, software\n", 33 | "# distributed under the License is distributed on an \"AS IS\" BASIS,\n", 34 | "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n", 35 | "# See the License for the specific language governing permissions and\n", 36 | "# limitations under the License." 37 | ], 38 | "execution_count": null, 39 | "outputs": [] 40 | }, 41 | { 42 | "cell_type": "code", 43 | "metadata": { 44 | "colab_type": "code", 45 | "id": "t9HrvPfrSlzS", 46 | "colab": {} 47 | }, 48 | "source": [ 49 | "import tensorflow as tf\n", 50 | "print(tf.__version__)\n" 51 | ], 52 | "execution_count": null, 53 | "outputs": [] 54 | }, 55 | { 56 | "cell_type": "code", 57 | "metadata": { 58 | "colab_type": "code", 59 | "id": "gqWabzlJ63nL", 60 | "colab": {} 61 | }, 62 | "source": [ 63 | "import numpy as np\n", 64 | "import matplotlib.pyplot as plt\n", 65 | "import tensorflow as tf\n", 66 | "from tensorflow import keras\n", 67 | "\n", 68 | "def plot_series(time, series, format=\"-\", start=0, end=None):\n", 69 | " plt.plot(time[start:end], series[start:end], format)\n", 70 | " plt.xlabel(\"Time\")\n", 71 | " plt.ylabel(\"Value\")\n", 72 | " plt.grid(True)\n", 73 | "\n", 74 | "def trend(time, slope=0):\n", 75 | " return slope * time\n", 76 | "\n", 77 | "def seasonal_pattern(season_time):\n", 78 | " \"\"\"Just an arbitrary pattern, you can change it if you wish\"\"\"\n", 79 | " return np.where(season_time < 0.1,\n", 80 | " np.cos(season_time * 7 * np.pi),\n", 81 | " 1 / np.exp(5 * season_time))\n", 82 | "\n", 83 | "def seasonality(time, period, amplitude=1, phase=0):\n", 84 | " \"\"\"Repeats the same pattern at each period\"\"\"\n", 85 | " season_time = ((time + phase) % period) / period\n", 86 | " return amplitude * seasonal_pattern(season_time)\n", 87 | "\n", 88 | "def noise(time, noise_level=1, seed=None):\n", 89 | " rnd = np.random.RandomState(seed)\n", 90 | " return rnd.randn(len(time)) * noise_level\n", 91 | "\n", 92 | "time = np.arange(4 * 365 + 1, dtype=\"float32\")\n", 93 | "baseline = 10\n", 94 | "series = trend(time, 0.1) \n", 95 | "baseline = 10\n", 96 | "amplitude = 40\n", 97 | "slope = 0.01\n", 98 | "noise_level = 2\n", 99 | "\n", 100 | "# Create the series\n", 101 | "series = baseline + trend(time, slope) + seasonality(time, period=365, amplitude=amplitude)\n", 102 | "# Update with noise\n", 103 | "series += noise(time, noise_level, seed=42)\n", 104 | "\n", 105 | "plt.figure(figsize=(10, 6))\n", 106 | "plot_series(time, series)\n", 107 | "plt.show()" 108 | ], 109 | "execution_count": null, 110 | "outputs": [] 111 | }, 112 | { 113 | "cell_type": "markdown", 114 | "metadata": { 115 | "colab_type": "text", 116 | "id": "UfdyqJJ1VZVu" 117 | }, 118 | "source": [ 119 | "Now that we have the time series, let's split it so we can start forecasting" 120 | ] 121 | }, 122 | { 123 | "cell_type": "code", 124 | "metadata": { 125 | "colab_type": "code", 126 | "id": "_w0eKap5uFNP", 127 | "colab": {} 128 | }, 129 | "source": [ 130 | "split_time = 1100\n", 131 | "time_train = time[:split_time]\n", 132 | "x_train = series[:split_time]\n", 133 | "time_valid = time[split_time:]\n", 134 | "x_valid = series[split_time:]\n", 135 | "plt.figure(figsize=(10, 6))\n", 136 | "plot_series(time_train, x_train)\n", 137 | "plt.show()\n", 138 | "\n", 139 | "plt.figure(figsize=(10, 6))\n", 140 | "plot_series(time_valid, x_valid)\n", 141 | "plt.show()" 142 | ], 143 | "execution_count": null, 144 | "outputs": [] 145 | }, 146 | { 147 | "cell_type": "markdown", 148 | "metadata": { 149 | "colab_type": "text", 150 | "id": "bjD8ncEZbjEW" 151 | }, 152 | "source": [ 153 | "# Naive Forecast" 154 | ] 155 | }, 156 | { 157 | "cell_type": "code", 158 | "metadata": { 159 | "colab_type": "code", 160 | "id": "Pj_-uCeYxcAb", 161 | "colab": {} 162 | }, 163 | "source": [ 164 | "naive_forecast = series[split_time - 1:-1]" 165 | ], 166 | "execution_count": null, 167 | "outputs": [] 168 | }, 169 | { 170 | "cell_type": "code", 171 | "metadata": { 172 | "colab_type": "code", 173 | "id": "JtxwHj9Ig0jT", 174 | "colab": {} 175 | }, 176 | "source": [ 177 | "plt.figure(figsize=(10, 6))\n", 178 | "plot_series(time_valid, x_valid)\n", 179 | "plot_series(time_valid, naive_forecast)" 180 | ], 181 | "execution_count": null, 182 | "outputs": [] 183 | }, 184 | { 185 | "cell_type": "markdown", 186 | "metadata": { 187 | "colab_type": "text", 188 | "id": "fw1SP5WeuixH" 189 | }, 190 | "source": [ 191 | "Let's zoom in on the start of the validation period:" 192 | ] 193 | }, 194 | { 195 | "cell_type": "code", 196 | "metadata": { 197 | "colab_type": "code", 198 | "id": "D0MKg7FNug9V", 199 | "colab": {} 200 | }, 201 | "source": [ 202 | "plt.figure(figsize=(10, 6))\n", 203 | "plot_series(time_valid, x_valid, start=0, end=150)\n", 204 | "plot_series(time_valid, naive_forecast, start=1, end=151)" 205 | ], 206 | "execution_count": null, 207 | "outputs": [] 208 | }, 209 | { 210 | "cell_type": "markdown", 211 | "metadata": { 212 | "colab_type": "text", 213 | "id": "35gIlQLfu0TT" 214 | }, 215 | "source": [ 216 | "You can see that the naive forecast lags 1 step behind the time series." 217 | ] 218 | }, 219 | { 220 | "cell_type": "markdown", 221 | "metadata": { 222 | "colab_type": "text", 223 | "id": "Uh_7244Gsxfx" 224 | }, 225 | "source": [ 226 | "Now let's compute the mean squared error and the mean absolute error between the forecasts and the predictions in the validation period:" 227 | ] 228 | }, 229 | { 230 | "cell_type": "code", 231 | "metadata": { 232 | "colab_type": "code", 233 | "id": "byNnC7IbsnMZ", 234 | "colab": {} 235 | }, 236 | "source": [ 237 | "print(keras.metrics.mean_squared_error(x_valid, naive_forecast).numpy())\n", 238 | "print(keras.metrics.mean_absolute_error(x_valid, naive_forecast).numpy())" 239 | ], 240 | "execution_count": null, 241 | "outputs": [] 242 | }, 243 | { 244 | "cell_type": "markdown", 245 | "metadata": { 246 | "colab_type": "text", 247 | "id": "WGPBC9QttI1u" 248 | }, 249 | "source": [ 250 | "That's our baseline, now let's try a moving average:" 251 | ] 252 | }, 253 | { 254 | "cell_type": "code", 255 | "metadata": { 256 | "colab_type": "code", 257 | "id": "YGz5UsUdf2tV", 258 | "colab": {} 259 | }, 260 | "source": [ 261 | "def moving_average_forecast(series, window_size):\n", 262 | " \"\"\"Forecasts the mean of the last few values.\n", 263 | " If window_size=1, then this is equivalent to naive forecast\"\"\"\n", 264 | " forecast = []\n", 265 | " for time in range(len(series) - window_size):\n", 266 | " forecast.append(series[time:time + window_size].mean())\n", 267 | " return np.array(forecast)" 268 | ], 269 | "execution_count": null, 270 | "outputs": [] 271 | }, 272 | { 273 | "cell_type": "code", 274 | "metadata": { 275 | "colab_type": "code", 276 | "id": "HHFhGXQji7_r", 277 | "colab": {} 278 | }, 279 | "source": [ 280 | "moving_avg = moving_average_forecast(series, 30)[split_time - 30:]\n", 281 | "\n", 282 | "plt.figure(figsize=(10, 6))\n", 283 | "plot_series(time_valid, x_valid)\n", 284 | "plot_series(time_valid, moving_avg)" 285 | ], 286 | "execution_count": null, 287 | "outputs": [] 288 | }, 289 | { 290 | "cell_type": "code", 291 | "metadata": { 292 | "colab_type": "code", 293 | "id": "wG7pTAd7z0e8", 294 | "colab": {} 295 | }, 296 | "source": [ 297 | "print(keras.metrics.mean_squared_error(x_valid, moving_avg).numpy())\n", 298 | "print(keras.metrics.mean_absolute_error(x_valid, moving_avg).numpy())" 299 | ], 300 | "execution_count": null, 301 | "outputs": [] 302 | }, 303 | { 304 | "cell_type": "markdown", 305 | "metadata": { 306 | "colab_type": "text", 307 | "id": "JMYPnJqwz8nS" 308 | }, 309 | "source": [ 310 | "That's worse than naive forecast! The moving average does not anticipate trend or seasonality, so let's try to remove them by using differencing. Since the seasonality period is 365 days, we will subtract the value at time *t* – 365 from the value at time *t*." 311 | ] 312 | }, 313 | { 314 | "cell_type": "code", 315 | "metadata": { 316 | "colab_type": "code", 317 | "id": "5pqySF7-rJR4", 318 | "colab": {} 319 | }, 320 | "source": [ 321 | "diff_series = (series[365:] - series[:-365])\n", 322 | "diff_time = time[365:]\n", 323 | "\n", 324 | "plt.figure(figsize=(10, 6))\n", 325 | "plot_series(diff_time, diff_series)\n", 326 | "plt.show()" 327 | ], 328 | "execution_count": null, 329 | "outputs": [] 330 | }, 331 | { 332 | "cell_type": "markdown", 333 | "metadata": { 334 | "colab_type": "text", 335 | "id": "xPlPlS7DskWg" 336 | }, 337 | "source": [ 338 | "Great, the trend and seasonality seem to be gone, so now we can use the moving average:" 339 | ] 340 | }, 341 | { 342 | "cell_type": "code", 343 | "metadata": { 344 | "colab_type": "code", 345 | "id": "QmZpz7arsjbb", 346 | "colab": {} 347 | }, 348 | "source": [ 349 | "diff_moving_avg = moving_average_forecast(diff_series, 50)[split_time - 365 - 50:]\n", 350 | "\n", 351 | "plt.figure(figsize=(10, 6))\n", 352 | "plot_series(time_valid, diff_series[split_time - 365:])\n", 353 | "plot_series(time_valid, diff_moving_avg)\n", 354 | "plt.show()" 355 | ], 356 | "execution_count": null, 357 | "outputs": [] 358 | }, 359 | { 360 | "cell_type": "markdown", 361 | "metadata": { 362 | "colab_type": "text", 363 | "id": "Gno9S2lyecnc" 364 | }, 365 | "source": [ 366 | "Now let's bring back the trend and seasonality by adding the past values from t – 365:" 367 | ] 368 | }, 369 | { 370 | "cell_type": "code", 371 | "metadata": { 372 | "colab_type": "code", 373 | "id": "Dv6RWFq7TFGB", 374 | "colab": {} 375 | }, 376 | "source": [ 377 | "diff_moving_avg_plus_past = series[split_time - 365:-365] + diff_moving_avg\n", 378 | "\n", 379 | "plt.figure(figsize=(10, 6))\n", 380 | "plot_series(time_valid, x_valid)\n", 381 | "plot_series(time_valid, diff_moving_avg_plus_past)\n", 382 | "plt.show()" 383 | ], 384 | "execution_count": null, 385 | "outputs": [] 386 | }, 387 | { 388 | "cell_type": "code", 389 | "metadata": { 390 | "colab_type": "code", 391 | "id": "59jmBrwcTFCx", 392 | "colab": {} 393 | }, 394 | "source": [ 395 | "print(keras.metrics.mean_squared_error(x_valid, diff_moving_avg_plus_past).numpy())\n", 396 | "print(keras.metrics.mean_absolute_error(x_valid, diff_moving_avg_plus_past).numpy())" 397 | ], 398 | "execution_count": null, 399 | "outputs": [] 400 | }, 401 | { 402 | "cell_type": "markdown", 403 | "metadata": { 404 | "colab_type": "text", 405 | "id": "vx9Et1Hkeusl" 406 | }, 407 | "source": [ 408 | "Better than naive forecast, good. However the forecasts look a bit too random, because we're just adding past values, which were noisy. Let's use a moving averaging on past values to remove some of the noise:" 409 | ] 410 | }, 411 | { 412 | "cell_type": "code", 413 | "metadata": { 414 | "colab_type": "code", 415 | "id": "K81dtROoTE_r", 416 | "colab": {} 417 | }, 418 | "source": [ 419 | "diff_moving_avg_plus_smooth_past = moving_average_forecast(series[split_time - 370:-360], 10) + diff_moving_avg\n", 420 | "\n", 421 | "plt.figure(figsize=(10, 6))\n", 422 | "plot_series(time_valid, x_valid)\n", 423 | "plot_series(time_valid, diff_moving_avg_plus_smooth_past)\n", 424 | "plt.show()" 425 | ], 426 | "execution_count": null, 427 | "outputs": [] 428 | }, 429 | { 430 | "cell_type": "code", 431 | "metadata": { 432 | "colab_type": "code", 433 | "id": "iN2MsBxWTE3m", 434 | "colab": {} 435 | }, 436 | "source": [ 437 | "print(keras.metrics.mean_squared_error(x_valid, diff_moving_avg_plus_smooth_past).numpy())\n", 438 | "print(keras.metrics.mean_absolute_error(x_valid, diff_moving_avg_plus_smooth_past).numpy())" 439 | ], 440 | "execution_count": null, 441 | "outputs": [] 442 | } 443 | ] 444 | } -------------------------------------------------------------------------------- /4.Sequence, Time Series and Prediction/week1/Week_1_Exercise_Question.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "nbformat": 4, 3 | "nbformat_minor": 0, 4 | "metadata": { 5 | "accelerator": "GPU", 6 | "colab": { 7 | "name": "Week 1 Exercise Question.ipynb", 8 | "provenance": [], 9 | "collapsed_sections": [], 10 | "toc_visible": true 11 | }, 12 | "kernelspec": { 13 | "display_name": "Python 3", 14 | "name": "python3" 15 | } 16 | }, 17 | "cells": [ 18 | { 19 | "cell_type": "code", 20 | "metadata": { 21 | "id": "zX4Kg8DUTKWO", 22 | "colab_type": "code", 23 | "colab": {} 24 | }, 25 | "source": [ 26 | "#@title Licensed under the Apache License, Version 2.0 (the \"License\");\n", 27 | "# you may not use this file except in compliance with the License.\n", 28 | "# You may obtain a copy of the License at\n", 29 | "#\n", 30 | "# https://www.apache.org/licenses/LICENSE-2.0\n", 31 | "#\n", 32 | "# Unless required by applicable law or agreed to in writing, software\n", 33 | "# distributed under the License is distributed on an \"AS IS\" BASIS,\n", 34 | "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n", 35 | "# See the License for the specific language governing permissions and\n", 36 | "# limitations under the License." 37 | ], 38 | "execution_count": null, 39 | "outputs": [] 40 | }, 41 | { 42 | "cell_type": "code", 43 | "metadata": { 44 | "colab_type": "code", 45 | "id": "t9HrvPfrSlzS", 46 | "colab": {} 47 | }, 48 | "source": [ 49 | "import tensorflow as tf\n", 50 | "print(tf.__version__)\n", 51 | "\n", 52 | "# EXPECTED OUTPUT\n", 53 | "# 2.0.0-beta1 (or later)\n" 54 | ], 55 | "execution_count": null, 56 | "outputs": [] 57 | }, 58 | { 59 | "cell_type": "code", 60 | "metadata": { 61 | "colab_type": "code", 62 | "id": "gqWabzlJ63nL", 63 | "colab": {} 64 | }, 65 | "source": [ 66 | "import numpy as np\n", 67 | "import matplotlib.pyplot as plt\n", 68 | "import tensorflow as tf\n", 69 | "from tensorflow import keras\n", 70 | "\n", 71 | "def plot_series(time, series, format=\"-\", start=0, end=None):\n", 72 | " plt.plot(time[start:end], series[start:end], format)\n", 73 | " plt.xlabel(\"Time\")\n", 74 | " plt.ylabel(\"Value\")\n", 75 | " plt.grid(True)\n", 76 | "\n", 77 | "def trend(time, slope=0):\n", 78 | " return slope * time\n", 79 | "\n", 80 | "def seasonal_pattern(season_time):\n", 81 | " \"\"\"Just an arbitrary pattern, you can change it if you wish\"\"\"\n", 82 | " return np.where(season_time < 0.1,\n", 83 | " np.cos(season_time * 7 * np.pi),\n", 84 | " 1 / np.exp(5 * season_time))\n", 85 | "\n", 86 | "def seasonality(time, period, amplitude=1, phase=0):\n", 87 | " \"\"\"Repeats the same pattern at each period\"\"\"\n", 88 | " season_time = ((time + phase) % period) / period\n", 89 | " return amplitude * seasonal_pattern(season_time)\n", 90 | "\n", 91 | "def noise(time, noise_level=1, seed=None):\n", 92 | " rnd = np.random.RandomState(seed)\n", 93 | " return rnd.randn(len(time)) * noise_level\n", 94 | "\n", 95 | "time = np.arange(4 * 365 + 1, dtype=\"float32\")\n", 96 | "baseline = 10\n", 97 | "series = trend(time, 0.1) \n", 98 | "baseline = 10\n", 99 | "amplitude = 40\n", 100 | "slope = 0.01\n", 101 | "noise_level = 2\n", 102 | "\n", 103 | "# Create the series\n", 104 | "series = baseline + trend(time, slope) + seasonality(time, period=365, amplitude=amplitude)\n", 105 | "# Update with noise\n", 106 | "series += noise(time, noise_level, seed=42)\n", 107 | "\n", 108 | "plt.figure(figsize=(10, 6))\n", 109 | "plot_series(time, series)\n", 110 | "plt.show()\n", 111 | "\n", 112 | "# EXPECTED OUTPUT\n", 113 | "# Chart as in the screencast. First should have 5 distinctive 'peaks'" 114 | ], 115 | "execution_count": null, 116 | "outputs": [] 117 | }, 118 | { 119 | "cell_type": "markdown", 120 | "metadata": { 121 | "colab_type": "text", 122 | "id": "UfdyqJJ1VZVu" 123 | }, 124 | "source": [ 125 | "Now that we have the time series, let's split it so we can start forecasting" 126 | ] 127 | }, 128 | { 129 | "cell_type": "code", 130 | "metadata": { 131 | "colab_type": "code", 132 | "id": "_w0eKap5uFNP", 133 | "colab": {} 134 | }, 135 | "source": [ 136 | "split_time = # YOUR CODE HERE\n", 137 | "time_train = time[:split_time]\n", 138 | "x_train = series[:split_time]\n", 139 | "time_valid = time[split_time:]\n", 140 | "x_valid = series[split_time:]\n", 141 | "plt.figure(figsize=(10, 6))\n", 142 | "plot_series(time_train, x_train)\n", 143 | "plt.show()\n", 144 | "\n", 145 | "plt.figure(figsize=(10, 6))\n", 146 | "plot_series(time_valid, x_valid)\n", 147 | "plt.show()\n", 148 | "\n", 149 | "# EXPECTED OUTPUT\n", 150 | "# Chart WITH 4 PEAKS between 50 and 65 and 3 troughs between -12 and 0\n", 151 | "# Chart with 2 Peaks, first at slightly above 60, last at a little more than that, should also have a single trough at about 0" 152 | ], 153 | "execution_count": null, 154 | "outputs": [] 155 | }, 156 | { 157 | "cell_type": "markdown", 158 | "metadata": { 159 | "colab_type": "text", 160 | "id": "bjD8ncEZbjEW" 161 | }, 162 | "source": [ 163 | "# Naive Forecast" 164 | ] 165 | }, 166 | { 167 | "cell_type": "code", 168 | "metadata": { 169 | "colab_type": "code", 170 | "id": "Pj_-uCeYxcAb", 171 | "colab": {} 172 | }, 173 | "source": [ 174 | "naive_forecast = series[#YOUR CODE HERE]" 175 | ], 176 | "execution_count": null, 177 | "outputs": [] 178 | }, 179 | { 180 | "cell_type": "code", 181 | "metadata": { 182 | "colab_type": "code", 183 | "id": "JtxwHj9Ig0jT", 184 | "colab": {} 185 | }, 186 | "source": [ 187 | "plt.figure(figsize=(10, 6))\n", 188 | "plot_series(time_valid, x_valid)\n", 189 | "plot_series(time_valid, naive_forecast)\n", 190 | "\n", 191 | "# Expected output: Chart similar to above, but with forecast overlay" 192 | ], 193 | "execution_count": null, 194 | "outputs": [] 195 | }, 196 | { 197 | "cell_type": "markdown", 198 | "metadata": { 199 | "colab_type": "text", 200 | "id": "fw1SP5WeuixH" 201 | }, 202 | "source": [ 203 | "Let's zoom in on the start of the validation period:" 204 | ] 205 | }, 206 | { 207 | "cell_type": "code", 208 | "metadata": { 209 | "colab_type": "code", 210 | "id": "D0MKg7FNug9V", 211 | "colab": {} 212 | }, 213 | "source": [ 214 | "plt.figure(figsize=(10, 6))\n", 215 | "plot_series(# YOUR CODE HERE)\n", 216 | "plot_series(# YOUR CODE HERE)\n", 217 | "\n", 218 | "# EXPECTED - Chart with X-Axis from 1100-1250 and Y Axes with series value and projections. Projections should be time stepped 1 unit 'after' series" 219 | ], 220 | "execution_count": null, 221 | "outputs": [] 222 | }, 223 | { 224 | "cell_type": "markdown", 225 | "metadata": { 226 | "colab_type": "text", 227 | "id": "Uh_7244Gsxfx" 228 | }, 229 | "source": [ 230 | "Now let's compute the mean squared error and the mean absolute error between the forecasts and the predictions in the validation period:" 231 | ] 232 | }, 233 | { 234 | "cell_type": "code", 235 | "metadata": { 236 | "colab_type": "code", 237 | "id": "byNnC7IbsnMZ", 238 | "colab": {} 239 | }, 240 | "source": [ 241 | "print(# YOUR CODE HERE)\n", 242 | "print(# YOUR CODE HERE)\n", 243 | "# Expected Output\n", 244 | "# 19.578304\n", 245 | "# 2.6011968" 246 | ], 247 | "execution_count": null, 248 | "outputs": [] 249 | }, 250 | { 251 | "cell_type": "markdown", 252 | "metadata": { 253 | "colab_type": "text", 254 | "id": "WGPBC9QttI1u" 255 | }, 256 | "source": [ 257 | "That's our baseline, now let's try a moving average:" 258 | ] 259 | }, 260 | { 261 | "cell_type": "code", 262 | "metadata": { 263 | "colab_type": "code", 264 | "id": "YGz5UsUdf2tV", 265 | "colab": {} 266 | }, 267 | "source": [ 268 | "def moving_average_forecast(series, window_size):\n", 269 | " \"\"\"Forecasts the mean of the last few values.\n", 270 | " If window_size=1, then this is equivalent to naive forecast\"\"\"\n", 271 | " # YOUR CODE HERE" 272 | ], 273 | "execution_count": null, 274 | "outputs": [] 275 | }, 276 | { 277 | "cell_type": "code", 278 | "metadata": { 279 | "colab_type": "code", 280 | "id": "HHFhGXQji7_r", 281 | "colab": {} 282 | }, 283 | "source": [ 284 | "moving_avg = moving_average_forecast(# YOUR CODE HERE)[# YOUR CODE HERE]\n", 285 | "\n", 286 | "plt.figure(figsize=(10, 6))\n", 287 | "plot_series(time_valid, x_valid)\n", 288 | "plot_series(time_valid, moving_avg)\n", 289 | " \n", 290 | "# EXPECTED OUTPUT\n", 291 | "# CHart with time series from 1100->1450+ on X\n", 292 | "# Time series plotted\n", 293 | "# Moving average plotted over it" 294 | ], 295 | "execution_count": null, 296 | "outputs": [] 297 | }, 298 | { 299 | "cell_type": "code", 300 | "metadata": { 301 | "colab_type": "code", 302 | "id": "wG7pTAd7z0e8", 303 | "colab": {} 304 | }, 305 | "source": [ 306 | "print(# YOUR CODE HERE)\n", 307 | "print(# YOUR CODE HERE)\n", 308 | "# EXPECTED OUTPUT\n", 309 | "# 65.786224\n", 310 | "# 4.3040023" 311 | ], 312 | "execution_count": null, 313 | "outputs": [] 314 | }, 315 | { 316 | "cell_type": "code", 317 | "metadata": { 318 | "colab_type": "code", 319 | "id": "5pqySF7-rJR4", 320 | "colab": {} 321 | }, 322 | "source": [ 323 | "diff_series = (# YOUR CODE HERE)\n", 324 | "diff_time = # YOUR CODE HERE\n", 325 | "\n", 326 | "plt.figure(figsize=(10, 6))\n", 327 | "plot_series(diff_time, diff_series)\n", 328 | "plt.show()\n", 329 | " \n", 330 | "# EXPECETED OUTPUT: CHart with diffs" 331 | ], 332 | "execution_count": null, 333 | "outputs": [] 334 | }, 335 | { 336 | "cell_type": "markdown", 337 | "metadata": { 338 | "colab_type": "text", 339 | "id": "xPlPlS7DskWg" 340 | }, 341 | "source": [ 342 | "Great, the trend and seasonality seem to be gone, so now we can use the moving average:" 343 | ] 344 | }, 345 | { 346 | "cell_type": "code", 347 | "metadata": { 348 | "colab_type": "code", 349 | "id": "QmZpz7arsjbb", 350 | "colab": {} 351 | }, 352 | "source": [ 353 | "diff_moving_avg = # YOUR CODE HERE\n", 354 | "\n", 355 | "plt.figure(figsize=(10, 6))\n", 356 | "plot_series(time_valid, # YOUR CODE HERE)\n", 357 | "plot_series(time_valid, # YOUR CODE HERE)\n", 358 | "plt.show()\n", 359 | " \n", 360 | "# Expected output. Diff chart from 1100->1450 +\n", 361 | "# Overlaid with moving average" 362 | ], 363 | "execution_count": null, 364 | "outputs": [] 365 | }, 366 | { 367 | "cell_type": "markdown", 368 | "metadata": { 369 | "colab_type": "text", 370 | "id": "Gno9S2lyecnc" 371 | }, 372 | "source": [ 373 | "Now let's bring back the trend and seasonality by adding the past values from t – 365:" 374 | ] 375 | }, 376 | { 377 | "cell_type": "code", 378 | "metadata": { 379 | "colab_type": "code", 380 | "id": "Dv6RWFq7TFGB", 381 | "colab": {} 382 | }, 383 | "source": [ 384 | "diff_moving_avg_plus_past = # YOUR CODE HERE\n", 385 | "\n", 386 | "plt.figure(figsize=(10, 6))\n", 387 | "plot_series(time_valid, # YOUR CODE HERE)\n", 388 | "plot_series(time_valid, # YOUR CODE HERE)\n", 389 | "plt.show()\n", 390 | "# Expected output: Chart from 1100->1450+ on X. Same chart as earlier for time series, but projection overlaid looks close in value to it" 391 | ], 392 | "execution_count": null, 393 | "outputs": [] 394 | }, 395 | { 396 | "cell_type": "code", 397 | "metadata": { 398 | "colab_type": "code", 399 | "id": "59jmBrwcTFCx", 400 | "colab": {} 401 | }, 402 | "source": [ 403 | "print(# YOUR CODE HERE)\n", 404 | "print(# YOUR CODE HERE)\n", 405 | "# EXPECTED OUTPUT\n", 406 | "# 8.498155\n", 407 | "# 2.327179" 408 | ], 409 | "execution_count": null, 410 | "outputs": [] 411 | }, 412 | { 413 | "cell_type": "markdown", 414 | "metadata": { 415 | "colab_type": "text", 416 | "id": "vx9Et1Hkeusl" 417 | }, 418 | "source": [ 419 | "Better than naive forecast, good. However the forecasts look a bit too random, because we're just adding past values, which were noisy. Let's use a moving averaging on past values to remove some of the noise:" 420 | ] 421 | }, 422 | { 423 | "cell_type": "code", 424 | "metadata": { 425 | "colab_type": "code", 426 | "id": "K81dtROoTE_r", 427 | "colab": {} 428 | }, 429 | "source": [ 430 | "diff_moving_avg_plus_smooth_past = # YOUR CODE HERE\n", 431 | "\n", 432 | "plt.figure(figsize=(10, 6))\n", 433 | "plot_series(time_valid, # YOUR CODE HERE)\n", 434 | "plot_series(time_valid, # YOUR CODE HERE)\n", 435 | "plt.show()\n", 436 | " \n", 437 | "# EXPECTED OUTPUT:\n", 438 | "# Similar chart to above, but the overlaid projections are much smoother" 439 | ], 440 | "execution_count": null, 441 | "outputs": [] 442 | }, 443 | { 444 | "cell_type": "code", 445 | "metadata": { 446 | "colab_type": "code", 447 | "id": "iN2MsBxWTE3m", 448 | "colab": {} 449 | }, 450 | "source": [ 451 | "print(# YOUR CODE HERE)\n", 452 | "print(# YOUR CODE HERE)\n", 453 | "# EXPECTED OUTPUT\n", 454 | "# 12.527958\n", 455 | "# 2.2034433" 456 | ], 457 | "execution_count": null, 458 | "outputs": [] 459 | } 460 | ] 461 | } -------------------------------------------------------------------------------- /4.Sequence, Time Series and Prediction/week2/S+P_Week_2_Exercise_Answer.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "nbformat": 4, 3 | "nbformat_minor": 0, 4 | "metadata": { 5 | "accelerator": "GPU", 6 | "colab": { 7 | "name": "S+P_Week_2_Exercise_Answer.ipynb", 8 | "provenance": [], 9 | "toc_visible": true 10 | }, 11 | "kernelspec": { 12 | "display_name": "Python 3", 13 | "name": "python3" 14 | } 15 | }, 16 | "cells": [ 17 | { 18 | "cell_type": "code", 19 | "metadata": { 20 | "id": "zX4Kg8DUTKWO", 21 | "colab_type": "code", 22 | "colab": {} 23 | }, 24 | "source": [ 25 | "#@title Licensed under the Apache License, Version 2.0 (the \"License\");\n", 26 | "# you may not use this file except in compliance with the License.\n", 27 | "# You may obtain a copy of the License at\n", 28 | "#\n", 29 | "# https://www.apache.org/licenses/LICENSE-2.0\n", 30 | "#\n", 31 | "# Unless required by applicable law or agreed to in writing, software\n", 32 | "# distributed under the License is distributed on an \"AS IS\" BASIS,\n", 33 | "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n", 34 | "# See the License for the specific language governing permissions and\n", 35 | "# limitations under the License." 36 | ], 37 | "execution_count": null, 38 | "outputs": [] 39 | }, 40 | { 41 | "cell_type": "code", 42 | "metadata": { 43 | "cellView": "both", 44 | "colab_type": "code", 45 | "id": "D1J15Vh_1Jih", 46 | "colab": {} 47 | }, 48 | "source": [ 49 | "!pip install tf-nightly-2.0-preview\n" 50 | ], 51 | "execution_count": null, 52 | "outputs": [] 53 | }, 54 | { 55 | "cell_type": "code", 56 | "metadata": { 57 | "colab_type": "code", 58 | "id": "BOjujz601HcS", 59 | "colab": {} 60 | }, 61 | "source": [ 62 | "import tensorflow as tf\n", 63 | "import numpy as np\n", 64 | "import matplotlib.pyplot as plt\n", 65 | "print(tf.__version__)" 66 | ], 67 | "execution_count": null, 68 | "outputs": [] 69 | }, 70 | { 71 | "cell_type": "code", 72 | "metadata": { 73 | "colab_type": "code", 74 | "id": "Zswl7jRtGzkk", 75 | "colab": {} 76 | }, 77 | "source": [ 78 | "def plot_series(time, series, format=\"-\", start=0, end=None):\n", 79 | " plt.plot(time[start:end], series[start:end], format)\n", 80 | " plt.xlabel(\"Time\")\n", 81 | " plt.ylabel(\"Value\")\n", 82 | " plt.grid(False)\n", 83 | "\n", 84 | "def trend(time, slope=0):\n", 85 | " return slope * time\n", 86 | "\n", 87 | "def seasonal_pattern(season_time):\n", 88 | " \"\"\"Just an arbitrary pattern, you can change it if you wish\"\"\"\n", 89 | " return np.where(season_time < 0.1,\n", 90 | " np.cos(season_time * 6 * np.pi),\n", 91 | " 2 / np.exp(9 * season_time))\n", 92 | "\n", 93 | "def seasonality(time, period, amplitude=1, phase=0):\n", 94 | " \"\"\"Repeats the same pattern at each period\"\"\"\n", 95 | " season_time = ((time + phase) % period) / period\n", 96 | " return amplitude * seasonal_pattern(season_time)\n", 97 | "\n", 98 | "def noise(time, noise_level=1, seed=None):\n", 99 | " rnd = np.random.RandomState(seed)\n", 100 | " return rnd.randn(len(time)) * noise_level\n", 101 | "\n", 102 | "time = np.arange(10 * 365 + 1, dtype=\"float32\")\n", 103 | "baseline = 10\n", 104 | "series = trend(time, 0.1) \n", 105 | "baseline = 10\n", 106 | "amplitude = 40\n", 107 | "slope = 0.005\n", 108 | "noise_level = 3\n", 109 | "\n", 110 | "# Create the series\n", 111 | "series = baseline + trend(time, slope) + seasonality(time, period=365, amplitude=amplitude)\n", 112 | "# Update with noise\n", 113 | "series += noise(time, noise_level, seed=51)\n", 114 | "\n", 115 | "split_time = 3000\n", 116 | "time_train = time[:split_time]\n", 117 | "x_train = series[:split_time]\n", 118 | "time_valid = time[split_time:]\n", 119 | "x_valid = series[split_time:]\n", 120 | "\n", 121 | "window_size = 20\n", 122 | "batch_size = 32\n", 123 | "shuffle_buffer_size = 1000\n", 124 | "\n", 125 | "plot_series(time, series)" 126 | ], 127 | "execution_count": null, 128 | "outputs": [] 129 | }, 130 | { 131 | "cell_type": "code", 132 | "metadata": { 133 | "colab_type": "code", 134 | "id": "4sTTIOCbyShY", 135 | "colab": {} 136 | }, 137 | "source": [ 138 | "def windowed_dataset(series, window_size, batch_size, shuffle_buffer):\n", 139 | " dataset = tf.data.Dataset.from_tensor_slices(series)\n", 140 | " dataset = dataset.window(window_size + 1, shift=1, drop_remainder=True)\n", 141 | " dataset = dataset.flat_map(lambda window: window.batch(window_size + 1))\n", 142 | " dataset = dataset.shuffle(shuffle_buffer).map(lambda window: (window[:-1], window[-1]))\n", 143 | " dataset = dataset.batch(batch_size).prefetch(1)\n", 144 | " return dataset" 145 | ], 146 | "execution_count": null, 147 | "outputs": [] 148 | }, 149 | { 150 | "cell_type": "code", 151 | "metadata": { 152 | "colab_type": "code", 153 | "id": "TW-vT7eLYAdb", 154 | "colab": {} 155 | }, 156 | "source": [ 157 | "dataset = windowed_dataset(x_train, window_size, batch_size, shuffle_buffer_size)\n", 158 | "\n", 159 | "\n", 160 | "model = tf.keras.models.Sequential([\n", 161 | " tf.keras.layers.Dense(100, input_shape=[window_size], activation=\"relu\"), \n", 162 | " tf.keras.layers.Dense(10, activation=\"relu\"), \n", 163 | " tf.keras.layers.Dense(1)\n", 164 | "])\n", 165 | "\n", 166 | "model.compile(loss=\"mse\", optimizer=tf.keras.optimizers.SGD(lr=1e-6, momentum=0.9))\n", 167 | "model.fit(dataset,epochs=100,verbose=0)\n" 168 | ], 169 | "execution_count": null, 170 | "outputs": [] 171 | }, 172 | { 173 | "cell_type": "code", 174 | "metadata": { 175 | "colab_type": "code", 176 | "id": "efhco2rYyIFF", 177 | "colab": {} 178 | }, 179 | "source": [ 180 | "forecast = []\n", 181 | "for time in range(len(series) - window_size):\n", 182 | " forecast.append(model.predict(series[time:time + window_size][np.newaxis]))\n", 183 | "\n", 184 | "forecast = forecast[split_time-window_size:]\n", 185 | "results = np.array(forecast)[:, 0, 0]\n", 186 | "\n", 187 | "\n", 188 | "plt.figure(figsize=(10, 6))\n", 189 | "\n", 190 | "plot_series(time_valid, x_valid)\n", 191 | "plot_series(time_valid, results)" 192 | ], 193 | "execution_count": null, 194 | "outputs": [] 195 | }, 196 | { 197 | "cell_type": "code", 198 | "metadata": { 199 | "colab_type": "code", 200 | "id": "-kT6j186YO6K", 201 | "colab": {} 202 | }, 203 | "source": [ 204 | "tf.keras.metrics.mean_absolute_error(x_valid, results).numpy()" 205 | ], 206 | "execution_count": null, 207 | "outputs": [] 208 | } 209 | ] 210 | } -------------------------------------------------------------------------------- /4.Sequence, Time Series and Prediction/week2/S+P_Week_2_Exercise_Question.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "nbformat": 4, 3 | "nbformat_minor": 0, 4 | "metadata": { 5 | "colab": { 6 | "name": "S+P_Week_2_Exercise_Question.ipynb", 7 | "provenance": [], 8 | "toc_visible": true 9 | }, 10 | "kernelspec": { 11 | "display_name": "Python 3", 12 | "name": "python3" 13 | } 14 | }, 15 | "cells": [ 16 | { 17 | "cell_type": "code", 18 | "metadata": { 19 | "id": "zX4Kg8DUTKWO", 20 | "colab_type": "code", 21 | "colab": {} 22 | }, 23 | "source": [ 24 | "#@title Licensed under the Apache License, Version 2.0 (the \"License\");\n", 25 | "# you may not use this file except in compliance with the License.\n", 26 | "# You may obtain a copy of the License at\n", 27 | "#\n", 28 | "# https://www.apache.org/licenses/LICENSE-2.0\n", 29 | "#\n", 30 | "# Unless required by applicable law or agreed to in writing, software\n", 31 | "# distributed under the License is distributed on an \"AS IS\" BASIS,\n", 32 | "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n", 33 | "# See the License for the specific language governing permissions and\n", 34 | "# limitations under the License." 35 | ], 36 | "execution_count": null, 37 | "outputs": [] 38 | }, 39 | { 40 | "cell_type": "code", 41 | "metadata": { 42 | "cellView": "both", 43 | "colab_type": "code", 44 | "id": "D1J15Vh_1Jih", 45 | "colab": {} 46 | }, 47 | "source": [ 48 | "!pip install tf-nightly-2.0-preview\n" 49 | ], 50 | "execution_count": null, 51 | "outputs": [] 52 | }, 53 | { 54 | "cell_type": "code", 55 | "metadata": { 56 | "colab_type": "code", 57 | "id": "BOjujz601HcS", 58 | "colab": {} 59 | }, 60 | "source": [ 61 | "import tensorflow as tf\n", 62 | "import numpy as np\n", 63 | "import matplotlib.pyplot as plt\n", 64 | "print(tf.__version__)" 65 | ], 66 | "execution_count": null, 67 | "outputs": [] 68 | }, 69 | { 70 | "cell_type": "code", 71 | "metadata": { 72 | "colab_type": "code", 73 | "id": "Zswl7jRtGzkk", 74 | "colab": {} 75 | }, 76 | "source": [ 77 | "def plot_series(time, series, format=\"-\", start=0, end=None):\n", 78 | " plt.plot(time[start:end], series[start:end], format)\n", 79 | " plt.xlabel(\"Time\")\n", 80 | " plt.ylabel(\"Value\")\n", 81 | " plt.grid(False)\n", 82 | "\n", 83 | "def trend(time, slope=0):\n", 84 | " return slope * time\n", 85 | "\n", 86 | "def seasonal_pattern(season_time):\n", 87 | " \"\"\"Just an arbitrary pattern, you can change it if you wish\"\"\"\n", 88 | " return np.where(season_time < 0.1,\n", 89 | " np.cos(season_time * # YOUR CODE HERE # * np.pi),\n", 90 | " #YOUR CODE HERE# / np.exp(#YOUR CODE HERE# * season_time))\n", 91 | "\n", 92 | "def seasonality(time, period, amplitude=1, phase=0):\n", 93 | " \"\"\"Repeats the same pattern at each period\"\"\"\n", 94 | " season_time = ((time + phase) % period) / period\n", 95 | " return amplitude * seasonal_pattern(season_time)\n", 96 | "\n", 97 | "def noise(time, noise_level=1, seed=None):\n", 98 | " rnd = np.random.RandomState(seed)\n", 99 | " return rnd.randn(len(time)) * noise_level\n", 100 | "\n", 101 | "time = np.arange(10 * 365 + 1, dtype=\"float32\")\n", 102 | "baseline = # YOUR CODE HERE #\n", 103 | "series = trend(time, # YOUR CODE HERE#) \n", 104 | "baseline = 10\n", 105 | "amplitude = 40\n", 106 | "slope = # YOUR CODE HERE#\n", 107 | "noise_level = # YOUR CODE HERE#\n", 108 | "\n", 109 | "# Create the series\n", 110 | "series = baseline + trend(time, slope) + seasonality(time, period=365, amplitude=amplitude)\n", 111 | "# Update with noise\n", 112 | "series += noise(time, noise_level, seed=51)\n", 113 | "\n", 114 | "split_time = 3000\n", 115 | "time_train = time[:split_time]\n", 116 | "x_train = series[:split_time]\n", 117 | "time_valid = time[split_time:]\n", 118 | "x_valid = series[split_time:]\n", 119 | "\n", 120 | "window_size = 20\n", 121 | "batch_size = 32\n", 122 | "shuffle_buffer_size = 1000\n", 123 | "\n", 124 | "plot_series(time, series)" 125 | ], 126 | "execution_count": null, 127 | "outputs": [] 128 | }, 129 | { 130 | "cell_type": "markdown", 131 | "metadata": { 132 | "colab_type": "text", 133 | "id": "GfUTNqti_lNC" 134 | }, 135 | "source": [ 136 | "Desired output -- a chart that looks like this:\n", 137 | "\n", 138 | "![Chart showing upward trend and seasonailty](http://www.laurencemoroney.com/wp-content/uploads/2019/07/plot1.png)" 139 | ] 140 | }, 141 | { 142 | "cell_type": "code", 143 | "metadata": { 144 | "colab_type": "code", 145 | "id": "4sTTIOCbyShY", 146 | "colab": {} 147 | }, 148 | "source": [ 149 | "def windowed_dataset(series, window_size, batch_size, shuffle_buffer):\n", 150 | " dataset = tf.data.Dataset.from_tensor_slices(series)\n", 151 | " dataset = dataset.window(window_size + 1, shift=1, drop_remainder=True)\n", 152 | " dataset = dataset.flat_map(lambda window: window.batch(window_size + 1))\n", 153 | " dataset = dataset.shuffle(shuffle_buffer).map(lambda window: (window[:-1], window[-1]))\n", 154 | " dataset = dataset.batch(batch_size).prefetch(1)\n", 155 | " return dataset" 156 | ], 157 | "execution_count": null, 158 | "outputs": [] 159 | }, 160 | { 161 | "cell_type": "code", 162 | "metadata": { 163 | "colab_type": "code", 164 | "id": "TW-vT7eLYAdb", 165 | "colab": {} 166 | }, 167 | "source": [ 168 | "dataset = windowed_dataset(x_train, window_size, batch_size, shuffle_buffer_size)\n", 169 | "\n", 170 | "\n", 171 | "model = tf.keras.models.Sequential([\n", 172 | " tf.keras.layers.Dense(# YOUR CODE HERE #),\n", 173 | " tf.keras.layers.Dense(# YOUR CODE HERE #, activation=\"relu\"), \n", 174 | " tf.keras.layers.Dense(1)\n", 175 | "])\n", 176 | "\n", 177 | "model.compile(loss=# YOUR CODE HERE #, optimizer=# YOUR CODE HERE#))\n", 178 | "model.fit(dataset,epochs=100,verbose=0)\n" 179 | ], 180 | "execution_count": null, 181 | "outputs": [] 182 | }, 183 | { 184 | "cell_type": "code", 185 | "metadata": { 186 | "colab_type": "code", 187 | "id": "efhco2rYyIFF", 188 | "colab": {} 189 | }, 190 | "source": [ 191 | "forecast = []\n", 192 | "for time in range(len(series) - window_size):\n", 193 | " forecast.append(model.predict(series[time:time + window_size][np.newaxis]))\n", 194 | "\n", 195 | "forecast = forecast[split_time-window_size:]\n", 196 | "results = np.array(forecast)[:, 0, 0]\n", 197 | "\n", 198 | "\n", 199 | "plt.figure(figsize=(10, 6))\n", 200 | "\n", 201 | "plot_series(time_valid, x_valid)\n", 202 | "plot_series(time_valid, results)" 203 | ], 204 | "execution_count": null, 205 | "outputs": [] 206 | }, 207 | { 208 | "cell_type": "code", 209 | "metadata": { 210 | "colab_type": "code", 211 | "id": "-kT6j186YO6K", 212 | "colab": {} 213 | }, 214 | "source": [ 215 | "tf.keras.metrics.mean_absolute_error(x_valid, results).numpy()\n", 216 | "# EXPECTED OUTPUT\n", 217 | "# A Value less than 3" 218 | ], 219 | "execution_count": null, 220 | "outputs": [] 221 | } 222 | ] 223 | } -------------------------------------------------------------------------------- /4.Sequence, Time Series and Prediction/week2/S+P_Week_2_Lesson_1.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "nbformat": 4, 3 | "nbformat_minor": 0, 4 | "metadata": { 5 | "accelerator": "GPU", 6 | "colab": { 7 | "name": "S+P Week 2 Lesson 1.ipynb", 8 | "provenance": [], 9 | "toc_visible": true 10 | }, 11 | "kernelspec": { 12 | "display_name": "Python 3", 13 | "name": "python3" 14 | } 15 | }, 16 | "cells": [ 17 | { 18 | "cell_type": "code", 19 | "metadata": { 20 | "id": "zX4Kg8DUTKWO", 21 | "colab_type": "code", 22 | "colab": {} 23 | }, 24 | "source": [ 25 | "#@title Licensed under the Apache License, Version 2.0 (the \"License\");\n", 26 | "# you may not use this file except in compliance with the License.\n", 27 | "# You may obtain a copy of the License at\n", 28 | "#\n", 29 | "# https://www.apache.org/licenses/LICENSE-2.0\n", 30 | "#\n", 31 | "# Unless required by applicable law or agreed to in writing, software\n", 32 | "# distributed under the License is distributed on an \"AS IS\" BASIS,\n", 33 | "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n", 34 | "# See the License for the specific language governing permissions and\n", 35 | "# limitations under the License." 36 | ], 37 | "execution_count": null, 38 | "outputs": [] 39 | }, 40 | { 41 | "cell_type": "code", 42 | "metadata": { 43 | "colab_type": "code", 44 | "id": "s6eq-RBcQ_Zr", 45 | "colab": {} 46 | }, 47 | "source": [ 48 | "try:\n", 49 | " # %tensorflow_version only exists in Colab.\n", 50 | " %tensorflow_version 2.x\n", 51 | "except Exception:\n", 52 | " pass" 53 | ], 54 | "execution_count": 1, 55 | "outputs": [] 56 | }, 57 | { 58 | "cell_type": "code", 59 | "metadata": { 60 | "colab_type": "code", 61 | "id": "BOjujz601HcS", 62 | "colab": { 63 | "base_uri": "https://localhost:8080/", 64 | "height": 34 65 | }, 66 | "outputId": "9276f0d2-ecec-433d-c72f-4af6d90c6a37" 67 | }, 68 | "source": [ 69 | "import tensorflow as tf\n", 70 | "import numpy as np\n", 71 | "import matplotlib.pyplot as plt\n", 72 | "print(tf.__version__)" 73 | ], 74 | "execution_count": 2, 75 | "outputs": [ 76 | { 77 | "output_type": "stream", 78 | "text": [ 79 | "2.3.0\n" 80 | ], 81 | "name": "stdout" 82 | } 83 | ] 84 | }, 85 | { 86 | "cell_type": "code", 87 | "metadata": { 88 | "colab_type": "code", 89 | "id": "asEdslR_05O_", 90 | "colab": { 91 | "base_uri": "https://localhost:8080/", 92 | "height": 187 93 | }, 94 | "outputId": "2cdc1e53-2a76-4c6a-c4ba-7dcb11ca697b" 95 | }, 96 | "source": [ 97 | "dataset = tf.data.Dataset.range(10)\n", 98 | "for val in dataset:\n", 99 | " print(val.numpy())" 100 | ], 101 | "execution_count": 3, 102 | "outputs": [ 103 | { 104 | "output_type": "stream", 105 | "text": [ 106 | "0\n", 107 | "1\n", 108 | "2\n", 109 | "3\n", 110 | "4\n", 111 | "5\n", 112 | "6\n", 113 | "7\n", 114 | "8\n", 115 | "9\n" 116 | ], 117 | "name": "stdout" 118 | } 119 | ] 120 | }, 121 | { 122 | "cell_type": "code", 123 | "metadata": { 124 | "colab_type": "code", 125 | "id": "Lrv_ghSt1lgQ", 126 | "colab": { 127 | "base_uri": "https://localhost:8080/", 128 | "height": 187 129 | }, 130 | "outputId": "795ec422-bade-455f-f6ce-f5309af7be47" 131 | }, 132 | "source": [ 133 | "dataset = tf.data.Dataset.range(10)\n", 134 | "dataset = dataset.window(5, shift=1)\n", 135 | "for window_dataset in dataset:\n", 136 | " for val in window_dataset:\n", 137 | " print(val.numpy(), end=\" \")\n", 138 | " print()" 139 | ], 140 | "execution_count": 4, 141 | "outputs": [ 142 | { 143 | "output_type": "stream", 144 | "text": [ 145 | "0 1 2 3 4 \n", 146 | "1 2 3 4 5 \n", 147 | "2 3 4 5 6 \n", 148 | "3 4 5 6 7 \n", 149 | "4 5 6 7 8 \n", 150 | "5 6 7 8 9 \n", 151 | "6 7 8 9 \n", 152 | "7 8 9 \n", 153 | "8 9 \n", 154 | "9 \n" 155 | ], 156 | "name": "stdout" 157 | } 158 | ] 159 | }, 160 | { 161 | "cell_type": "code", 162 | "metadata": { 163 | "colab_type": "code", 164 | "id": "QLEq6MG-2DN2", 165 | "colab": { 166 | "base_uri": "https://localhost:8080/", 167 | "height": 119 168 | }, 169 | "outputId": "9c2232ab-ed60-49a5-9d14-9aec418cb44c" 170 | }, 171 | "source": [ 172 | "dataset = tf.data.Dataset.range(10)\n", 173 | "dataset = dataset.window(5, shift=1, drop_remainder=True)\n", 174 | "for window_dataset in dataset:\n", 175 | " for val in window_dataset:\n", 176 | " print(val.numpy(), end=\" \")\n", 177 | " print()" 178 | ], 179 | "execution_count": 5, 180 | "outputs": [ 181 | { 182 | "output_type": "stream", 183 | "text": [ 184 | "0 1 2 3 4 \n", 185 | "1 2 3 4 5 \n", 186 | "2 3 4 5 6 \n", 187 | "3 4 5 6 7 \n", 188 | "4 5 6 7 8 \n", 189 | "5 6 7 8 9 \n" 190 | ], 191 | "name": "stdout" 192 | } 193 | ] 194 | }, 195 | { 196 | "cell_type": "code", 197 | "metadata": { 198 | "colab_type": "code", 199 | "id": "PJ9CAHlJ2ODe", 200 | "colab": { 201 | "base_uri": "https://localhost:8080/", 202 | "height": 119 203 | }, 204 | "outputId": "a6791f59-8740-4234-8c45-7ee5468a78cb" 205 | }, 206 | "source": [ 207 | "dataset = tf.data.Dataset.range(10)\n", 208 | "dataset = dataset.window(5, shift=1, drop_remainder=True)\n", 209 | "dataset = dataset.flat_map(lambda window: window.batch(5))\n", 210 | "for window in dataset:\n", 211 | " print(window.numpy())\n" 212 | ], 213 | "execution_count": 6, 214 | "outputs": [ 215 | { 216 | "output_type": "stream", 217 | "text": [ 218 | "[0 1 2 3 4]\n", 219 | "[1 2 3 4 5]\n", 220 | "[2 3 4 5 6]\n", 221 | "[3 4 5 6 7]\n", 222 | "[4 5 6 7 8]\n", 223 | "[5 6 7 8 9]\n" 224 | ], 225 | "name": "stdout" 226 | } 227 | ] 228 | }, 229 | { 230 | "cell_type": "code", 231 | "metadata": { 232 | "colab_type": "code", 233 | "id": "DryEZ2Mz2nNV", 234 | "colab": { 235 | "base_uri": "https://localhost:8080/", 236 | "height": 119 237 | }, 238 | "outputId": "db302d27-fa9a-40c3-fa44-205d33a546fa" 239 | }, 240 | "source": [ 241 | "dataset = tf.data.Dataset.range(10)\n", 242 | "dataset = dataset.window(5, shift=1, drop_remainder=True)\n", 243 | "dataset = dataset.flat_map(lambda window: window.batch(5))\n", 244 | "dataset = dataset.map(lambda window: (window[:-1], window[-1:]))\n", 245 | "for x,y in dataset:\n", 246 | " print(x.numpy(), y.numpy())" 247 | ], 248 | "execution_count": 7, 249 | "outputs": [ 250 | { 251 | "output_type": "stream", 252 | "text": [ 253 | "[0 1 2 3] [4]\n", 254 | "[1 2 3 4] [5]\n", 255 | "[2 3 4 5] [6]\n", 256 | "[3 4 5 6] [7]\n", 257 | "[4 5 6 7] [8]\n", 258 | "[5 6 7 8] [9]\n" 259 | ], 260 | "name": "stdout" 261 | } 262 | ] 263 | }, 264 | { 265 | "cell_type": "code", 266 | "metadata": { 267 | "colab_type": "code", 268 | "id": "1tl-0BOKkEtk", 269 | "colab": { 270 | "base_uri": "https://localhost:8080/", 271 | "height": 119 272 | }, 273 | "outputId": "df405540-fb15-42ab-d426-f9f8ccb787c7" 274 | }, 275 | "source": [ 276 | "dataset = tf.data.Dataset.range(10)\n", 277 | "dataset = dataset.window(5, shift=1, drop_remainder=True)\n", 278 | "dataset = dataset.flat_map(lambda window: window.batch(5))\n", 279 | "dataset = dataset.map(lambda window: (window[:-1], window[-1:]))\n", 280 | "dataset = dataset.shuffle(buffer_size=10)\n", 281 | "for x,y in dataset:\n", 282 | " print(x.numpy(), y.numpy())\n" 283 | ], 284 | "execution_count": 8, 285 | "outputs": [ 286 | { 287 | "output_type": "stream", 288 | "text": [ 289 | "[2 3 4 5] [6]\n", 290 | "[1 2 3 4] [5]\n", 291 | "[0 1 2 3] [4]\n", 292 | "[5 6 7 8] [9]\n", 293 | "[4 5 6 7] [8]\n", 294 | "[3 4 5 6] [7]\n" 295 | ], 296 | "name": "stdout" 297 | } 298 | ] 299 | }, 300 | { 301 | "cell_type": "code", 302 | "metadata": { 303 | "colab_type": "code", 304 | "id": "Wa0PNwxMGapy", 305 | "colab": { 306 | "base_uri": "https://localhost:8080/", 307 | "height": 221 308 | }, 309 | "outputId": "55203ac6-3138-4295-f065-d2c2590b914c" 310 | }, 311 | "source": [ 312 | "dataset = tf.data.Dataset.range(10)\n", 313 | "dataset = dataset.window(5, shift=1, drop_remainder=True)\n", 314 | "dataset = dataset.flat_map(lambda window: window.batch(5))\n", 315 | "dataset = dataset.map(lambda window: (window[:-1], window[-1:]))\n", 316 | "dataset = dataset.shuffle(buffer_size=10)\n", 317 | "dataset = dataset.batch(2).prefetch(1)\n", 318 | "for x,y in dataset:\n", 319 | " print(\"x = \", x.numpy())\n", 320 | " print(\"y = \", y.numpy())\n" 321 | ], 322 | "execution_count": 9, 323 | "outputs": [ 324 | { 325 | "output_type": "stream", 326 | "text": [ 327 | "x = [[0 1 2 3]\n", 328 | " [4 5 6 7]]\n", 329 | "y = [[4]\n", 330 | " [8]]\n", 331 | "x = [[5 6 7 8]\n", 332 | " [3 4 5 6]]\n", 333 | "y = [[9]\n", 334 | " [7]]\n", 335 | "x = [[2 3 4 5]\n", 336 | " [1 2 3 4]]\n", 337 | "y = [[6]\n", 338 | " [5]]\n" 339 | ], 340 | "name": "stdout" 341 | } 342 | ] 343 | }, 344 | { 345 | "cell_type": "code", 346 | "metadata": { 347 | "id": "eEmlDJsayH7l", 348 | "colab_type": "code", 349 | "colab": {} 350 | }, 351 | "source": [ 352 | "" 353 | ], 354 | "execution_count": null, 355 | "outputs": [] 356 | } 357 | ] 358 | } -------------------------------------------------------------------------------- /4.Sequence, Time Series and Prediction/week3/S+P_Week_3_Exercise_Answer.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "nbformat": 4, 3 | "nbformat_minor": 0, 4 | "metadata": { 5 | "accelerator": "GPU", 6 | "colab": { 7 | "name": "S+P Week 3 Exercise Answer.ipynb", 8 | "provenance": [], 9 | "collapsed_sections": [], 10 | "toc_visible": true 11 | }, 12 | "kernelspec": { 13 | "display_name": "Python 3", 14 | "name": "python3" 15 | } 16 | }, 17 | "cells": [ 18 | { 19 | "cell_type": "code", 20 | "metadata": { 21 | "id": "zX4Kg8DUTKWO", 22 | "colab_type": "code", 23 | "colab": {} 24 | }, 25 | "source": [ 26 | "#@title Licensed under the Apache License, Version 2.0 (the \"License\");\n", 27 | "# you may not use this file except in compliance with the License.\n", 28 | "# You may obtain a copy of the License at\n", 29 | "#\n", 30 | "# https://www.apache.org/licenses/LICENSE-2.0\n", 31 | "#\n", 32 | "# Unless required by applicable law or agreed to in writing, software\n", 33 | "# distributed under the License is distributed on an \"AS IS\" BASIS,\n", 34 | "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n", 35 | "# See the License for the specific language governing permissions and\n", 36 | "# limitations under the License." 37 | ], 38 | "execution_count": null, 39 | "outputs": [] 40 | }, 41 | { 42 | "cell_type": "code", 43 | "metadata": { 44 | "colab_type": "code", 45 | "id": "D1J15Vh_1Jih", 46 | "colab": {} 47 | }, 48 | "source": [ 49 | "!pip install tf-nightly-2.0-preview\n" 50 | ], 51 | "execution_count": null, 52 | "outputs": [] 53 | }, 54 | { 55 | "cell_type": "code", 56 | "metadata": { 57 | "colab_type": "code", 58 | "id": "BOjujz601HcS", 59 | "colab": {} 60 | }, 61 | "source": [ 62 | "import tensorflow as tf\n", 63 | "import numpy as np\n", 64 | "import matplotlib.pyplot as plt\n", 65 | "print(tf.__version__)" 66 | ], 67 | "execution_count": null, 68 | "outputs": [] 69 | }, 70 | { 71 | "cell_type": "code", 72 | "metadata": { 73 | "colab_type": "code", 74 | "id": "Zswl7jRtGzkk", 75 | "colab": {} 76 | }, 77 | "source": [ 78 | "def plot_series(time, series, format=\"-\", start=0, end=None):\n", 79 | " plt.plot(time[start:end], series[start:end], format)\n", 80 | " plt.xlabel(\"Time\")\n", 81 | " plt.ylabel(\"Value\")\n", 82 | " plt.grid(False)\n", 83 | "\n", 84 | "def trend(time, slope=0):\n", 85 | " return slope * time\n", 86 | "\n", 87 | "def seasonal_pattern(season_time):\n", 88 | " \"\"\"Just an arbitrary pattern, you can change it if you wish\"\"\"\n", 89 | " return np.where(season_time < 0.1,\n", 90 | " np.cos(season_time * 6 * np.pi),\n", 91 | " 2 / np.exp(9 * season_time))\n", 92 | "\n", 93 | "def seasonality(time, period, amplitude=1, phase=0):\n", 94 | " \"\"\"Repeats the same pattern at each period\"\"\"\n", 95 | " season_time = ((time + phase) % period) / period\n", 96 | " return amplitude * seasonal_pattern(season_time)\n", 97 | "\n", 98 | "def noise(time, noise_level=1, seed=None):\n", 99 | " rnd = np.random.RandomState(seed)\n", 100 | " return rnd.randn(len(time)) * noise_level\n", 101 | "\n", 102 | "time = np.arange(10 * 365 + 1, dtype=\"float32\")\n", 103 | "baseline = 10\n", 104 | "series = trend(time, 0.1) \n", 105 | "baseline = 10\n", 106 | "amplitude = 40\n", 107 | "slope = 0.005\n", 108 | "noise_level = 3\n", 109 | "\n", 110 | "# Create the series\n", 111 | "series = baseline + trend(time, slope) + seasonality(time, period=365, amplitude=amplitude)\n", 112 | "# Update with noise\n", 113 | "series += noise(time, noise_level, seed=51)\n", 114 | "\n", 115 | "split_time = 3000\n", 116 | "time_train = time[:split_time]\n", 117 | "x_train = series[:split_time]\n", 118 | "time_valid = time[split_time:]\n", 119 | "x_valid = series[split_time:]\n", 120 | "\n", 121 | "window_size = 20\n", 122 | "batch_size = 32\n", 123 | "shuffle_buffer_size = 1000\n", 124 | "\n", 125 | "plot_series(time, series)" 126 | ], 127 | "execution_count": null, 128 | "outputs": [] 129 | }, 130 | { 131 | "cell_type": "code", 132 | "metadata": { 133 | "colab_type": "code", 134 | "id": "4sTTIOCbyShY", 135 | "colab": {} 136 | }, 137 | "source": [ 138 | "def windowed_dataset(series, window_size, batch_size, shuffle_buffer):\n", 139 | " dataset = tf.data.Dataset.from_tensor_slices(series)\n", 140 | " dataset = dataset.window(window_size + 1, shift=1, drop_remainder=True)\n", 141 | " dataset = dataset.flat_map(lambda window: window.batch(window_size + 1))\n", 142 | " dataset = dataset.shuffle(shuffle_buffer).map(lambda window: (window[:-1], window[-1]))\n", 143 | " dataset = dataset.batch(batch_size).prefetch(1)\n", 144 | " return dataset" 145 | ], 146 | "execution_count": null, 147 | "outputs": [] 148 | }, 149 | { 150 | "cell_type": "code", 151 | "metadata": { 152 | "colab_type": "code", 153 | "id": "A1Hl39rklkLm", 154 | "colab": {} 155 | }, 156 | "source": [ 157 | "tf.keras.backend.clear_session()\n", 158 | "tf.random.set_seed(51)\n", 159 | "np.random.seed(51)\n", 160 | "\n", 161 | "tf.keras.backend.clear_session()\n", 162 | "dataset = windowed_dataset(x_train, window_size, batch_size, shuffle_buffer_size)\n", 163 | "\n", 164 | "model = tf.keras.models.Sequential([\n", 165 | " tf.keras.layers.Lambda(lambda x: tf.expand_dims(x, axis=-1),\n", 166 | " input_shape=[None]),\n", 167 | " tf.keras.layers.Bidirectional(tf.keras.layers.LSTM(32, return_sequences=True)),\n", 168 | " tf.keras.layers.Bidirectional(tf.keras.layers.LSTM(32)),\n", 169 | " tf.keras.layers.Dense(1),\n", 170 | " tf.keras.layers.Lambda(lambda x: x * 10.0)\n", 171 | "])\n", 172 | "\n", 173 | "lr_schedule = tf.keras.callbacks.LearningRateScheduler(\n", 174 | " lambda epoch: 1e-8 * 10**(epoch / 20))\n", 175 | "optimizer = tf.keras.optimizers.SGD(lr=1e-8, momentum=0.9)\n", 176 | "model.compile(loss=tf.keras.losses.Huber(),\n", 177 | " optimizer=optimizer,\n", 178 | " metrics=[\"mae\"])\n", 179 | "history = model.fit(dataset, epochs=100, callbacks=[lr_schedule])" 180 | ], 181 | "execution_count": null, 182 | "outputs": [] 183 | }, 184 | { 185 | "cell_type": "code", 186 | "metadata": { 187 | "colab_type": "code", 188 | "id": "AkBsrsXMzoWR", 189 | "colab": {} 190 | }, 191 | "source": [ 192 | "plt.semilogx(history.history[\"lr\"], history.history[\"loss\"])\n", 193 | "plt.axis([1e-8, 1e-4, 0, 30])" 194 | ], 195 | "execution_count": null, 196 | "outputs": [] 197 | }, 198 | { 199 | "cell_type": "code", 200 | "metadata": { 201 | "colab_type": "code", 202 | "id": "4uh-97bpLZCA", 203 | "colab": {} 204 | }, 205 | "source": [ 206 | "tf.keras.backend.clear_session()\n", 207 | "tf.random.set_seed(51)\n", 208 | "np.random.seed(51)\n", 209 | "\n", 210 | "tf.keras.backend.clear_session()\n", 211 | "dataset = windowed_dataset(x_train, window_size, batch_size, shuffle_buffer_size)\n", 212 | "\n", 213 | "model = tf.keras.models.Sequential([\n", 214 | " tf.keras.layers.Lambda(lambda x: tf.expand_dims(x, axis=-1),\n", 215 | " input_shape=[None]),\n", 216 | " tf.keras.layers.Bidirectional(tf.keras.layers.LSTM(32, return_sequences=True)),\n", 217 | " tf.keras.layers.Bidirectional(tf.keras.layers.LSTM(32)),\n", 218 | " tf.keras.layers.Dense(1),\n", 219 | " tf.keras.layers.Lambda(lambda x: x * 100.0)\n", 220 | "])\n", 221 | "\n", 222 | "\n", 223 | "model.compile(loss=\"mse\", optimizer=tf.keras.optimizers.SGD(lr=1e-5, momentum=0.9),metrics=[\"mae\"])\n", 224 | "history = model.fit(dataset,epochs=500,verbose=1)" 225 | ], 226 | "execution_count": null, 227 | "outputs": [] 228 | }, 229 | { 230 | "cell_type": "code", 231 | "metadata": { 232 | "colab_type": "code", 233 | "id": "icGDaND7z0ne", 234 | "colab": {} 235 | }, 236 | "source": [ 237 | "forecast = []\n", 238 | "results = []\n", 239 | "for time in range(len(series) - window_size):\n", 240 | " forecast.append(model.predict(series[time:time + window_size][np.newaxis]))\n", 241 | "\n", 242 | "forecast = forecast[split_time-window_size:]\n", 243 | "results = np.array(forecast)[:, 0, 0]\n", 244 | "\n", 245 | "\n", 246 | "plt.figure(figsize=(10, 6))\n", 247 | "\n", 248 | "plot_series(time_valid, x_valid)\n", 249 | "plot_series(time_valid, results)" 250 | ], 251 | "execution_count": null, 252 | "outputs": [] 253 | }, 254 | { 255 | "cell_type": "code", 256 | "metadata": { 257 | "colab_type": "code", 258 | "id": "KfPeqI7rz4LD", 259 | "colab": {} 260 | }, 261 | "source": [ 262 | "tf.keras.metrics.mean_absolute_error(x_valid, results).numpy()" 263 | ], 264 | "execution_count": null, 265 | "outputs": [] 266 | }, 267 | { 268 | "cell_type": "code", 269 | "metadata": { 270 | "colab_type": "code", 271 | "id": "JUsdZB_tzDLe", 272 | "colab": {} 273 | }, 274 | "source": [ 275 | "import matplotlib.image as mpimg\n", 276 | "import matplotlib.pyplot as plt\n", 277 | "\n", 278 | "#-----------------------------------------------------------\n", 279 | "# Retrieve a list of list results on training and test data\n", 280 | "# sets for each training epoch\n", 281 | "#-----------------------------------------------------------\n", 282 | "mae=history.history['mae']\n", 283 | "loss=history.history['loss']\n", 284 | "\n", 285 | "epochs=range(len(loss)) # Get number of epochs\n", 286 | "\n", 287 | "#------------------------------------------------\n", 288 | "# Plot MAE and Loss\n", 289 | "#------------------------------------------------\n", 290 | "plt.plot(epochs, mae, 'r')\n", 291 | "plt.plot(epochs, loss, 'b')\n", 292 | "plt.title('MAE and Loss')\n", 293 | "plt.xlabel(\"Epochs\")\n", 294 | "plt.ylabel(\"Accuracy\")\n", 295 | "plt.legend([\"MAE\", \"Loss\"])\n", 296 | "\n", 297 | "plt.figure()\n", 298 | "\n", 299 | "epochs_zoom = epochs[200:]\n", 300 | "mae_zoom = mae[200:]\n", 301 | "loss_zoom = loss[200:]\n", 302 | "\n", 303 | "#------------------------------------------------\n", 304 | "# Plot Zoomed MAE and Loss\n", 305 | "#------------------------------------------------\n", 306 | "plt.plot(epochs_zoom, mae_zoom, 'r')\n", 307 | "plt.plot(epochs_zoom, loss_zoom, 'b')\n", 308 | "plt.title('MAE and Loss')\n", 309 | "plt.xlabel(\"Epochs\")\n", 310 | "plt.ylabel(\"Accuracy\")\n", 311 | "plt.legend([\"MAE\", \"Loss\"])\n", 312 | "\n", 313 | "plt.figure()" 314 | ], 315 | "execution_count": null, 316 | "outputs": [] 317 | } 318 | ] 319 | } -------------------------------------------------------------------------------- /4.Sequence, Time Series and Prediction/week3/S+P_Week_3_Exercise_Question.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "nbformat": 4, 3 | "nbformat_minor": 0, 4 | "metadata": { 5 | "accelerator": "GPU", 6 | "colab": { 7 | "name": "S+P Week 3 Exercise Question.ipynb", 8 | "provenance": [], 9 | "collapsed_sections": [], 10 | "toc_visible": true 11 | }, 12 | "kernelspec": { 13 | "display_name": "Python 3", 14 | "name": "python3" 15 | } 16 | }, 17 | "cells": [ 18 | { 19 | "cell_type": "code", 20 | "metadata": { 21 | "id": "zX4Kg8DUTKWO", 22 | "colab_type": "code", 23 | "colab": {} 24 | }, 25 | "source": [ 26 | "#@title Licensed under the Apache License, Version 2.0 (the \"License\");\n", 27 | "# you may not use this file except in compliance with the License.\n", 28 | "# You may obtain a copy of the License at\n", 29 | "#\n", 30 | "# https://www.apache.org/licenses/LICENSE-2.0\n", 31 | "#\n", 32 | "# Unless required by applicable law or agreed to in writing, software\n", 33 | "# distributed under the License is distributed on an \"AS IS\" BASIS,\n", 34 | "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n", 35 | "# See the License for the specific language governing permissions and\n", 36 | "# limitations under the License." 37 | ], 38 | "execution_count": null, 39 | "outputs": [] 40 | }, 41 | { 42 | "cell_type": "code", 43 | "metadata": { 44 | "colab_type": "code", 45 | "id": "D1J15Vh_1Jih", 46 | "colab": {} 47 | }, 48 | "source": [ 49 | "!pip install tf-nightly-2.0-preview\n" 50 | ], 51 | "execution_count": null, 52 | "outputs": [] 53 | }, 54 | { 55 | "cell_type": "code", 56 | "metadata": { 57 | "colab_type": "code", 58 | "id": "BOjujz601HcS", 59 | "colab": {} 60 | }, 61 | "source": [ 62 | "import tensorflow as tf\n", 63 | "import numpy as np\n", 64 | "import matplotlib.pyplot as plt\n", 65 | "print(tf.__version__)" 66 | ], 67 | "execution_count": null, 68 | "outputs": [] 69 | }, 70 | { 71 | "cell_type": "code", 72 | "metadata": { 73 | "colab_type": "code", 74 | "id": "Zswl7jRtGzkk", 75 | "colab": {} 76 | }, 77 | "source": [ 78 | "def plot_series(time, series, format=\"-\", start=0, end=None):\n", 79 | " plt.plot(time[start:end], series[start:end], format)\n", 80 | " plt.xlabel(\"Time\")\n", 81 | " plt.ylabel(\"Value\")\n", 82 | " plt.grid(False)\n", 83 | "\n", 84 | "def trend(time, slope=0):\n", 85 | " return slope * time\n", 86 | "\n", 87 | "def seasonal_pattern(season_time):\n", 88 | " \"\"\"Just an arbitrary pattern, you can change it if you wish\"\"\"\n", 89 | " return np.where(season_time < 0.1,\n", 90 | " np.cos(season_time * 6 * np.pi),\n", 91 | " 2 / np.exp(9 * season_time))\n", 92 | "\n", 93 | "def seasonality(time, period, amplitude=1, phase=0):\n", 94 | " \"\"\"Repeats the same pattern at each period\"\"\"\n", 95 | " season_time = ((time + phase) % period) / period\n", 96 | " return amplitude * seasonal_pattern(season_time)\n", 97 | "\n", 98 | "def noise(time, noise_level=1, seed=None):\n", 99 | " rnd = np.random.RandomState(seed)\n", 100 | " return rnd.randn(len(time)) * noise_level\n", 101 | "\n", 102 | "time = np.arange(10 * 365 + 1, dtype=\"float32\")\n", 103 | "baseline = 10\n", 104 | "series = trend(time, 0.1) \n", 105 | "baseline = 10\n", 106 | "amplitude = 40\n", 107 | "slope = 0.005\n", 108 | "noise_level = 3\n", 109 | "\n", 110 | "# Create the series\n", 111 | "series = baseline + trend(time, slope) + seasonality(time, period=365, amplitude=amplitude)\n", 112 | "# Update with noise\n", 113 | "series += noise(time, noise_level, seed=51)\n", 114 | "\n", 115 | "split_time = 3000\n", 116 | "time_train = time[:split_time]\n", 117 | "x_train = series[:split_time]\n", 118 | "time_valid = time[split_time:]\n", 119 | "x_valid = series[split_time:]\n", 120 | "\n", 121 | "window_size = 20\n", 122 | "batch_size = 32\n", 123 | "shuffle_buffer_size = 1000\n", 124 | "\n", 125 | "plot_series(time, series)" 126 | ], 127 | "execution_count": null, 128 | "outputs": [] 129 | }, 130 | { 131 | "cell_type": "code", 132 | "metadata": { 133 | "colab_type": "code", 134 | "id": "4sTTIOCbyShY", 135 | "colab": {} 136 | }, 137 | "source": [ 138 | "def windowed_dataset(series, window_size, batch_size, shuffle_buffer):\n", 139 | " dataset = tf.data.Dataset.from_tensor_slices(series)\n", 140 | " dataset = dataset.window(window_size + 1, shift=1, drop_remainder=True)\n", 141 | " dataset = dataset.flat_map(lambda window: window.batch(window_size + 1))\n", 142 | " dataset = dataset.shuffle(shuffle_buffer).map(lambda window: (window[:-1], window[-1]))\n", 143 | " dataset = dataset.batch(batch_size).prefetch(1)\n", 144 | " return dataset" 145 | ], 146 | "execution_count": null, 147 | "outputs": [] 148 | }, 149 | { 150 | "cell_type": "code", 151 | "metadata": { 152 | "colab_type": "code", 153 | "id": "A1Hl39rklkLm", 154 | "colab": {} 155 | }, 156 | "source": [ 157 | "tf.keras.backend.clear_session()\n", 158 | "tf.random.set_seed(51)\n", 159 | "np.random.seed(51)\n", 160 | "\n", 161 | "tf.keras.backend.clear_session()\n", 162 | "dataset = windowed_dataset(x_train, window_size, batch_size, shuffle_buffer_size)\n", 163 | "\n", 164 | "model = tf.keras.models.Sequential([\n", 165 | " tf.keras.layers.Lambda(# YOUR CODE HERE),\n", 166 | " # YOUR CODE HERE\n", 167 | " tf.keras.layers.Lambda(# YOUR CODE HERE)\n", 168 | "])\n", 169 | "\n", 170 | "lr_schedule = tf.keras.callbacks.LearningRateScheduler(\n", 171 | " lambda epoch: 1e-8 * 10**(epoch / 20))\n", 172 | "optimizer = tf.keras.optimizers.SGD(lr=1e-8, momentum=0.9)\n", 173 | "model.compile(loss=tf.keras.losses.Huber(),\n", 174 | " optimizer=optimizer,\n", 175 | " metrics=[\"mae\"])\n", 176 | "history = model.fit(dataset, epochs=100, callbacks=[lr_schedule])" 177 | ], 178 | "execution_count": null, 179 | "outputs": [] 180 | }, 181 | { 182 | "cell_type": "code", 183 | "metadata": { 184 | "colab_type": "code", 185 | "id": "AkBsrsXMzoWR", 186 | "colab": {} 187 | }, 188 | "source": [ 189 | "plt.semilogx(history.history[\"lr\"], history.history[\"loss\"])\n", 190 | "plt.axis([1e-8, 1e-4, 0, 30])\n", 191 | "\n", 192 | "# FROM THIS PICK A LEARNING RATE" 193 | ], 194 | "execution_count": null, 195 | "outputs": [] 196 | }, 197 | { 198 | "cell_type": "code", 199 | "metadata": { 200 | "colab_type": "code", 201 | "id": "4uh-97bpLZCA", 202 | "colab": {} 203 | }, 204 | "source": [ 205 | "tf.keras.backend.clear_session()\n", 206 | "tf.random.set_seed(51)\n", 207 | "np.random.seed(51)\n", 208 | "\n", 209 | "tf.keras.backend.clear_session()\n", 210 | "dataset = windowed_dataset(x_train, window_size, batch_size, shuffle_buffer_size)\n", 211 | "\n", 212 | "model = tf.keras.models.Sequential([\n", 213 | " tf.keras.layers.Lambda(# YOUR CODE HERE),\n", 214 | " # YOUR CODE HERE\n", 215 | " tf.keras.layers.Lambda(# YOUR CODE HERE)\n", 216 | "])\n", 217 | "\n", 218 | "model.compile(loss=\"mse\", optimizer=tf.keras.optimizers.SGD(lr=# PUT YOUR LEARNING RATE HERE#, momentum=0.9),metrics=[\"mae\"])\n", 219 | "history = model.fit(dataset,epochs=500,verbose=1)\n", 220 | " \n", 221 | "# FIND A MODEL AND A LR THAT TRAINS TO AN MAE < 3 " 222 | ], 223 | "execution_count": null, 224 | "outputs": [] 225 | }, 226 | { 227 | "cell_type": "code", 228 | "metadata": { 229 | "colab_type": "code", 230 | "id": "icGDaND7z0ne", 231 | "colab": {} 232 | }, 233 | "source": [ 234 | "forecast = []\n", 235 | "results = []\n", 236 | "for time in range(len(series) - window_size):\n", 237 | " forecast.append(model.predict(series[time:time + window_size][np.newaxis]))\n", 238 | "\n", 239 | "forecast = forecast[split_time-window_size:]\n", 240 | "results = np.array(forecast)[:, 0, 0]\n", 241 | "\n", 242 | "\n", 243 | "plt.figure(figsize=(10, 6))\n", 244 | "\n", 245 | "plot_series(time_valid, x_valid)\n", 246 | "plot_series(time_valid, results)" 247 | ], 248 | "execution_count": null, 249 | "outputs": [] 250 | }, 251 | { 252 | "cell_type": "code", 253 | "metadata": { 254 | "colab_type": "code", 255 | "id": "KfPeqI7rz4LD", 256 | "colab": { 257 | "base_uri": "https://localhost:8080/", 258 | "height": 164 259 | }, 260 | "outputId": "5cbee203-d2e8-455e-f603-204d9df993dc" 261 | }, 262 | "source": [ 263 | "tf.keras.metrics.mean_absolute_error(x_valid, results).numpy()\n", 264 | "\n", 265 | "# YOUR RESULT HERE SHOULD BE LESS THAN 4" 266 | ], 267 | "execution_count": null, 268 | "outputs": [ 269 | { 270 | "output_type": "error", 271 | "ename": "NameError", 272 | "evalue": "ignored", 273 | "traceback": [ 274 | "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m", 275 | "\u001b[0;31mNameError\u001b[0m Traceback (most recent call last)", 276 | "\u001b[0;32m\u001b[0m in \u001b[0;36m\u001b[0;34m()\u001b[0m\n\u001b[0;32m----> 1\u001b[0;31m \u001b[0mtf\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mkeras\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mmetrics\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mmean_absolute_error\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mx_valid\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mresults\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mnumpy\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m", 277 | "\u001b[0;31mNameError\u001b[0m: name 'tf' is not defined" 278 | ] 279 | } 280 | ] 281 | }, 282 | { 283 | "cell_type": "code", 284 | "metadata": { 285 | "colab_type": "code", 286 | "id": "JUsdZB_tzDLe", 287 | "colab": {} 288 | }, 289 | "source": [ 290 | "import matplotlib.image as mpimg\n", 291 | "import matplotlib.pyplot as plt\n", 292 | "\n", 293 | "#-----------------------------------------------------------\n", 294 | "# Retrieve a list of list results on training and test data\n", 295 | "# sets for each training epoch\n", 296 | "#-----------------------------------------------------------\n", 297 | "mae=history.history['mae']\n", 298 | "loss=history.history['loss']\n", 299 | "\n", 300 | "epochs=range(len(loss)) # Get number of epochs\n", 301 | "\n", 302 | "#------------------------------------------------\n", 303 | "# Plot MAE and Loss\n", 304 | "#------------------------------------------------\n", 305 | "plt.plot(epochs, mae, 'r')\n", 306 | "plt.plot(epochs, loss, 'b')\n", 307 | "plt.title('MAE and Loss')\n", 308 | "plt.xlabel(\"Epochs\")\n", 309 | "plt.ylabel(\"Accuracy\")\n", 310 | "plt.legend([\"MAE\", \"Loss\"])\n", 311 | "\n", 312 | "plt.figure()\n", 313 | "\n", 314 | "epochs_zoom = epochs[200:]\n", 315 | "mae_zoom = mae[200:]\n", 316 | "loss_zoom = loss[200:]\n", 317 | "\n", 318 | "#------------------------------------------------\n", 319 | "# Plot Zoomed MAE and Loss\n", 320 | "#------------------------------------------------\n", 321 | "plt.plot(epochs_zoom, mae_zoom, 'r')\n", 322 | "plt.plot(epochs_zoom, loss_zoom, 'b')\n", 323 | "plt.title('MAE and Loss')\n", 324 | "plt.xlabel(\"Epochs\")\n", 325 | "plt.ylabel(\"Accuracy\")\n", 326 | "plt.legend([\"MAE\", \"Loss\"])\n", 327 | "\n", 328 | "plt.figure()" 329 | ], 330 | "execution_count": null, 331 | "outputs": [] 332 | } 333 | ] 334 | } -------------------------------------------------------------------------------- /Practise/CNN.py: -------------------------------------------------------------------------------- 1 | import tensorflow as tf 2 | 3 | mnist = tf.keras.datasets.fashion_mnist 4 | (training_images, training_labels), (test_images, test_labels) = mnist.load_data() 5 | 6 | #reshae the image and scale 7 | training_images=training_images.reshape(60000, 28, 28, 1) 8 | training_images=training_images / 255.0 9 | 10 | test_images = test_images.reshape(10000, 28, 28, 1) 11 | test_images=test_images/255.0 12 | 13 | model = tf.keras.models.Sequential([ 14 | tf.keras.layers.Conv2D(64, (3,3), activation='relu', input_shape=(28, 28, 1)), 15 | tf.keras.layers.MaxPooling2D(2, 2), 16 | tf.keras.layers.Conv2D(64, (3,3), activation='relu'), 17 | tf.keras.layers.MaxPooling2D(2,2), 18 | tf.keras.layers.Flatten(), 19 | tf.keras.layers.Dense(128, activation='relu'), 20 | tf.keras.layers.Dense(10, activation='softmax') 21 | ]) 22 | 23 | model.compile(optimizer='adam', loss='sparse_categorical_crossentropy', metrics=['accuracy']) 24 | model.summary() 25 | 26 | model.fit(training_images, training_labels, epochs=5) 27 | 28 | test_loss = model.evaluate(test_images, test_labels) 29 | 30 | 31 | 32 | #warp training in one function 33 | def train_happy_sad_model(): 34 | 35 | DESIRED_ACCURACY = 0.999 36 | 37 | class myCallback(tf.keras.callbacks.Callback): 38 | def on_epoch_end(self, epoch, logs={}): 39 | if logs.get('acc') > DESIRED_ACCURACY: 40 | print("\n Reached training accuracy of >.999, stop training process ... ") 41 | self.model.stop_training = True 42 | 43 | callbacks = myCallback() 44 | 45 | # This Code Block should Define and Compile the Model. Please assume the images are 150 X 150 in your implementation. 46 | model = tf.keras.models.Sequential([ 47 | tf.keras.layers.Conv2D(16,(3,3),activation='relu',input_shape=(150,150,3)), 48 | tf.keras.layers.MaxPooling2D(2,2), 49 | tf.keras.layers.Conv2D(32,(3,3),activation='relu'), 50 | tf.keras.layers.MaxPooling2D(2,2), 51 | tf.keras.layers.Conv2D(64,(3,3),activation='relu'), 52 | tf.keras.layers.MaxPooling2D(2,2), 53 | tf.keras.layers.Flatten(), 54 | tf.keras.layers.Dense(512,activation='relu'), 55 | tf.keras.layers.Dense(1, activation='sigmoid') 56 | 57 | ]) 58 | 59 | from tensorflow.keras.optimizers import RMSprop 60 | model.compile(loss='binary_crossentropy', 61 | optimizer= RMSprop(lr=0.001), 62 | metrics=['acc']) 63 | 64 | 65 | from tensorflow.keras.preprocessing.image import ImageDataGenerator 66 | train_datagen = ImageDataGenerator(rescale=1./255) 67 | 68 | # Please use a target_size of 150 X 150. 69 | train_generator = train_datagen.flow_from_directory( 70 | '/tmp/h-or-s', 71 | target_size=(150,150), 72 | batch_size=10, 73 | class_mode='binary') 74 | 75 | history = model.fit_generator( 76 | train_generator, 77 | steps_per_epoch=8, 78 | epochs = 15, 79 | verbose = 1, 80 | callbacks=[callbacks]) 81 | # model fitting 82 | return history.history['acc'][-1] -------------------------------------------------------------------------------- /Practise/CNN_imageGenerator.py: -------------------------------------------------------------------------------- 1 | #BASH 2 | """ 3 | wget --no-check-certificate \ 4 | https://storage.googleapis.com/laurencemoroney-blog.appspot.com/horse-or-human.zip \ 5 | -O /tmp/horse-or-human.zip 6 | 7 | wget --no-check-certificate \ 8 | https://storage.googleapis.com/laurencemoroney-blog.appspot.com/validation-horse-or-human.zip \ 9 | -O /tmp/validation-horse-or-human.zip 10 | """ 11 | import os 12 | import zipfile 13 | 14 | local_zip = '/tmp/horse-or-human.zip' 15 | zip_ref = zipfile.ZipFile(local_zip, 'r') 16 | zip_ref.extractall('/tmp/horse-or-human') 17 | 18 | local_zip = '/tmp/validation-horse-or-human.zip' 19 | zip_ref = zipfile.ZipFile(local_zip, 'r') 20 | zip_ref.extractall('/tmp/validation-horse-or-human') 21 | 22 | zip_ref.close() 23 | 24 | 25 | # Directory with our training horse pictures 26 | train_horse_dir = os.path.join('/tmp/horse-or-human/horses') 27 | train_human_dir = os.path.join('/tmp/horse-or-human/humans') 28 | 29 | # Directory with our training horse pictures 30 | validation_horse_dir = os.path.join('/tmp/validation-horse-or-human/horses') 31 | validation_human_dir = os.path.join('/tmp/validation-horse-or-human/humans') 32 | 33 | 34 | #Check size of the training and testing dataset 35 | print('total training horse images:', len(os.listdir(train_horse_dir))) 36 | print('total training human images:', len(os.listdir(train_human_dir))) 37 | print('total validation horse images:', len(os.listdir(validation_horse_dir))) 38 | print('total validation human images:', len(os.listdir(validation_human_dir))) 39 | 40 | 41 | import tensorflow as tf 42 | from tensorflow.keras.optimizers import RMSprop 43 | from tensorflow.keras.preprocessing.image import ImageDataGenerator 44 | 45 | model = tf.keras.models.Sequential([ 46 | # Note the input shape is the desired size of the image 300x300 with 3 bytes color 47 | # This is the first convolution 48 | tf.keras.layers.Conv2D(16, (3,3), activation='relu', input_shape=(300, 300, 3)), 49 | tf.keras.layers.MaxPooling2D(2, 2), 50 | # The second convolution 51 | tf.keras.layers.Conv2D(32, (3,3), activation='relu'), 52 | tf.keras.layers.MaxPooling2D(2,2), 53 | # The third convolution 54 | tf.keras.layers.Conv2D(64, (3,3), activation='relu'), 55 | tf.keras.layers.MaxPooling2D(2,2), 56 | # The fourth convolution 57 | tf.keras.layers.Conv2D(64, (3,3), activation='relu'), 58 | tf.keras.layers.MaxPooling2D(2,2), 59 | # The fifth convolution 60 | tf.keras.layers.Conv2D(64, (3,3), activation='relu'), 61 | tf.keras.layers.MaxPooling2D(2,2), 62 | # Flatten the results to feed into a DNN 63 | tf.keras.layers.Flatten(), 64 | # 512 neuron hidden layer 65 | tf.keras.layers.Dense(512, activation='relu'), 66 | # Only 1 output neuron. It will contain a value from 0-1 where 0 for 1 class ('horses') and 1 for the other ('humans') 67 | tf.keras.layers.Dense(1, activation='sigmoid') 68 | ]) 69 | 70 | model.summary() 71 | 72 | model.compile(loss='binary_crossentropy', 73 | optimizer=RMSprop(lr=0.001), 74 | metrics=['accuracy']) 75 | 76 | 77 | # All images will be rescaled by 1./255 78 | train_datagen_ex = ImageDataGenerator(rescale=1/255) 79 | train_datagen = ImageDataGenerator( 80 | rescale=1./255, 81 | rotation_range=40, 82 | width_shift_range=0.2, 83 | height_shift_range=0.2, 84 | shear_range=0.2, 85 | zoom_range=0.2, 86 | horizontal_flip=True, 87 | fill_mode='nearest') 88 | 89 | validation_datagen = ImageDataGenerator(rescale=1/255) 90 | 91 | # Flow training images in batches of 128 using train_datagen generator 92 | train_generator = train_datagen.flow_from_directory( 93 | '/tmp/horse-or-human/', # This is the source directory for training images 94 | target_size=(300, 300), # All images will be resized to 300x300 95 | batch_size=128, 96 | # Since we use binary_crossentropy loss, we need binary labels 97 | class_mode='binary') 98 | 99 | # Flow training images in batches of 128 using train_datagen generator 100 | validation_generator = validation_datagen.flow_from_directory( 101 | '/tmp/validation-horse-or-human/', # This is the source directory for training images 102 | target_size=(300, 300), # All images will be resized to 300x300 103 | batch_size=32, 104 | # Since we use binary_crossentropy loss, we need binary labels 105 | class_mode='binary') 106 | 107 | 108 | history = model.fit( 109 | train_generator, 110 | steps_per_epoch=8, #iamge = batach _size * steps 111 | epochs=15, 112 | verbose=1, 113 | validation_data = validation_generator, 114 | validation_steps=8) 115 | -------------------------------------------------------------------------------- /Practise/CNN_preTrain.py: -------------------------------------------------------------------------------- 1 | import os 2 | from tensorflow.keras import layers 3 | from tensorflow.keras import Model 4 | from tensorflow.keras.optimizers import RMSprop 5 | """ 6 | wget --no-check-certificate \ 7 | https://storage.googleapis.com/mledu-datasets/inception_v3_weights_tf_dim_ordering_tf_kernels_notop.h5 \ 8 | -O /tmp/inception_v3_weights_tf_dim_ordering_tf_kernels_notop.h5 9 | """ 10 | 11 | from tensorflow.keras.applications.inception_v3 import InceptionV3 12 | 13 | local_weights_file = '/tmp/inception_v3_weights_tf_dim_ordering_tf_kernels_notop.h5' 14 | 15 | pre_trained_model = InceptionV3(input_shape = (150, 150, 3), 16 | include_top = False, 17 | weights = None) 18 | 19 | pre_trained_model.load_weights(local_weights_file) 20 | 21 | for layer in pre_trained_model.layers: 22 | layer.trainable = False 23 | 24 | # pre_trained_model.summary() 25 | 26 | last_layer = pre_trained_model.get_layer('mixed7') 27 | print('last layer output shape: ', last_layer.output_shape) 28 | last_output = last_layer.output 29 | 30 | """ 31 | Conv2D 32 | BatchNormalization 33 | Activation 34 | MaxPooling2D 35 | AveragePooling2D 36 | 37 | """ 38 | 39 | # Flatten the output layer to 1 dimension 40 | x = layers.Flatten()(last_output) 41 | # Add a fully connected layer with 1,024 hidden units and ReLU activation 42 | x = layers.Dense(1024, activation='relu')(x) 43 | # Add a dropout rate of 0.2 44 | x = layers.Dropout(0.2)(x) 45 | # Add a final sigmoid layer for classification 46 | x = layers.Dense(1, activation='sigmoid')(x) 47 | 48 | model = Model( pre_trained_model.input, x) 49 | 50 | model.compile(optimizer = RMSprop(lr=0.0001), 51 | loss = 'binary_crossentropy', 52 | metrics = ['accuracy']) 53 | 54 | 55 | 56 | """ 57 | wget --no-check-certificate \ 58 | https://storage.googleapis.com/mledu-datasets/cats_and_dogs_filtered.zip \ 59 | -O /tmp/cats_and_dogs_filtered.zip 60 | """ 61 | from tensorflow.keras.preprocessing.image import ImageDataGenerator 62 | import zipfile 63 | 64 | local_zip = '//tmp/cats_and_dogs_filtered.zip' 65 | zip_ref = zipfile.ZipFile(local_zip, 'r') 66 | zip_ref.extractall('/tmp') 67 | zip_ref.close() 68 | 69 | # Define our example directories and files 70 | base_dir = '/tmp/cats_and_dogs_filtered' 71 | 72 | train_dir = os.path.join( base_dir, 'train') 73 | validation_dir = os.path.join( base_dir, 'validation') 74 | 75 | 76 | train_cats_dir = os.path.join(train_dir, 'cats') # Directory with our training cat pictures 77 | train_dogs_dir = os.path.join(train_dir, 'dogs') # Directory with our training dog pictures 78 | validation_cats_dir = os.path.join(validation_dir, 'cats') # Directory with our validation cat pictures 79 | validation_dogs_dir = os.path.join(validation_dir, 'dogs')# Directory with our validation dog pictures 80 | 81 | train_cat_fnames = os.listdir(train_cats_dir) 82 | train_dog_fnames = os.listdir(train_dogs_dir) 83 | 84 | # Add our data-augmentation parameters to ImageDataGenerator 85 | train_datagen = ImageDataGenerator(rescale = 1./255., 86 | rotation_range = 40, 87 | width_shift_range = 0.2, 88 | height_shift_range = 0.2, 89 | shear_range = 0.2, 90 | zoom_range = 0.2, 91 | horizontal_flip = True) 92 | 93 | # Note that the validation data should not be augmented! 94 | test_datagen = ImageDataGenerator( rescale = 1.0/255. ) 95 | 96 | # Flow training images in batches of 20 using train_datagen generator 97 | train_generator = train_datagen.flow_from_directory(train_dir, 98 | batch_size = 20, 99 | class_mode = 'binary', 100 | target_size = (150, 150)) 101 | 102 | # Flow validation images in batches of 20 using test_datagen generator 103 | validation_generator = test_datagen.flow_from_directory( validation_dir, 104 | batch_size = 20, 105 | class_mode = 'binary', 106 | target_size = (150, 150)) 107 | 108 | 109 | history = model.fit( 110 | train_generator, 111 | validation_data = validation_generator, 112 | steps_per_epoch = 100, 113 | epochs = 20, 114 | validation_steps = 50, 115 | verbose = 2) 116 | 117 | #VISUAL 118 | import matplotlib.pyplot as plt 119 | acc = history.history['accuracy'] 120 | val_acc = history.history['val_accuracy'] 121 | loss = history.history['loss'] 122 | val_loss = history.history['val_loss'] 123 | 124 | epochs = range(len(acc)) 125 | 126 | plt.plot(epochs, acc, 'r', label='Training accuracy') 127 | plt.plot(epochs, val_acc, 'b', label='Validation accuracy') 128 | plt.title('Training and validation accuracy') 129 | plt.legend(loc=0) 130 | plt.figure() 131 | 132 | 133 | plt.show() 134 | 135 | 136 | -------------------------------------------------------------------------------- /Practise/Natural_language_processing.py: -------------------------------------------------------------------------------- 1 | import tensorflow as tf 2 | import tensorflow_datasets as tfds 3 | 4 | #import the data 5 | imdb, info = tfds.load("imdb_reviews/subwords8k", with_inf= True, as_supervised= True) 6 | train_data , test_data = imdb['train'] , imdb['test'] 7 | 8 | 9 | #prep the data 10 | 11 | BUFFER_SIZE = 10000 12 | BATCH_SIZE = 64 13 | 14 | train_dataset = train_data.shuffle(BUFFER_SIZE) 15 | train_dataset = train_dataset.padded_batch(BATCH_SIZE, tf.compat.v1.data.get_output_shapes(train_dataset)) 16 | test_dataset = train_dataset.padded_batch(BATCH_SIZE, tf.compat.v1.data.get_output_shapes(test_data)) 17 | 18 | #Text(shape=(None,), dtype=tf.int64, encoder=) 19 | tokenizer = info.features['text'].encoder 20 | 21 | #build model 22 | embedding_dim = 64 23 | 24 | model = tf.keras.Sequential([ 25 | tf.keras.layers.Embedding(tokenizer.vocab_size, embedding_dim), 26 | tf.keras.layers.GlobalAveragePooling1D(), 27 | tf.keras.layers.Dense(6, activation='relu'), 28 | tf.keras.layers.Dense(1, activation='sigmoid') 29 | ]) 30 | 31 | model.summary() 32 | 33 | #training 34 | num_epochs = 10 35 | 36 | model.complie(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy']) 37 | history = model.fit(train_dataset, epochs = num_epochs, validation_data = test_dataset) 38 | 39 | 40 | #visulisation 41 | import matplotlib.pyplot as plt 42 | 43 | 44 | def plot_graphs(history, string): 45 | plt.plot(history.history[string]) 46 | plt.plot(history.history['val_'+string]) 47 | plt.xlabel("Epochs") 48 | plt.ylabel(string) 49 | plt.legend([string, 'val_'+string]) 50 | plt.show() 51 | 52 | plot_graphs(history, "accuracy") 53 | plot_graphs(history, "loss") 54 | 55 | -------------------------------------------------------------------------------- /Practise/Natural_language_processing_1.py: -------------------------------------------------------------------------------- 1 | import tensorflow as tf 2 | import tensorflow_datasets as tfds 3 | from tensorflow.keras.preprocessing.text import Tokenizer 4 | from tensorflow.keras.preprocessing.sequence import pad_sequences 5 | 6 | 7 | #import the data 8 | imdb, info = tfds.load("imdb_reviews/subwords8k", with_inf= True, as_supervised= True) 9 | train_data , c = imdb['train'] , imdb['test'] 10 | 11 | training_sentences = [] 12 | training_labels = [] 13 | 14 | testing_sentences = [] 15 | testing_labels = [] 16 | 17 | for s, l in train_data: 18 | training_sentences.append(s.numpy().decode('utf8')) 19 | training_labels.append(l.numpy()) 20 | 21 | for s, l in test_data: 22 | testing_sentence.append(s.numpy().decode('utf8')) 23 | training_labels.append(l.numpy()) 24 | 25 | training_labels_final = np.array(training_labels) 26 | testing_labels_final = np.array(testing_labels) 27 | 28 | 29 | #traing the tokenizer 30 | vocab_size = 10000 31 | max_length = 120 # can try different value 100 32 | #padding_type = 'post' 33 | trunc_type = 'post' 34 | cov_tok = '' 35 | 36 | tokenizer = Tokenizer(num_words=vocab_size, oov_token =cov_tok) 37 | tokenizer.fit_on_texts(training_sentences) 38 | 39 | training_sequences = tokenizer.texts_to_sequences(training_sentences) 40 | training_padded = pad_sequences(sequences, maxlen = max_length, truncating = trunc_type) 41 | 42 | testing_sequences = tokenizer.texts_to_sequences(testing_sentences) 43 | testing_padded = pad_sequences(sequences, maxlen = max_length) 44 | 45 | #model 46 | embedding_dim = 16 #try different 32, 64 47 | 48 | model = tf.keras.Sequential([ 49 | tf.keras.layers.Embedding(vocab_size, embedding_dim, input_length = max_length), 50 | tf.keras.layers.Flattern(), 51 | tf.keras.layers.Dense(6, activation='relu'), #try different , not improving result 52 | tf.keras.layers.Dense(1, activation='sigmoid') 53 | ]) 54 | 55 | model.compile(loss='binary_crossentropy', optimizer = 'adam', metrics =['accuracy']) 56 | 57 | model.summary() 58 | 59 | #training 60 | num_epochs = 10 61 | model.fit(training_padded, training_labels_final, epochs = num_epochs, validation_data = (testing_padded, testing_labels_final)) 62 | 63 | #Investgation 64 | e = model.layers[0] 65 | weights = e.get_weights()[0] 66 | print(weights.shape) 67 | 68 | 69 | 70 | 71 | 72 | 73 | -------------------------------------------------------------------------------- /Practise/Natural_language_processing_lstm.py: -------------------------------------------------------------------------------- 1 | import tensorflow_datasets as tfds 2 | import tensorflow as tf 3 | 4 | dataset, info = tfds.load('imdb_reviews/subwords8k', with_info=True, as_supervised=True) 5 | train_dataset, test_dataset = dataset['train'], dataset['test'] 6 | 7 | tokenizer = info.features['text'].encoder 8 | 9 | FFER_SIZE = 10000 10 | BATCH_SIZE = 64 11 | 12 | train_dataset = train_dataset.shuffle(BUFFER_SIZE) 13 | train_dataset = train_dataset.padded_batch(BATCH_SIZE, tf.compat.v1.data.get_output_shapes(train_dataset)) 14 | test_dataset = test_dataset.padded_batch(BATCH_SIZE, tf.compat.v1.data.get_output_shapes(test_dataset)) 15 | 16 | #0.8211 17 | model = tf.keras.Sequential([ 18 | tf.keras.layers.Embedding(tokenizer.vocab_size, 64), 19 | tf.keras.layers.Bidirectional(tf.keras.layers.LSTM(64)), 20 | tf.keras.layers.Dense(64, activation='relu'), 21 | tf.keras.layers.Dense(1, activation='sigmoid') 22 | ]) 23 | 24 | #0.8631 25 | model_lstm = tf.keras.Sequential([ 26 | tf.keras.layers.Embedding(tokenizer.vocab_size, 64), 27 | tf.keras.layers.Bidirectional(tf.keras.layers.LSTM(64, return_sequences = True)), 28 | tf.keras.layers.Bidirectional(tf.keras.layers.LSTM(32)), 29 | tf.keras.layers.Dense(64, activation = 'relu'), 30 | tf.keras.layers.Dense(1, activation = 'sigmoid') 31 | ]) 32 | 33 | model = tf.keras.Sequential([ 34 | tf.keras.layers.Embedding(tokenizer.vocab_size, 64), 35 | tf.keras.layers.Conv1D(128, 5, activation='relu'), 36 | tf.keras.layers.GlobalAveragePooling1D(), 37 | tf.keras.layers.Dense(64, activation='relu'), 38 | tf.keras.layers.Dense(1, activation='sigmoid') 39 | ]) 40 | 41 | model.summary() 42 | 43 | model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy']) 44 | 45 | NUM_EPOCHS = 10 46 | history = model.fit(train_dataset, epochs=NUM_EPOCHS, validation_data = test_dataset) 47 | 48 | -------------------------------------------------------------------------------- /Practise/Natural_language_processing_lstm_1.py: -------------------------------------------------------------------------------- 1 | import numpy as np 2 | import json 3 | import tensorflow as tf 4 | 5 | from tensorflow.keras.preprocessing.text import Tokenizer 6 | from tensorflow.keras.preprocessing.sequence import pad_sequences 7 | 8 | 9 | #downloading data from google storge to local 10 | !wget --no-check-certificate \ 11 | https://storage.googleapis.com/laurencemoroney-blog.appspot.com/sarcasm.json \ 12 | -O /tmp/sarcasm.json 13 | 14 | #setting the parameters 15 | vocab_size = 1000 #change vocab 10000 16 | embedding_dim = 16 17 | max_length = 120 18 | trunc_type='post' 19 | padding_type='post' 20 | oov_tok = "" 21 | training_size = 20000 22 | 23 | #building training and testing dataset 24 | with open("/tmp/sarcasm.json", 'r') as f: 25 | datastore = json.load(f) 26 | 27 | sentences = [] 28 | labels = [] 29 | urls = [] 30 | for item in datastore: 31 | sentences.append(item['headline']) 32 | labels.append(item['is_sarcastic']) 33 | 34 | training_sentences = sentences[0:training_size] 35 | testing_sentences = sentences[training_size:] 36 | training_labels = labels[0:training_size] 37 | testing_labels = labels[training_size:] 38 | 39 | #training tokenizer 40 | tokenizer = Tokenizer(num_words=vocab_size, oov_token=oov_tok) 41 | tokenizer.fit_on_texts(training_sentences) 42 | 43 | word_index = tokenizer.word_index 44 | 45 | training_sequences = tokenizer.texts_to_sequences(training_sentences) 46 | training_padded = pad_sequences(training_sequences, maxlen=max_length, padding=padding_type, truncating=trunc_type) 47 | 48 | testing_sequences = tokenizer.texts_to_sequences(testing_sentences) 49 | testing_padded = pad_sequences(testing_sequences, maxlen=max_length, padding=padding_type, truncating=trunc_type) 50 | 51 | #prep data for tensorflow 2 52 | training_padded = np.array(training_padded) 53 | training_labels = np.array(training_labels) 54 | testing_padded = np.array(testing_padded) 55 | testing_labels = np.array(testing_labels) 56 | 57 | 58 | 59 | model = tf.keras.Sequential([ 60 | tf.keras.layers.Embedding(vocab_size, embedding_dim, input_length=max_length), 61 | tf.keras.layers.Bidirectional(tf.keras.layers.LSTM(32)), 62 | tf.keras.layers.Dense(24, activation='relu'), 63 | tf.keras.layers.Dense(1, activation='sigmoid') 64 | ]) 65 | 66 | model_1 = tf.keras.Sequential([ 67 | tf.keras.layers.Embedding(vocab_size, embedding_dim, input_length=max_length), 68 | tf.keras.layers.Conv1D(128, 5, activation='relu'), 69 | tf.keras.layers.GlobalMaxPooling1D(), 70 | tf.keras.layers.Dense(24, activation='relu'), 71 | tf.keras.layers.Dense(1, activation='sigmoid') 72 | ]) 73 | 74 | model_3 = tf.keras.Sequential([ 75 | tf.keras.layers.Embedding(vocab_size, embedding_dim, input_length=max_length), 76 | tf.keras.layers.Bidirectional(tf.keras.layers.GRU(32)), 77 | tf.keras.layers.Dense(6, activation='relu'), 78 | tf.keras.layers.Dense(1, activation='sigmoid') 79 | ]) 80 | 81 | model.compile(loss='binary_crossentropy',optimizer='adam',metrics=['accuracy']) 82 | model.summary() 83 | 84 | num_epochs = 50 85 | history = model.fit(training_padded, training_labels, epochs=num_epochs, validation_data=(testing_padded, testing_labels), verbose=1) 86 | 87 | 88 | #plot the accuracy curve 89 | 90 | import matplotlib.pyplot as plt 91 | 92 | 93 | def plot_graphs(history, string): 94 | plt.plot(history.history[string]) 95 | plt.plot(history.history['val_'+string]) 96 | plt.xlabel("Epochs") 97 | plt.ylabel(string) 98 | plt.legend([string, 'val_'+string]) 99 | plt.show() 100 | 101 | plot_graphs(history, 'accuracy') 102 | plot_graphs(history, 'loss') 103 | 104 | #save model 105 | model.save("test.h5") 106 | 107 | 108 | 109 | 110 | 111 | -------------------------------------------------------------------------------- /Practise/Natural_language_processing_ngram.py: -------------------------------------------------------------------------------- 1 | import tensorflow as tf 2 | 3 | from tensorflow.keras.preprocessing.sequence import pad_sequences 4 | from tensorflow.keras.layers import Embedding, LSTM, Dense, Bidirectional 5 | from tensorflow.keras.preprocessing.text import Tokenizer 6 | from tensorflow.keras.models import Sequential 7 | from tensorflow.keras.optimizers import Adam 8 | import numpy as np 9 | 10 | tokenizer = Tokenizer() 11 | 12 | data = open('/tmp/irish-lyrics-eof.txt').read() 13 | 14 | corpus = data.lower().split('\n') 15 | 16 | tokenizer.fit_on_text(corpus) 17 | totoal_words = len(tokenizer.word_index) +1 18 | 19 | input_sequences = [] 20 | 21 | #build ngram tokens 22 | for line in corpurs: 23 | token_list = tokenizer.texts_to_sequence([line])[0] 24 | for i in range(1, len(token_list)): 25 | n_gram_sequence = token_list[:i+1] 26 | input_sequences.append(n_gram_sequence) 27 | 28 | #pad the input sequence up to the max length 29 | max_sequence_len = max([len(x) for x in input_sequences]) 30 | input_sequence = np.array(pad_sequences(input_sequence, maxlen = max_sequence_len, padding= 'pre')) 31 | 32 | #build training, validation and labels 33 | xs , labels = input_sequence[:,:-1], input_sequence[:,-1] 34 | ys = tf.keras.utils.to_categorical(labels, num_classes=total_words) 35 | 36 | 37 | model = Sequential() 38 | model.add(Embedding(total_words, 100, input_length=max_sequence_len-1)) 39 | model.add(Bidirectional(LSTM(150))) 40 | model.add(Dense(total_words, activation='softmax')) 41 | adam = Adam(lr=0.01) 42 | model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy']) 43 | callback = tf.keras.callbacks.EarlyStopping(monitor='loss', min_delta=0, patience=5, verbose=0, mode='auto') 44 | history = model.fit(xs, ys, epochs=500, verbose=1) 45 | 46 | 47 | 48 | -------------------------------------------------------------------------------- /Practise/Natural_language_processing_text_generation.py: -------------------------------------------------------------------------------- 1 | import tensorflow as tf 2 | 3 | from tensorflow.keras.preprocessing.sequence import pad_sequences 4 | from tensorflow.keras.layers import Embedding, LSTM, Dense, Bidirectional 5 | from tensorflow.keras.preprocessing.text import Tokenizer 6 | from tensorflow.keras.models import Sequential 7 | from tensorflow.keras.optimizers import Adam 8 | import numpy as np 9 | 10 | tokenizer = Tokenizer() 11 | 12 | data = open('/tmp/irish-lyrics-eof.txt').read() 13 | 14 | corpus = data.lower().split('\n') 15 | 16 | tokenizer.fit_on_text(corpus) 17 | totoal_words = len(tokenizer.word_index) +1 18 | 19 | input_sequences = [] 20 | 21 | #build ngram tokens 22 | for line in corpurs: 23 | token_list = tokenizer.texts_to_sequence([line])[0] 24 | for i in range(1, len(token_list)): 25 | n_gram_sequence = token_list[:i+1] 26 | input_sequences.append(n_gram_sequence) 27 | 28 | max_sequence_len = max([len(x) for x in input_sequences]) 29 | input_sequence = np.array(pad_sequences(input_sequence, maxlen = max_sequence_len, padding= 'pre')) 30 | 31 | xs , labels = input_sequence[:,:-1], input_sequence[:,-1] 32 | ys = tf.keras.utils.to_categorical(labels, num_classes=total_words) 33 | 34 | model = Sequential() 35 | model.add(Embedding(total_words, 100, input_length=max_sequence_len-1)) 36 | model.add(Bidirectional(LSTM(150))) 37 | model.add(Dense(total_words, activation='softmax')) 38 | adam = Adam(lr=0.01) 39 | model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy']) 40 | callback = tf.keras.callbacks.EarlyStopping(monitor='loss', min_delta=0, patience=5, verbose=0, mode='auto') 41 | history = model.fit(xs, ys, epochs=500, verbose=1) 42 | -------------------------------------------------------------------------------- /Practise/time_series_2.py: -------------------------------------------------------------------------------- 1 | import tensorflow as tf 2 | 3 | import numpy as np 4 | import matplotlib.pyplot as plt 5 | 6 | print(tf.__version__) 7 | 8 | 9 | def plot_series(time, series, format="-", start=0, end=None): 10 | plt.plot(time[start:end], series[start:end], format) 11 | plt.xlabel("Time") 12 | plt.ylabel("Value") 13 | plt.grid(True) 14 | 15 | 16 | def trend(time, slope: int = 0) -> int: 17 | return slope * time 18 | 19 | 20 | def seasonal_pattern(season_time): 21 | """Just an arbitrary pattern, you can change it if you wish""" 22 | return np.where(season_time < 0.4, 23 | np.cos(season_time * 2 * np.pi), 24 | 1 / np.exp(3 * season_time)) 25 | 26 | 27 | def seasonality(time, period, amplitude=1, phase=0): 28 | """Repeats the same pattern at each period""" 29 | season_time = ((time + phase) % period) / period 30 | return amplitude * seasonal_pattern(season_time) 31 | 32 | 33 | def noise(time, noise_level=1, seed=None): 34 | rnd = np.random.RandomState(seed) 35 | return rnd.randn(len(time)) * noise_level 36 | 37 | 38 | # Build training set 39 | time = np.arange(4 * 365 + 1, dtype="float32") 40 | baseline = 10 41 | amplitude = 40 42 | slope = 0.05 43 | noise_level = 5 44 | 45 | series = baseline + trend(time, slope) + seasonality(time, period=365, amplitude=amplitude) 46 | series += noise(time, noise_level, seed=42) 47 | 48 | split_time = 1000 49 | time_train = time[:split_time] 50 | x_train = series[:split_time] 51 | time_valid = time[split_time:] 52 | x_valid = series[split_time:] 53 | 54 | # plot training set 55 | plt.figure(figsize=(10, 6)) 56 | plot_series(time_valid, x_valid) 57 | 58 | 59 | def windowed_dataset(series, window_size, batch_size, shuffle_buffer): 60 | dataset = tf.data.Dataset.from_tensor_slices(series) 61 | dataset = dataset.window(window_size + 1, shift=1, drop_remainder=True) 62 | dataset = dataset.flat_map(lambda window: window.batch(window_size + 1)) 63 | dataset = dataset.shuffle(shuffle_buffer).map(lambda window: (window[:-1], window[-1])) 64 | dataset = dataset.batch(batch_size).prefetch(1) 65 | 66 | return dataset 67 | 68 | 69 | # building model 70 | window_size = 20 71 | batch_size = 32 72 | shuffle_buffer_size = 1000 73 | dataset = windowed_dataset(x_train, window_size, batch_size, shuffle_buffer_size) 74 | 75 | model = tf.keras.models.Sequential([ 76 | tf.keras.layers.Dense(10, input_shape=[window_size], activation="relu"), 77 | tf.keras.layers.Dense(10, activation="relu"), 78 | tf.keras.layers.Dense(1) 79 | ]) 80 | 81 | lr_schedule = tf.keras.callbacks.LearningRateScheduler( 82 | lambda epoch: 1e-8 * 10 ** (epoch / 20)) 83 | optimizer = tf.keras.optimizers.SGD(lr=1e-8, momentum=0.9) 84 | 85 | model.compile(loss="mse", optimizer=optimizer) 86 | history = model.fit(dataset, epochs=100, callbacks=[lr_schedule], verbose=1) 87 | 88 | # Find the optimal learning rate 89 | lrs = 1e-8 * (10 ** (np.arange(100) / 20)) 90 | plt.semilogx(lrs, history.history["loss"]) 91 | plt.axis([1e-8, 1e-3, 0, 300]) 92 | 93 | 94 | # Train model with new learning rate 95 | window_size = 30 96 | dataset = windowed_dataset(x_train, window_size, batch_size, shuffle_buffer_size) 97 | 98 | model = tf.keras.models.Sequential([ 99 | tf.keras.layers.Dense(10, input_shape=[window_size], activation="relu"), 100 | tf.keras.layers.Dense(10, activation="relu"), 101 | tf.keras.layers.Dense(1) 102 | ]) 103 | 104 | optimizer = tf.keras.optimizers.SGD(lr=8e-6, momentum=0.9) 105 | model.compile(loss='mse', optimizer= optimizer) 106 | history = model.fit(dataset, epochs=500, verbose=1) 107 | 108 | loss = history.history['loss'] 109 | epochs = range(len(loss)) 110 | plt.plot(epochs, loss, 'b', label='Training Loss') 111 | plt.show() 112 | 113 | 114 | # Plot all but the first 10 115 | loss = history.history['loss'] 116 | epochs = range(10, len(loss)) 117 | plot_loss = loss[10:] 118 | print(plot_loss) 119 | plt.plot(epochs, plot_loss, 'b', label='Training Loss') 120 | plt.show() 121 | 122 | 123 | # Plot the result 124 | forecast = [] 125 | for time in range(len(series) - window_size): 126 | forecast.append(model.predict(series[time:time + window_size][np.newaxis])) 127 | 128 | forecast = forecast[split_time - window_size:] 129 | results = np.array(forecast)[:, 0, 0] 130 | 131 | plt.figure(figsize=(10, 6)) 132 | 133 | plot_series(time_valid, x_valid) 134 | plot_series(time_valid, results) 135 | 136 | tf.keras.metrics.mean_absolute_error(x_valid, results).numpy() 137 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Tensorflow-developer 2 | The prep for getting a Tensorflow Developer Certificate 3 | 4 | * TensorFlow in Practice Specialization from Coursera 5 | 1. Introduction to TensorFlow for Artificial Intelligence, Machine Learning, and Deep Learning 6 | 2. Converlutional Neural Networks in Tensorflow 7 | 3. Natural Language Processing in TensorFlow 8 | 4. Sequences, Time Series and Prediction 9 | 10 | * Practise 11 | --------------------------------------------------------------------------------