├── 00_test_install.ipynb ├── 01_linear_regression.ipynb ├── 02_tensorflow_logistic_regression.ipynb ├── 03a_tensorflow_deep_network.ipynb ├── 03b_deep_mnist_visualize.ipynb ├── 04_mnist_cnn.ipynb ├── 05_data_prep.ipynb ├── 06_transfer_learning.ipynb ├── startup └── startup.sh └── transfer_learning_data.ipynb /00_test_install.ipynb: -------------------------------------------------------------------------------- 1 | {"nbformat":4,"nbformat_minor":0,"metadata":{"colab":{"name":"00_test_install.ipynb","provenance":[{"file_id":"https://github.com/edge-ai-vision/dlcvtf2/blob/master/00_test_install.ipynb","timestamp":1591898281644}]},"kernelspec":{"name":"python3","display_name":"Python 3"},"accelerator":"GPU"},"cells":[{"cell_type":"markdown","metadata":{"id":"I-S9EwVjCGwX","colab_type":"text"},"source":["# Deep Learning for Computer Vision with TensorFlow 2.0\n","\n","Table of contents:\n","[Lab 0](https://colab.research.google.com/github/edge-ai-vision/dlcvtf2/blob/master/00_test_install.ipynb) | \n","[Lab 1](https://colab.research.google.com/github/edge-ai-vision/dlcvtf2/blob/master/01_linear_regression.ipynb) | \n","[Lab 2](https://colab.research.google.com/github/edge-ai-vision/dlcvtf2/blob/master/02_tensorflow_logistic_regression.ipynb) | \n","[Lab 3a](https://colab.research.google.com/github/edge-ai-vision/dlcvtf2/blob/master/03a_tensorflow_deep_network.ipynb) | \n","[Lab 3b](https://colab.research.google.com/github/edge-ai-vision/dlcvtf2/blob/master/03b_deep_mnist_visualize.ipynb) | \n","[Lab 4](https://colab.research.google.com/github/edge-ai-vision/dlcvtf2/blob/master/04_mnist_cnn.ipynb) | \n","[Lab 5](https://colab.research.google.com/github/edge-ai-vision/dlcvtf2/blob/master/05_data_prep.ipynb) | \n","[Lab 6](https://colab.research.google.com/github/edge-ai-vision/dlcvtf2/blob/master/06_transfer_learning.ipynb) | \n","\n","# Lab 0: Basic Setup and Runtime Instance Configuration\n","\n","This notebook is used to make sure that your Google Colab TensorFlow environment is set up properly.\n","\n","**Important: before you go further, go to the Runtime menu and select \"Change runtime type\" to select GPU.**\n","\n","You can press `shift + enter` to quickly advance through each line of a notebook. Try it! (The arrow just left of the square box will also run a cell if you want to do it that way.)\n","\n","If you ever want to start over just go to the menu at the top and select Runtime -> Restart runtime. (Please do not use Runtime -> Reset all runtimes.)"]},{"cell_type":"markdown","metadata":{"id":"ABCPyQdxCGwZ","colab_type":"text"},"source":["This first cell makes sure that you have TensorFlow 2.0 installed, and imports some needed libraries."]},{"cell_type":"code","metadata":{"id":"WWekDRKhCGwa","colab_type":"code","colab":{}},"source":["# Cell 0.1\n","%tensorflow_version 2.x\n","import tensorflow as tf\n","\n","import numpy as np\n","from tensorflow import keras\n","from tensorflow.keras import layers\n","\n","# A special command for IPython Notebooks that\n","# intructs Matplotlib to display plots in the notebook\n","%pylab inline\n","\n","print(\"You have version %s\" % tf.__version__)\n","\n","print ('cell finished')\n","\n"],"execution_count":0,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"d4GKPhQOCGwe","colab_type":"text"},"source":["Check if Matplotlib is working. After running this cell, you should see a plot appear below."]},{"cell_type":"code","metadata":{"id":"Wh97WnaRCGwf","colab_type":"code","colab":{}},"source":["# Cell 0.2\n","%pylab inline\n","\n","import numpy as np\n","\n","# create some data using numpy. y = x * 0.1 + 0.3 + noise\n","x = np.random.rand(100).astype(np.float32)\n","noise = np.random.normal(scale=0.01, size=len(x))\n","y = x * 0.1 + 0.3 + noise\n","\n","# plot it\n","pylab.plot(x, y, '.')"],"execution_count":0,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"Cf2DTWYfCGwh","colab_type":"text"},"source":["Check if Numpy and Pillow are working. After running this cell, you should see a random image appear below."]},{"cell_type":"code","metadata":{"id":"hnUNHK3OCGwi","colab_type":"code","colab":{}},"source":["# Cell 0.3\n","import PIL.Image as Image\n","import numpy as np\n","from matplotlib.pyplot import imshow\n","\n","image_array = np.random.rand(200,200,3) * 255\n","img = Image.fromarray(image_array.astype('uint8')).convert('RGBA')\n","imshow(np.asarray(img))"],"execution_count":0,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"epZbmVwQCGwo","colab_type":"text"},"source":["That's it! You're ready to start the training!"]},{"cell_type":"code","metadata":{"id":"Nvil6RqwCGwp","colab_type":"code","colab":{}},"source":[""],"execution_count":0,"outputs":[]}]} 2 | -------------------------------------------------------------------------------- /01_linear_regression.ipynb: -------------------------------------------------------------------------------- 1 | {"nbformat":4,"nbformat_minor":0,"metadata":{"colab":{"name":"01_linear_regression.ipynb","provenance":[{"file_id":"https://github.com/edge-ai-vision/dlcvtf2/blob/master/01_linear_regression.ipynb","timestamp":1591898427347}],"collapsed_sections":[]},"kernelspec":{"name":"python3","display_name":"Python 3"},"accelerator":"GPU"},"cells":[{"cell_type":"markdown","metadata":{"id":"I-S9EwVjCGwX","colab_type":"text"},"source":["# Deep Learning for Computer Vision with TensorFlow 2.0\n","\n","Table of contents:\n","[Lab 0](https://colab.research.google.com/github/edge-ai-vision/dlcvtf2/blob/master/00_test_install.ipynb) | \n","[Lab 1](https://colab.research.google.com/github/edge-ai-vision/dlcvtf2/blob/master/01_linear_regression.ipynb) | \n","[Lab 2](https://colab.research.google.com/github/edge-ai-vision/dlcvtf2/blob/master/02_tensorflow_logistic_regression.ipynb) | \n","[Lab 3a](https://colab.research.google.com/github/edge-ai-vision/dlcvtf2/blob/master/03a_tensorflow_deep_network.ipynb) | \n","[Lab 3b](https://colab.research.google.com/github/edge-ai-vision/dlcvtf2/blob/master/03b_deep_mnist_visualize.ipynb) | \n","[Lab 4](https://colab.research.google.com/github/edge-ai-vision/dlcvtf2/blob/master/04_mnist_cnn.ipynb) | \n","[Lab 5](https://colab.research.google.com/github/edge-ai-vision/dlcvtf2/blob/master/05_data_prep.ipynb) | \n","[Lab 6](https://colab.research.google.com/github/edge-ai-vision/dlcvtf2/blob/master/06_transfer_learning.ipynb) | \n","\n","# Lab 1: Linear Regression\n","\n","If you ever want to start over just go to the menu at the top and select Runtime -> Restart runtime. (Please do not use Runtime -> Reset all runtimes.)"]},{"cell_type":"markdown","metadata":{"id":"K8OIW124_xS6","colab_type":"text"},"source":["In this exerise you will apply what you learned in the lecture to experiment with a working linear regression model. The model will be very similar to the one discussed in the lecture. You will run the Colab cells below, to create the model, train the model, and test the output."]},{"cell_type":"markdown","metadata":{"id":"veeOvh1NAO-E","colab_type":"text"},"source":["This first cell makes sure that you have TensorFlow 2.0 installed, and imports needed libraries."]},{"cell_type":"code","metadata":{"id":"ZIVGU2V5_xS8","colab_type":"code","colab":{}},"source":["# Cell 1.1\n","%tensorflow_version 2.x\n","import tensorflow as tf\n","\n","import numpy as np\n","from tensorflow import keras\n","from tensorflow.keras import layers\n","\n","# A special command for IPython Notebooks that\n","# intructs Matplotlib to display plots in the notebook\n","%pylab inline\n","\n","print ('cell finished')"],"execution_count":0,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"vGNCUm_u7UKF","colab_type":"text"},"source":["This cell checks to make sure you are running on a GPU colab instance. If you don't, you'll need to change the runtime type."]},{"cell_type":"code","metadata":{"id":"dyFqajrM7PXY","colab_type":"code","colab":{}},"source":["# Cell 1.2\n","device_name = tf.test.gpu_device_name()\n","if device_name != '/device:GPU:0':\n"," raise SystemError('GPU not found. Please go to Runtime -> Change runtime type and select a GPU hardware accelerator')\n","print ('Found GPU at: {}'.format(device_name))\n","print ('cell finished')"],"execution_count":0,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"CjKOF2lK_xS_","colab_type":"text"},"source":["This next cell defines variables PERSQFT and BASE that will be used to generate the test data. Variable PERSQFT is the cost per house square foot divided by 1000. Variable BASE helps with tiny houses so that the price doesn't go to 0 for a very small house. (Notice that the price per sq ft is based on a cheap house in California)\n","\n","Variables PERSQFT and BASE will be the correct values that the model tries to learn from the data. Here we are using them to generate the training data so that we can see if the model can learn those values.\n","\n","Defining them in a separate cell allows you to easily change the values. You can experiment with the learning rate and number of EPOCHS to see if it makes a difference during training"]},{"cell_type":"code","metadata":{"id":"zjtdcqvm_xTA","colab_type":"code","colab":{}},"source":["# Cell 1.3\n","\n","PERSQFT = .4 # $400 per sq ft\n","BASE = .35 # $350K base price \n","\n","LEARNING_RATE = .01\n","EPOCHS = 200\n","ITEMS = 100\n","\n","print ('cell finished')"],"execution_count":0,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"9XyqSDkI_xTC","colab_type":"text"},"source":["This cell uses the constants from the cell above with a standard python function to create the training data as shown in the lecture."]},{"cell_type":"code","metadata":{"id":"Q8X8GrYw_xTD","colab_type":"code","colab":{}},"source":["# Cell 1.4\n","\n","def make_noisy_data(w=PERSQFT, b=BASE, n=ITEMS):\n"," sqft = np.random.rand(n).astype(np.float32) * 6 # multiply by 6 for max sq ft of 6000\n"," noise = np.random.normal(scale=0.08, size=len(sqft))\n"," price = w * sqft + b + noise\n"," return sqft, price\n","\n","print ('cell finished')"],"execution_count":0,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"u0Z7ibMx_xTE","colab_type":"text"},"source":["This cell calls the make_noisy_data function and assigns the outputs to two variables sqft_train and price_train. Variable sqft_train will be the x values fed into your linear regression model and price_train will be the expected output."]},{"cell_type":"code","metadata":{"id":"fgZh0nMM_xTF","colab_type":"code","colab":{}},"source":["# Cell 1.5\n","\n","sqft_train, price_train = make_noisy_data()\n","\n","\n","print ('cell finished')"],"execution_count":0,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"3-3DQMEg_xTH","colab_type":"text"},"source":["This cell plots our training data so that we can get an idea of its general distribution. It is a linear distribution with a bit of noise as specified in the function definition. You can experiment with different noise values to see how the results change."]},{"cell_type":"code","metadata":{"id":"aeCLZkmt_xTH","colab_type":"code","colab":{}},"source":["# Cell 1.6\n","\n","# Run this cell to plot the data.\n","\n","pylab.plot(sqft_train, price_train, 'b.')\n"],"execution_count":0,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"N23ZygdC_xTJ","colab_type":"text"},"source":["A Keras model will be used to describe the network. As discussed in the lecture, a Keras Dense layer includes variables for each input as well as a bias. The values of these variables will initially be random and learned by the training process. At the end of the training these values should approximate the per sq ft price and the base used to create the training data. Why wouldn't they be exact?"]},{"cell_type":"code","metadata":{"id":"1TPObKlz_xTK","colab_type":"code","colab":{}},"source":["# Cell 1.7\n","\n","def model():\n"," model = keras.Sequential([\n"," layers.Dense(1, input_shape=[1])\n"," ])\n"," return model\n","\n","print ('cell finished')"],"execution_count":0,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"TAr5B_jD_xTL","colab_type":"text"},"source":["The next cell selects an optimizer to update the weight and bias values of the Keras Dense layer. We'll chose the stochastic gradient descent (SGD) optimizer, with a learning rate of 0.01."]},{"cell_type":"code","metadata":{"id":"pb8eepOB_xTM","colab_type":"code","colab":{}},"source":["# Cell 1.8\n","\n","optimizer = tf.keras.optimizers.SGD(0.01)\n","\n","print ('cell finished')"],"execution_count":0,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"eUkFNR5t_xTO","colab_type":"text"},"source":["The next cell calls the function that creates the Keras Sequential model. An instance of the model will be stored in variable m1 for use in the training process"]},{"cell_type":"code","metadata":{"id":"jRgXfIOn_xTP","colab_type":"code","colab":{}},"source":["# Cell 1.9\n","\n","m1 = model()\n","\n","print ('cell finished')"],"execution_count":0,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"WrD__1nR_xTQ","colab_type":"text"},"source":["Before the Keras model can be used it needs to be compiled. The compile will describe a number of parameters that will be used during the training and evaluation process. This includes the optimizer for training, the type of loss to be optimized, and the metrics to report. \n","\n","In this example the optimizer will be the one created above. The loss to optimize will be mse, or mean squared error. The metrics will report both mean squared error and mean absolute error. "]},{"cell_type":"code","metadata":{"id":"xQV6xsCJ_xTR","colab_type":"code","colab":{}},"source":["# Cell 1.10\n","\n","m1.compile(loss='mse',\n"," optimizer=optimizer,\n"," metrics=['mae', 'mse'])\n","\n","print ('cell finished')"],"execution_count":0,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"6SMNXPdO_xTS","colab_type":"text"},"source":["The following cell calls the summary function on the Keras model. This function will report information about the structure of the Keras model. The report shows that the Keras model has one layer named dense of type Dense. The output shape is 1 and there are 2 trainable parameters. One parameter will be the input weight and the second will be the bias for the output. "]},{"cell_type":"code","metadata":{"id":"ZgR4drno_xTT","colab_type":"code","colab":{}},"source":["# Cell 1.11\n","\n","m1.summary()\n","\n","print ('cell finished')"],"execution_count":0,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"r09UOSDb_xTU","colab_type":"text"},"source":["The next cell will try out the model before training so that we can see that the random weights and bias do not produce the correct results. The model needs to be trained first. The predict function expects an array of values so test_sqft is created as an array. That value is indexed to obtain the expected price. "]},{"cell_type":"code","metadata":{"id":"Q4rz83pv_xTV","colab_type":"code","colab":{}},"source":["# Cell 1.12\n","\n","test_sqft = [[4.0]]\n","expPrice = test_sqft[0][0] * PERSQFT + BASE\n","\n","outval = m1.predict(test_sqft)\n","print (\"price should be [{}] but is {}\".format(expPrice,outval))"],"execution_count":0,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"1hxlXcmu_xTW","colab_type":"text"},"source":["This cell defines how the linear regression model is trained. The training will use the number of EPOCHS specified in the cell near the top to determine how many iterations to train. Each training iteration will update the values of w and b based on the loss calculated by the optimizer. \n","\n","The metrics are printed for each training epoch so that the progress of the training can be observed. The added value of this is that it can seen when the values start to stabilize and determine whether the learning rate and number of epochs are too small or too large."]},{"cell_type":"code","metadata":{"id":"FHSlZM9s_xTX","colab_type":"code","colab":{}},"source":["# Cell 1.13\n","\n","history = m1.fit(\n"," sqft_train, price_train,\n"," epochs=EPOCHS, verbose=2)\n","\n","print ('cell finished')"],"execution_count":0,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"kICHEWbN_xTY","colab_type":"text"},"source":["This cell simply prints out the values of our trainable values for price per sq ft and base price. The get_weights function will retrieve the weight values from the model. These are multiplied by 1000 to put them in the format used for describing the model. \n","\n","Since the weight and bias values are arrays, indexing is needed to retrieve them."]},{"cell_type":"code","metadata":{"id":"ffyPGv1t_xTY","colab_type":"code","colab":{}},"source":["# Cell 1.14\n","\n","wval = m1.get_weights()[0][0][0]\n","bval = m1.get_weights()[1][0]\n","print(wval)\n","print (bval)\n","\n","print('per sq ft = {}'.format(wval*1000))\n","print('base price = {}'.format(bval*1000))"],"execution_count":0,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"M7Y-oEeI_xTa","colab_type":"text"},"source":["Now that the training has run we can plot the values for w and b and see how closely they matched the training data. Ideally the line should bisect the training data. What would be reasons that it did not?"]},{"cell_type":"code","metadata":{"id":"FfmlMkmE_xTa","colab_type":"code","colab":{}},"source":["# Cell 1.15\n","\n","print (\"w: %f, b: %f\" % (wval, bval))\n","pylab.plot(sqft_train, price_train, 'b.')\n","x = np.linspace(0,6,100)\n","y = x * wval + bval\n","pylab.plot(x,y, 'r.')"],"execution_count":0,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"lHITXB2F_xTc","colab_type":"text"},"source":["As you can see the line fits the data pretty well with the default parameters for EPOCHS and LEARNING_RATE. Are those optimal values? Can you get just as good results with different values?\n","\n","Also experiment with the price per sq ft and base price to make sure the model can still learn the values"]},{"cell_type":"code","metadata":{"id":"jU_d0XA7fyU_","colab_type":"code","colab":{}},"source":[""],"execution_count":0,"outputs":[]}]} 2 | -------------------------------------------------------------------------------- /02_tensorflow_logistic_regression.ipynb: -------------------------------------------------------------------------------- 1 | {"nbformat":4,"nbformat_minor":0,"metadata":{"colab":{"name":"02_tensorflow_logistic_regression.ipynb","provenance":[{"file_id":"https://github.com/edge-ai-vision/dlcvtf2/blob/master/02_tensorflow_logistic_regression.ipynb","timestamp":1592006992626},{"file_id":"https://github.com/edge-ai-vision/dlcvtf2/blob/master/02_tensorflow_logistic_regression.ipynb","timestamp":1591898574484}]},"kernelspec":{"name":"python3","display_name":"Python 3"},"accelerator":"GPU"},"cells":[{"cell_type":"markdown","metadata":{"id":"I-S9EwVjCGwX","colab_type":"text"},"source":["# Deep Learning for Computer Vision with TensorFlow 2.0\n","\n","Table of contents:\n","[Lab 0](https://colab.research.google.com/github/edge-ai-vision/dlcvtf2/blob/master/00_test_install.ipynb) | \n","[Lab 1](https://colab.research.google.com/github/edge-ai-vision/dlcvtf2/blob/master/01_linear_regression.ipynb) | \n","[Lab 2](https://colab.research.google.com/github/edge-ai-vision/dlcvtf2/blob/master/02_tensorflow_logistic_regression.ipynb) | \n","[Lab 3a](https://colab.research.google.com/github/edge-ai-vision/dlcvtf2/blob/master/03a_tensorflow_deep_network.ipynb) | \n","[Lab 3b](https://colab.research.google.com/github/edge-ai-vision/dlcvtf2/blob/master/03b_deep_mnist_visualize.ipynb) | \n","[Lab 4](https://colab.research.google.com/github/edge-ai-vision/dlcvtf2/blob/master/04_mnist_cnn.ipynb) | \n","[Lab 5](https://colab.research.google.com/github/edge-ai-vision/dlcvtf2/blob/master/05_data_prep.ipynb) | \n","[Lab 6](https://colab.research.google.com/github/edge-ai-vision/dlcvtf2/blob/master/06_transfer_learning.ipynb) | \n","\n","# Lab 2: Logistic Regression\n","\n","If you ever want to start over just go to the menu at the top and select Runtime -> Restart runtime. (Please do not use Runtime -> Reset all runtimes.)"]},{"cell_type":"markdown","metadata":{"id":"A3kDN3R5f3rf","colab_type":"text"},"source":["In this exercise you will again apply what you've learned in the lecture. This model is for a classifier and will use the standard MNIST handwritten zip code dataset. Run each of the cells to create the classifier model, train the model, and test the model to see how well the model performs. \n","\n","You can modify parameters in certain cells to change the way that the training is performed and see what happens. Experiment to see if you can get better results than the defaults. \n","\n","This cell imports tensorflow as well as Keras which will be used for the layer description."]},{"cell_type":"code","metadata":{"id":"Ep6B6cmaf3rg","colab_type":"code","colab":{}},"source":["# Cell 2.1\n","%tensorflow_version 2.x\n","import tensorflow as tf\n","\n","from tensorflow import keras\n","from tensorflow.keras import layers\n","\n","import numpy as np\n","\n","%pylab inline\n","\n","print ('cell finished')"],"execution_count":0,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"md3KS_KBf3rj","colab_type":"text"},"source":["In TensorFlow 2.0 the datasets are loaded differently. Most of the commonly used instruction datasets are located in the datasets directory of Keras. The first line of the next cell will read in the training and testing data for MNIST into the appropriate variables x_train, y_train, x_test, and y_test. \n","\n","The data needs to be slightly modified to work with the Keras model we will use for classification. The image data will be converted from float64 to float32 and the labels will be converted to int64 type. \n"]},{"cell_type":"code","metadata":{"id":"TJdQaPJzf3rk","colab_type":"code","colab":{}},"source":["# Cell 2.2\n","\n","(x_train, y_train), (x_test, y_test) = tf.keras.datasets.mnist.load_data()\n","# Numpy defaults to dtype=float64; TF defaults to float32. Stick with float32.\n","x_train, x_test = x_train / np.float32(255), x_test / np.float32(255)\n","y_train, y_test = y_train.astype(np.int64), y_test.astype(np.int64)\n","\n","\n","print ('cell finished')"],"execution_count":0,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"KBiuy-Vaf3rm","colab_type":"text"},"source":["In TensorFlow 2.0 the best way to feed data to the model is using tf.data and especially the Dataset capabilities. Datasets can make use of batching as well as other capabilities available with Datasets. \n","\n","Variables x_train, y_train, etc. are tensors that can be converted into a Dataset with the statements shown below"]},{"cell_type":"code","metadata":{"id":"rTKjwG_ef3rm","colab_type":"code","colab":{}},"source":["# Cell 2.3\n","\n","train_ds = tf.data.Dataset.from_tensor_slices((x_train, y_train)).repeat()\n","test_ds = tf.data.Dataset.from_tensor_slices((x_test, y_test))\n","\n","\n","print ('cell finished')"],"execution_count":0,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"8g7iB02Mf3ro","colab_type":"text"},"source":["The next cell contains a number of constants used in the model and training. Putting these variables in one location allows for easy modification throughout the model. "]},{"cell_type":"code","metadata":{"id":"eBaT07VIf3rp","colab_type":"code","colab":{}},"source":["# Cell 2.4\n","\n","BATCH_SIZE=100\n","\n","NUM_CLASSES = 10\n","IMG_HT = 28\n","IMG_W = 28\n","\n","print ('cell finished')"],"execution_count":0,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"gougX685f3rr","colab_type":"text"},"source":["The next two statements are used to specify the batch sizes for the training and testing datasets. Additionally the training dataset is shuffled so that successive runs will use a different training image order. "]},{"cell_type":"code","metadata":{"id":"NVdWU6_af3rr","colab_type":"code","colab":{}},"source":["# Cell 2.5\n","\n","train_ds = train_ds.shuffle(60000).batch(BATCH_SIZE)\n","test_ds = test_ds.batch(BATCH_SIZE)\n","\n","print ('cell finished')"],"execution_count":0,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"VPDe1jrGf3ru","colab_type":"text"},"source":["This cell is meant to help understand the details of the MNIST dataset. You will notice a number of print statements that will print out different aspects of MNIST. Uncomment these and run the cell to see the results. \n","\n","The first print statement (2 lines) will print out the sizes of the train and test sets. \n","\n","The second shows how the labels are formatted. You can change the index used to pick a particular image to see different labels. \n","\n","The next print statement shows the shape of the fields of training data set\n","\n","The next statements show the length of a row of image data, and the contents of one row in the middle\n","\n","The final two statements plot the image from the index. Again feel free to modify the indices to see different images."]},{"cell_type":"code","metadata":{"id":"kHE388cof3rv","colab_type":"code","colab":{}},"source":["# Cell 2.6\n","\n","# Run this cell to understand the format of the dataset. Uncomment the print commands one by one to understand what\n","# the data set looks like. As you uncomment a line use shift-enter to run the cell once more. In the first print you\n","# actually have to uncomment two lines or you will get an error. \n","\n","img_index = 9\n","\n","# 1. There are 55k, 5k, and 10k examples in train, validation, and test.\n","print ('Train, test: %d, %d' % \n"," (len(x_train), len(x_test)))\n","\n","# 2. The label is an integer\n","print ('label = {}'.format(y_train[img_index]))\n","\n","# 3. The shapes of the two different data types in the Dataset is shown here\n","print ('x shape = {}'.format(np.shape(x_train)))\n","print ('y_shape = {}'.format(np.shape(y_train)))\n","\n","# 4. An image is a 28 by 288 array of pixels.\n","print ('length of first row = {}'.format(len(x_train[img_index])))\n","print (x_train[img_index][10]) # This prints the 11th row of 28. The nonzero values represent the pixel values through\n"," # the middle of the digit\n","\n","# 5. To display an image\n","pylab.imshow(x_train[img_index], cmap=pylab.cm.gray_r) \n","pylab.title('Label: %d' % y_train[img_index])\n"],"execution_count":0,"outputs":[]},{"cell_type":"markdown","metadata":{"collapsed":true,"id":"OiBZeRZFf3rx","colab_type":"text"},"source":["The next cell contains a function that describes the classification network. A Keras Sequential model is used to describe a single classification layer. The classification layer is a dense layer so it will input all of the pixels of an image at a time. Since the MNIST data is an array of 28 x 28 a Flatten layer is needed to convert it into a flat array of 784 pixels. Notice that the Flatten layer uses the IMG_HT and IMG_W constants to determine how to flatten the image. \n","\n","The dense layer is then 784 pixels as input and NUM_CLASSES(10) output. One class for each of our 10 digit types. \n","\n","The activation is specified as softmax so that probabilities are used for neuron output. "]},{"cell_type":"code","metadata":{"id":"6hkOvIi1f3rx","colab_type":"code","colab":{}},"source":["# Cell 2.7\n","\n","def model():\n"," model = keras.Sequential([\n"," layers.Flatten(input_shape=[IMG_HT,IMG_W]),\n"," layers.Dense(NUM_CLASSES, activation='softmax')\n"," ])\n"," return model\n","\n","print ('cell finished')"],"execution_count":0,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"0ZZjk1VFf3rz","colab_type":"text"},"source":["Just as in the previous notebook the model function will be called to create the Keras model for the classification. \n"," "]},{"cell_type":"code","metadata":{"id":"ybu2FV_jf3rz","colab_type":"code","colab":{}},"source":["# Cell 2.8\n","\n","mnist_model = model()\n","\n","mnist_model.summary()\n","\n","print ('cell finished')"],"execution_count":0,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"1Q4kyLPOf3r1","colab_type":"text"},"source":["Also as previously discussed the model needs to be compiled for use. This time the optimizer will be a built-in one called Adam. We will use default parameters to the optimizer, though these can be directly specified. \n","\n","The loss will be something called sparse categorical cross entropy. The sparse designation specifies that the label is an integer versus a vector value from the neurons. Categorical specifies that the operation is a classification type, not a linear regression operation. Lastly the cross entropy is a type of error that works well with classification. \n","\n","The metrics that will be displayed are the classification accuracy, though feel free to add others. "]},{"cell_type":"code","metadata":{"id":"XCevzh3tf3r1","colab_type":"code","colab":{}},"source":["# Cell 2.9\n","\n","optimizer = tf.keras.optimizers.SGD(0.1)\n","\n","mnist_model.compile(optimizer=optimizer, \n"," loss='sparse_categorical_crossentropy',\n"," metrics=['accuracy'])\n","\n","\n","print ('cell finished')"],"execution_count":0,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"CxZNtOC_f3r3","colab_type":"text"},"source":["Let's try the model out before it is trained to see how well it does. Since the weights and biases are random values there is a chance (about 1 in 10) that the correct value is returned from classification. This code will allow you to try an example to see if it works. \n","\n","The `image_index` variable is used to grab an example from the training dataset. The expected label is captured in variable `exp_label`. The pixel data is captured in `img`. These values are fed to the network and the predict function called to generate the output in variable `label`. \n","\n","The results are printed out and the image plotted so you can see which one it is. \n","\n","Experiment with the `image_index` variable to see if you can find images where the untrained model correctly predicts the label. You will need to change the index and rerun this cell. "]},{"cell_type":"code","metadata":{"id":"aGZFBoeJf3r3","colab_type":"code","colab":{}},"source":["# Cell 2.10\n","\n","image_index = 101\n","exp_label = y_train[image_index]\n","\n","img = (np.expand_dims(x_train[image_index],0)) # remember the predict needs an array\n","\n","label = mnist_model.predict(img)\n","\n","print (\"calculated label = {} expected label = {}\".format(np.argmax(label), exp_label))\n","pylab.imshow(x_train[image_index], cmap=pylab.cm.gray_r) \n","pylab.title('Label: %d' % y_train[image_index]) "],"execution_count":0,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"aRRUImXgf3r5","colab_type":"text"},"source":["This next cell is where all the magic happens. The training dataset train_ds is input to the fit method and the training is run using the number of epochs specified. The steps_per_epoch is usually specified as the length of the dataset divided by the batch size so that the there are enough steps in each epoch to run all of the examples. \n","\n","The training process will use the batch size that was applied to the dataset earlier. \n","\n","The `verbose` keyword specifies how much data is output for each epoch. A setting of 2 will output a single line per epoch. "]},{"cell_type":"code","metadata":{"id":"mxcdjTE4f3r5","colab_type":"code","colab":{}},"source":["# Cell 2.11\n","\n","history = mnist_model.fit(\n"," train_ds,\n"," epochs=50, steps_per_epoch=600,verbose=2)\n","\n","\n","print ('cell finished')"],"execution_count":0,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"fmZyMcB8f3r7","colab_type":"text"},"source":["During the training the accuracy and loss are displayed for each epoch. When the accuracy plateaus there is no need to run further epochs. \n","\n","This is the training accuracy. It is not the same as the testing accuracy. As mentioned in the lecture the data has been split into a training set and a testing set. The test data was never seen by the model during training but is used to test how well the model generalizes. \n"]},{"cell_type":"markdown","metadata":{"id":"oFLVx3qFf3r7","colab_type":"text"},"source":["\n","The previous cell showed the training accuracy because it used the training images. To get the testing accuracy let's run the accuracy on the test images. "]},{"cell_type":"code","metadata":{"id":"vV2ErifRf3r8","colab_type":"code","colab":{}},"source":["# Cell 2.12 \n","\n","mnist_model.evaluate(\n"," test_ds, steps=32)\n"],"execution_count":0,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"OfAq1_Twf3r9","colab_type":"text"},"source":["The testing accuracy is slightly different, but remember the model was not trained on those images, the model has never seen those images before, so having a similar accuracy is a good sign. \n","\n","The accuracy displayed on the first line is for the last epoch of the testing data. The final accuracy is displayed on the last line. \n","\n","You might think that over 90% accuracy is good, but the accuracy of this model has great room for improvement. Going forward more accurate models will be developed."]},{"cell_type":"markdown","metadata":{"collapsed":true,"id":"xQcJz3cef3r-","colab_type":"text"},"source":["Go back to cell 10 and change the image_index once more. The model now uses the trained weight values. Nine out of ten times as we see by the accuracy it should pick the correct label."]},{"cell_type":"code","metadata":{"id":"ZblmYnDCf3r-","colab_type":"code","colab":{}},"source":[""],"execution_count":0,"outputs":[]}]} 2 | -------------------------------------------------------------------------------- /03a_tensorflow_deep_network.ipynb: -------------------------------------------------------------------------------- 1 | {"nbformat":4,"nbformat_minor":0,"metadata":{"colab":{"name":"03a_tensorflow_deep_network.ipynb","provenance":[{"file_id":"https://github.com/edge-ai-vision/dlcvtf2/blob/master/03a_tensorflow_deep_network.ipynb","timestamp":1592007087838},{"file_id":"https://github.com/edge-ai-vision/dlcvtf2/blob/master/03a_tensorflow_deep_network.ipynb","timestamp":1591898699670}]},"kernelspec":{"name":"python3","display_name":"Python 3"},"accelerator":"GPU"},"cells":[{"cell_type":"markdown","metadata":{"id":"I-S9EwVjCGwX","colab_type":"text"},"source":["# Deep Learning for Computer Vision with TensorFlow 2.0\n","\n","Table of contents:\n","[Lab 0](https://colab.research.google.com/github/edge-ai-vision/dlcvtf2/blob/master/00_test_install.ipynb) | \n","[Lab 1](https://colab.research.google.com/github/edge-ai-vision/dlcvtf2/blob/master/01_linear_regression.ipynb) | \n","[Lab 2](https://colab.research.google.com/github/edge-ai-vision/dlcvtf2/blob/master/02_tensorflow_logistic_regression.ipynb) | \n","[Lab 3a](https://colab.research.google.com/github/edge-ai-vision/dlcvtf2/blob/master/03a_tensorflow_deep_network.ipynb) | \n","[Lab 3b](https://colab.research.google.com/github/edge-ai-vision/dlcvtf2/blob/master/03b_deep_mnist_visualize.ipynb) | \n","[Lab 4](https://colab.research.google.com/github/edge-ai-vision/dlcvtf2/blob/master/04_mnist_cnn.ipynb) | \n","[Lab 5](https://colab.research.google.com/github/edge-ai-vision/dlcvtf2/blob/master/05_data_prep.ipynb) | \n","[Lab 6](https://colab.research.google.com/github/edge-ai-vision/dlcvtf2/blob/master/06_transfer_learning.ipynb) | \n","\n","# Lab 3: Deep MNIST\n","\n","If you ever want to start over just go to the menu at the top and select Runtime -> Restart runtime. (Please do not use Runtime -> Reset all runtimes.)"]},{"cell_type":"markdown","metadata":{"id":"IvaNoa6igi_Q","colab_type":"text"},"source":["In this exercise you will again apply what you've learned in the lecture. This is another model for an MNIST classifier but this model will use a deep network instead of a shallow one as in the last example. \n","\n","Run each of the cells to create the classifier model, train the model, and test the model to see how well the model performs. \n","\n","You can modify parameters in certain cells to change the way that the training is performed and see what happens. Experiment to see if you can get better results than the defaults. \n","\n","This cell imports tensorflow as well as Keras which will be used for the layer description."]},{"cell_type":"code","metadata":{"id":"J20xTt_Bgi_Q","colab_type":"code","colab":{}},"source":["# Cell 3a.1\n","%tensorflow_version 2.x\n","import tensorflow as tf\n","\n","from tensorflow import keras\n","from tensorflow.keras import layers\n","\n","import numpy as np\n","\n","%pylab inline\n","\n","print ('cell finished')"],"execution_count":0,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"3yluZqG_gi_T","colab_type":"text"},"source":["Notice that the dataset loading and modification code is exactly the same as in the shallow model.\n"]},{"cell_type":"code","metadata":{"id":"ICz8RUpsgi_T","colab_type":"code","colab":{}},"source":["# Cell 3a.2\n","\n","(x_train, y_train), (x_test, y_test) = tf.keras.datasets.mnist.load_data()\n","# Numpy defaults to dtype=float64; TF defaults to float32. Stick with float32.\n","x_train, x_test = x_train / np.float32(255), x_test / np.float32(255)\n","y_train, y_test = y_train.astype(np.int64), y_test.astype(np.int64)\n","\n","\n","print ('cell finished')"],"execution_count":0,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"KYiHOctGgi_V","colab_type":"text"},"source":["The dataset conversion code is also exactly the same.\n","\n","Variables x_train, y_train, etc. are tensors that can be converted into a Dataset with the statements shown below"]},{"cell_type":"code","metadata":{"id":"idDgToPVgi_W","colab_type":"code","colab":{}},"source":["# Cell 3a.3\n","\n","train_ds = tf.data.Dataset.from_tensor_slices((x_train, y_train)).repeat()\n","test_ds = tf.data.Dataset.from_tensor_slices((x_test, y_test))\n","\n","\n","print ('cell finished')"],"execution_count":0,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"KveZEOYmgi_Y","colab_type":"text"},"source":["The next cell contains a number of constants used in the model and training. Putting these variables in one location allows for easy modification throughout the model. Notice that a new variable HIDDEN_SIZE has been added. This will control the size of the hidden layer that is added for the deep network."]},{"cell_type":"code","metadata":{"id":"A7pcEKxYgi_Y","colab_type":"code","colab":{}},"source":["# Cell 3a.4\n","\n","BATCH_SIZE=100\n","\n","HIDDEN_SIZE = 300\n","\n","NUM_CLASSES = 10\n","IMG_HT = 28\n","IMG_W = 28\n","\n","print ('cell finished')"],"execution_count":0,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"ss441jrJgi_a","colab_type":"text"},"source":["The next two statements are used to specify the batch sizes for the training and testing datasets. Additionally the training dataset is shuffled so that successive runs will use a different training image order. "]},{"cell_type":"code","metadata":{"id":"9WDaiD-fgi_a","colab_type":"code","colab":{}},"source":["# Cell 3a.5\n","\n","train_ds = train_ds.shuffle(60000).batch(BATCH_SIZE)\n","test_ds = test_ds.batch(BATCH_SIZE)\n","\n","print ('cell finished')"],"execution_count":0,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"TEwhYHZqgi_c","colab_type":"text"},"source":["This cell is again the same as in the last example. \n","\n","This cell is meant to help understand the details of the MNIST dataset. You will notice a number of print statements that will print out different aspects of MNIST. Uncomment these and run the cell to see the results. \n","\n","The first print statement (2 lines) will print out the sizes of the train and test sets. \n","\n","The second shows how the labels are formatted. You can change the index used to pick a particular image to see different labels. \n","\n","The next print statement shows the shape of the fields of training data set\n","\n","The next statements show the length of a row of image data, and the contents of one row in the middle\n","\n","The final two statements plot the image from the index. Again feel free to modify the indices to see different images."]},{"cell_type":"code","metadata":{"id":"fEuvdhbfgi_c","colab_type":"code","colab":{}},"source":["# Cell 3a.6\n","\n","# Run this cell to understand the format of the dataset. Uncomment the print commands one by one to understand what\n","# the data set looks like. As you uncomment a line use shift-enter to run the cell once more. In the first print you\n","# actually have to uncomment two lines or you will get an error. \n","\n","img_index = 7\n","\n","# 1. There are 55k, 5k, and 10k examples in train, validation, and test.\n","print ('Train, test: %d, %d' % \n"," (len(x_train), len(x_test)))\n","\n","# 2. The label is an integer\n","print ('label = {}'.format(y_train[img_index]))\n","\n","# 3. The shapes of the two different data types in the Dataset is shown here\n","print ('x shape = {}'.format(np.shape(x_train)))\n","print ('y_shape = {}'.format(np.shape(y_train)))\n","\n","# 4. An image is a 28 by 288 array of pixels.\n","print ('length of first row = {}'.format(len(x_train[img_index])))\n","print (x_train[img_index][10]) # This prints the 11th row of 28. The nonzero values represent the pixel values through\n"," # the middle of the digit\n","\n","# 5. To display an image\n","pylab.imshow(x_train[img_index], cmap=pylab.cm.gray_r) \n","pylab.title('Label: %d' % y_train[img_index])"],"execution_count":0,"outputs":[]},{"cell_type":"markdown","metadata":{"collapsed":true,"id":"xP99prBAgi_e","colab_type":"text"},"source":["The next cell contains a function that describes the classification network. A Keras Sequential model is used to describe a single classification layer. The classification layer is a dense layer so it will input all of the pixels of an image at a time. Since the MNIST data is an array of 28 x 28 a Flatten layer is needed to convert it into a flat array of 784 pixels. Notice that the Flatten layer uses the IMG_HT and IMG_W constants to determine how to flatten the image. \n","\n","This model now contains an extra hidden layer that will be used to add non-linearity for more complex model behavior. The size of the hidden layer is specified by the new constant HIDDEN_SIZE that was added earlier. \n","\n","The dense layer is then 784 pixels as input and NUM_CLASSES(10) output. One class for each of our 10 digit types. \n","\n","The activation is specified as softmax so that probabilities are used for neuron output. "]},{"cell_type":"code","metadata":{"id":"6JuP2HlPgi_f","colab_type":"code","colab":{}},"source":["# Cell 3a.7\n","\n","def model():\n"," model = keras.Sequential([\n"," layers.Flatten(input_shape=[IMG_HT,IMG_W]),\n"," layers.Dense(HIDDEN_SIZE, activation='relu'),\n"," layers.Dense(NUM_CLASSES, activation='softmax')\n"," ])\n"," return model\n","\n","print ('cell finished')"],"execution_count":0,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"yHfQAA0sgi_g","colab_type":"text"},"source":["Just as in the previous notebook the model function will be called to create the Keras model for the classification. \n"," "]},{"cell_type":"code","metadata":{"id":"rI4c-khcgi_h","colab_type":"code","colab":{}},"source":["# Cell 3a.8\n","\n","mnist_model = model()\n","\n","mnist_model.summary()\n","\n","print ('cell finished')"],"execution_count":0,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"m0byQ2XQgi_j","colab_type":"text"},"source":["Also as previously discussed the model needs to be compiled for use. \n","\n","Notice that this code does not change for a deep network, it is exactly the same except that we decided to use the adam optimizer. \n","\n","The metrics that will be displayed are the classification accuracy, though feel free to add others. "]},{"cell_type":"code","metadata":{"id":"b6la4rSbgi_j","colab_type":"code","colab":{}},"source":["# Cell 3a.9\n","\n","mnist_model.compile(optimizer='adam', \n"," loss='sparse_categorical_crossentropy',\n"," metrics=['accuracy'])\n","\n","\n","print ('cell finished')"],"execution_count":0,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"XRUyJNQJgi_l","colab_type":"text"},"source":["Let's try the model out before it is trained to see how well it does. Since the weights and biases are random values there is a chance (about 1 in 10) that the correct value is returned from classification. This code will allow you to try an example to see if it works. \n","\n","The `image_index` variable is used to grab an example from the training dataset. The expected label is captured in variable `exp_label`. The pixel data is captured in `img`. These values are fed to the network and the predict function called to generate the output in variable `label`. \n","\n","The results are printed out and the image plotted so you can see which one it is. \n","\n","Experiment with the `image_index` variable to see if you can find images where the untrained model correctly predicts the label. You will need to change the index and rerun this cell. "]},{"cell_type":"code","metadata":{"id":"VKvmfYCvgi_l","colab_type":"code","colab":{}},"source":["# Cell 3a.10\n","\n","image_index = 109\n","exp_label = y_train[image_index]\n","\n","img = (np.expand_dims(x_train[image_index],0))\n","\n","label = mnist_model.predict(img)\n","\n","print (\"calculated label = {} expected label = {}\".format(np.argmax(label), exp_label))\n","pylab.imshow(x_train[image_index], cmap=pylab.cm.gray_r) \n","pylab.title('Label: %d' % y_train[image_index]) "],"execution_count":0,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"BEg5QKslgi_n","colab_type":"text"},"source":["With a deep network you will notice that the training takes more time. This is especially noticeable on a CPU vs a GPU. \n","\n","The training dataset train_ds is input to the fit method and the training is run using the number of epochs specified. The steps_per_epoch is usually specified as the length of the dataset divided by the batch size so that the there are enough steps in each epoch to run all of the examples. \n","\n","The training process will use the batch size that was applied to the dataset earlier. \n","\n","This time you might also notice some lines with perfect accuracy. That is not for the complete training set but only a single batch.\n","\n","The `verbose` keyword specifies how much data is output for each epoch. A setting of 2 will output a single line per epoch. "]},{"cell_type":"code","metadata":{"id":"bhO4ZpHQgi_n","colab_type":"code","colab":{}},"source":["# Cell 3a.11\n","\n","history = mnist_model.fit(\n"," train_ds,\n"," epochs=50, steps_per_epoch=600,verbose=2)\n","\n","\n","print ('cell finished')"],"execution_count":0,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"jjcXYgDvgi_p","colab_type":"text"},"source":["During the training the accuracy and loss are displayed for each epoch. When the accuracy plateaus there is no need to run further epochs. \n","\n","This is the training accuracy. It is not the same as the testing accuracy. As mentioned in the lecture the data has been split into a training set and a testing set. The test data was never seen by the model during training but is used to test how well the model generalizes. \n"]},{"cell_type":"markdown","metadata":{"id":"XAGWfLYdgi_r","colab_type":"text"},"source":["\n","The previous cell showed the training accuracy because it used the training images. To get the testing accuracy let's run the accuracy on the test images. "]},{"cell_type":"code","metadata":{"id":"yOJBHYuagi_r","colab_type":"code","colab":{}},"source":["# Cell 3a.12 \n","\n","mnist_model.evaluate(\n"," test_ds, steps=32)\n"],"execution_count":0,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"Q81-1oYCgi_t","colab_type":"text"},"source":["The testing accuracy should be much improved over the shallow model because the deep model is better able to handle the complexities in the data. \n","\n","The testing accuracy is slightly different, but remember the model was not trained on those images, the model has never seen those images before, so having a similar accuracy is a good sign. \n","\n","The accuracy displayed on the first line is for the last epoch of the testing data. The final accuracy is displayed on the last line. \n","\n","You might think that over 90% accuracy is good, but the accuracy of this model has great room for improvement. Going forward more accurate models will be developed."]},{"cell_type":"markdown","metadata":{"collapsed":true,"id":"C2jWPTe0gi_t","colab_type":"text"},"source":["Go back to cell 10 and change the image_index once more. The model now uses the trained weight values. Nine out of ten times as we see by the accuracy it should pick the correct label."]},{"cell_type":"code","metadata":{"id":"Mx0HBEGJgi_u","colab_type":"code","colab":{}},"source":[""],"execution_count":0,"outputs":[]}]} 2 | -------------------------------------------------------------------------------- /03b_deep_mnist_visualize.ipynb: -------------------------------------------------------------------------------- 1 | {"nbformat":4,"nbformat_minor":0,"metadata":{"colab":{"name":"03b_deep_mnist_visualize.ipynb","provenance":[{"file_id":"https://github.com/edge-ai-vision/dlcvtf2/blob/master/03b_deep_mnist_visualize.ipynb","timestamp":1591898808174}]},"kernelspec":{"name":"python3","display_name":"Python 3"},"accelerator":"GPU"},"cells":[{"cell_type":"markdown","metadata":{"id":"I-S9EwVjCGwX","colab_type":"text"},"source":["# Deep Learning for Computer Vision with TensorFlow 2.0\n","\n","Table of contents:\n","[Lab 0](https://colab.research.google.com/github/edge-ai-vision/dlcvtf2/blob/master/00_test_install.ipynb) | \n","[Lab 1](https://colab.research.google.com/github/edge-ai-vision/dlcvtf2/blob/master/01_linear_regression.ipynb) | \n","[Lab 2](https://colab.research.google.com/github/edge-ai-vision/dlcvtf2/blob/master/02_tensorflow_logistic_regression.ipynb) | \n","[Lab 3a](https://colab.research.google.com/github/edge-ai-vision/dlcvtf2/blob/master/03a_tensorflow_deep_network.ipynb) | \n","[Lab 3b](https://colab.research.google.com/github/edge-ai-vision/dlcvtf2/blob/master/03b_deep_mnist_visualize.ipynb) | \n","[Lab 4](https://colab.research.google.com/github/edge-ai-vision/dlcvtf2/blob/master/04_mnist_cnn.ipynb) | \n","[Lab 5](https://colab.research.google.com/github/edge-ai-vision/dlcvtf2/blob/master/05_data_prep.ipynb) | \n","[Lab 6](https://colab.research.google.com/github/edge-ai-vision/dlcvtf2/blob/master/06_transfer_learning.ipynb) | \n","\n","# Lab 3B: Deep MNIST with Visualization\n","\n","If you ever want to start over just go to the menu at the top and select Runtime -> Restart runtime. (Please do not use Runtime -> Reset all runtimes.)"]},{"cell_type":"markdown","metadata":{"collapsed":true,"id":"4xy2zSZZg9O6","colab_type":"text"},"source":["In this exercise you will use the deep model used in the last exercise but be we are going to closely monitor the training process. \n","\n","The training process is setup similar to the previous example, but there is an issue with one of the cells. It will be pointed out, but run it as provided and update the model as instructed later to make it work properly. \n"]},{"cell_type":"code","metadata":{"id":"HeziEqZ-g9O7","colab_type":"code","colab":{}},"source":["# Cell 3b.1\n","%tensorflow_version 2.x\n","import tensorflow as tf\n","\n","from tensorflow import keras\n","from tensorflow.keras import layers\n","\n","import numpy as np\n","%pylab inline\n","\n","print ('cell finished')"],"execution_count":0,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"t3PPgggYg9O9","colab_type":"text"},"source":["The next cell sets up the default settings for our training variables once again. \n","\n","The default starting value of the `LEARNING_RATE` variable is wrong on purpose. This is to let us see what happens in the training process when something goes wrong. Run through the process once to see the abysmal values, then come back to this cell and change `LEARNING_RATE` to .001. \n","\n","Run the process again to see that it is working properly."]},{"cell_type":"code","metadata":{"id":"gtSjjj4Sg9O9","colab_type":"code","colab":{}},"source":["# Cell 3b.2\n","\n","BATCH_SIZE=100\n","\n","HIDDEN_SIZE = 300\n","LEARNING_RATE = 0.5 # This is much too large, but run with it initally and change it later as instructed\n","\n","NUM_CLASSES = 10\n","IMG_HT = 28\n","IMG_W = 28\n","\n","print ('cell finished')"],"execution_count":0,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"amCuSD4Ug9O_","colab_type":"text"},"source":["The next cell reads in the MNIST dataset as before and sets up our model parameters. "]},{"cell_type":"code","metadata":{"id":"1V3xTyEkg9PA","colab_type":"code","colab":{}},"source":["# Cell 3b.3\n","\n","(x_train, y_train), (x_test, y_test) = tf.keras.datasets.mnist.load_data()\n","# Numpy defaults to dtype=float64; TF defaults to float32. Stick with float32.\n","x_train, x_test = x_train / np.float32(255), x_test / np.float32(255)\n","y_train, y_test = y_train.astype(np.int64), y_test.astype(np.int64)\n","\n","train_ds = tf.data.Dataset.from_tensor_slices((x_train, y_train)).repeat()\n","test_ds = tf.data.Dataset.from_tensor_slices((x_test, y_test))\n","\n","# DON'T CHANGE THESE VARIABLES OR YOU WILL BREAK THINGS\n","NUM_PIXELS = 28 * 28\n","NUM_CLASSES = 10\n","\n","print ('cell finished')"],"execution_count":0,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"UMoNjHrag9PC","colab_type":"text"},"source":["This is the same code to setup our placeholders"]},{"cell_type":"code","metadata":{"id":"OFHRBJDQg9PC","colab_type":"code","colab":{}},"source":["# Cell 3b.4\n","\n","train_ds = train_ds.shuffle(60000).batch(BATCH_SIZE)\n","test_ds = test_ds.batch(BATCH_SIZE)\n","\n","print ('cell finished')\n"],"execution_count":0,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"Ronbd2fSg9PE","colab_type":"text"},"source":["This cell is the same model as before. It uses the constants from cell 2 to size the network"]},{"cell_type":"code","metadata":{"id":"GMiqZZZFg9PE","colab_type":"code","colab":{}},"source":["# Cell 3b.5\n","\n","def model():\n"," model = keras.Sequential([\n"," layers.Flatten(input_shape=[IMG_HT,IMG_W]),\n"," layers.Dense(HIDDEN_SIZE, activation='relu'),\n"," layers.Dense(NUM_CLASSES, activation='softmax')\n"," ])\n"," return model\n","\n","tb_callback = tf.keras.callbacks.TensorBoard(log_dir='log_dir')\n","\n","print ('cell finished')\n"],"execution_count":0,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"s_e7hXAdg9PG","colab_type":"text"},"source":["Creating the model and displaying the summary as before "]},{"cell_type":"code","metadata":{"id":"Rnz9S_nTg9PG","colab_type":"code","colab":{}},"source":["# Cell 3b.6\n","\n","mnist_model = model()\n","\n","mnist_model.summary()\n","\n","print ('cell finished')\n"],"execution_count":0,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"BBerfZtLg9PI","colab_type":"text"},"source":["This cell creates an optimizer directly using the `LEARNING_RATE` constant from cell 2"]},{"cell_type":"code","metadata":{"id":"XoCadd6ng9PJ","colab_type":"code","colab":{}},"source":["# Cell 3b.7\n","\n","optimizer = tf.keras.optimizers.Adam(LEARNING_RATE)\n"],"execution_count":0,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"eidOETA8g9PL","colab_type":"text"},"source":["The model is compiled and instead of specifying the built-in optimizer with default values the optimizer object created above is used as the optimizer argument."]},{"cell_type":"code","metadata":{"id":"S8M3PEaOg9PM","colab_type":"code","colab":{}},"source":["# cell 3b.8\n","\n","mnist_model.compile(optimizer=optimizer, \n"," loss='sparse_categorical_crossentropy',\n"," metrics=['accuracy'])\n","\n","print ('cell finished')\n"],"execution_count":0,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"49Y3le21g9PO","colab_type":"text"},"source":["Now the model will be fit to the data using our standard technique. However the first time through the loss will be high and the accuracy low. After the learning process is fixed, the second time through should show much better results"]},{"cell_type":"code","metadata":{"id":"oPdHu1ahg9PO","colab_type":"code","colab":{}},"source":["# cell 3b.9\n","\n","history = mnist_model.fit(\n"," train_ds,\n"," epochs=20, \n"," steps_per_epoch=600,verbose=2,callbacks=[tb_callback])\n","\n","print ('cell finished')"],"execution_count":0,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"2MwKE0x1g9PP","colab_type":"text"},"source":["When the fit method is run data about the run is returned. In our example this data is gathered in an object named `history` which was provided as the output variable from the fit process. \n","\n","The data is a dictionary object whose contents are based on the metrics requested for the training process. In this example the accuracy `acc` and loss `loss` were captured and can be plot over each epoch of the training. "]},{"cell_type":"code","metadata":{"id":"hjX8QNmpg9PQ","colab_type":"code","colab":{}},"source":["# Cell 3b.10\n","\n","#pylab.plot(sqft_train, price_train, 'b.')\n","\n","pylab.plot(history.history['accuracy'],'b')\n","pylab.title('Model accuracy')\n","pylab.ylabel('Accuracy')\n","pylab.xlabel('Epoch')\n","pylab.legend(['Train'], loc='upper left')\n","pylab.show()\n","\n","# Plot training & validation loss values\n","pylab.plot(history.history['loss'], 'r')\n","#plt.plot(history.history['val_loss'])\n","pylab.title('Model loss')\n","pylab.ylabel('Loss')\n","pylab.xlabel('Epoch')\n","pylab.legend(['Train'], loc='upper left')\n","pylab.show()\n","\n","print ('cell finished')"],"execution_count":0,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"d-jQbtQ-g9PR","colab_type":"text"},"source":["The plots above were for the accuracy and loss over time for each epoch. Below the accuracy of model using the testing data will be evaluated using the evaluate method. A history object is once again returned but since the evaluation only runs for one epoch, the history object will return one value. "]},{"cell_type":"code","metadata":{"id":"IvniuIpxg9PR","colab_type":"code","colab":{}},"source":["# Cell 3b.11\n","\n","history=mnist_model.evaluate(\n"," test_ds, steps=100)\n","\n","\n","print ('\\n cell finished')"],"execution_count":0,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"FllXxmyig9PT","colab_type":"text"},"source":["This cell just reformats the data for ease of understanding."]},{"cell_type":"code","metadata":{"id":"EPtuL7V8g9PT","colab_type":"code","colab":{}},"source":["# Cell 3b.12\n","\n","print ('loss = {} accuracy = {}'.format (history[0], history[1]))\n","\n","print ('cell finished')\n"],"execution_count":0,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"kKUBp9TUg9PW","colab_type":"text"},"source":["As you can see the accuracy is terrible. Something is obviously wrong with the learning process. While there are lots of things that can go wrong, such as bad data, bad data pipelining, etc. hyperparameter settings can also affect the output. In this case the `LEARNING_RATE` is much too high for the first pass. \n","\n","Do the following:\n","\n","1. Restart the notebook - from the Runtime menu at the top of the page select **Restart runtime** (note: do not select \"Reset all runtimes\")\n","2. Change LEARNING_RATE - go to cell 2 and modify LEARNING_RATE to 0.001\n","3. Re-run all cells - start at cell 1 and run every cell in the notebook to cell 12\n","\n"]},{"cell_type":"markdown","metadata":{"id":"N-loTSObg9PW","colab_type":"text"},"source":["\n","## If you have time\n","You can experiment with other learning rates to see what happens"]},{"cell_type":"code","metadata":{"id":"csy_xAtjg9PW","colab_type":"code","colab":{}},"source":[""],"execution_count":0,"outputs":[]}]} 2 | -------------------------------------------------------------------------------- /04_mnist_cnn.ipynb: -------------------------------------------------------------------------------- 1 | {"nbformat":4,"nbformat_minor":0,"metadata":{"accelerator":"GPU","colab":{"name":"04_mnist_cnn.ipynb","provenance":[{"file_id":"https://github.com/edge-ai-vision/dlcvtf2/blob/master/04_mnist_cnn.ipynb","timestamp":1592152263028},{"file_id":"https://github.com/edge-ai-vision/dlcvtf2/blob/master/04_mnist_cnn.ipynb","timestamp":1591898914910}]},"kernelspec":{"name":"python3","display_name":"Python 3"}},"cells":[{"cell_type":"markdown","metadata":{"colab_type":"text","id":"I-S9EwVjCGwX"},"source":["# Deep Learning for Computer Vision with TensorFlow 2.0\n","\n","Table of contents:\n","[Lab 0](https://colab.research.google.com/github/edge-ai-vision/dlcvtf2/blob/master/00_test_install.ipynb) | \n","[Lab 1](https://colab.research.google.com/github/edge-ai-vision/dlcvtf2/blob/master/01_linear_regression.ipynb) | \n","[Lab 2](https://colab.research.google.com/github/edge-ai-vision/dlcvtf2/blob/master/02_tensorflow_logistic_regression.ipynb) | \n","[Lab 3a](https://colab.research.google.com/github/edge-ai-vision/dlcvtf2/blob/master/03a_tensorflow_deep_network.ipynb) | \n","[Lab 3b](https://colab.research.google.com/github/edge-ai-vision/dlcvtf2/blob/master/03b_deep_mnist_visualize.ipynb) | \n","[Lab 4](https://colab.research.google.com/github/edge-ai-vision/dlcvtf2/blob/master/04_mnist_cnn.ipynb) | \n","[Lab 5](https://colab.research.google.com/github/edge-ai-vision/dlcvtf2/blob/master/05_data_prep.ipynb) | \n","[Lab 6](https://colab.research.google.com/github/edge-ai-vision/dlcvtf2/blob/master/06_transfer_learning.ipynb) | \n","\n","# Lab 4: MNIST CNN\n","\n","If you ever want to start over just go to the menu at the top and select Runtime -> Restart runtime. (Please do not use Runtime -> Reset all runtimes.)"]},{"cell_type":"markdown","metadata":{"colab_type":"text","id":"QRvcL9Rah7sL"},"source":["As we saw in the lecture fully connected models are impractical for larger images. We need a better way to classify larger images and CNN models can help significantly. \n","\n","TensorFlow with Keras makes describing even complex CNN models easy. The other beauty of this framework is that the only part that will change is the model layer description. Virtually the rest of the model remains the same as used previously. \n","\n","This model takes much more computation to run on the CPU so this is where the GPU is necessary. Notice that there is also a TPU option but this model is not setup to optimally use the TPU so don't use that right now.\n"]},{"cell_type":"code","metadata":{"colab_type":"code","id":"lWFEEzLGh7sM","colab":{}},"source":["# Cell 4.1\n","%tensorflow_version 2.x\n","import tensorflow as tf\n","\n","from tensorflow import keras\n","from tensorflow.keras import layers\n","\n","import numpy as np\n","%pylab inline\n","\n","print ('cell finished')\n"],"execution_count":0,"outputs":[]},{"cell_type":"markdown","metadata":{"colab_type":"text","id":"lp8FUvyLh7sO"},"source":["The first cell sets up a number of constants for the training process."]},{"cell_type":"code","metadata":{"colab_type":"code","id":"Vf_yo4-Ah7sP","colab":{}},"source":["# Cell 4.2\n","\n","BATCH_SIZE=100\n","\n","HIDDEN_SIZE = 1024\n","\n","NUM_CLASSES = 10\n","IMG_HT = 28\n","IMG_W = 28\n","\n","print ('cell finished')\n"],"execution_count":0,"outputs":[]},{"cell_type":"markdown","metadata":{"colab_type":"text","id":"-oBxjmwVh7sS"},"source":["Now we load the MNIST dataset and create TensorFlow datasets to be used in the training process, just as we have been doing in previous examples.\n"]},{"cell_type":"code","metadata":{"colab_type":"code","id":"WwBpB5j0h7sS","colab":{}},"source":["# Cell 4.3\n","\n","(x_train, y_train), (x_test, y_test) = tf.keras.datasets.mnist.load_data()\n","# Numpy defaults to dtype=float64; TF defaults to float32. Stick with float32.\n","x_train, x_test = x_train / np.float32(255), x_test / np.float32(255)\n","y_train, y_test = y_train.astype(np.int64), y_test.astype(np.int64)\n","\n","train_ds = tf.data.Dataset.from_tensor_slices((x_train, y_train)).repeat()\n","test_ds = tf.data.Dataset.from_tensor_slices((x_test, y_test))\n","\n","# DON'T CHANGE THESE VARIABLES OR YOU WILL BREAK THINGS\n","NUM_PIXELS = 28 * 28\n","NUM_CLASSES = 10\n","\n","print ('cell finished')\n"],"execution_count":0,"outputs":[]},{"cell_type":"code","metadata":{"colab_type":"code","id":"Oy2Ztpp2h7sU","colab":{}},"source":["# Cell 4.4\n","\n","train_ds = train_ds.shuffle(60000).batch(BATCH_SIZE)\n","test_ds = test_ds.batch(BATCH_SIZE)\n","\n","print ('cell finished')\n"],"execution_count":0,"outputs":[]},{"cell_type":"markdown","metadata":{"colab_type":"text","id":"bDKwvbUyh7sW"},"source":["This cell shows how to describe the CNN model using TensorFlow and Keras. The model function contains a sequential model that describes all of the layers of our network. \n","\n","This function accepts a single input 2D image and each of the layers passes the output tensor to the next layer. \n","\n","The first convolutional layer accepts the input image and generates an output tensor which is the input to the next layer. The next layer generates a new output applied to the next layer. This occurs successively until all of the layers have been defined. \n","\n","Notice that each layer has a number of parameters that allow specification of the activation type, number of filters, kernel size, etc. There are a lot more parameters that are defaulted but can be specified if needed to get better results. \n","\n","The Dropout layer is used when the model is fit to the data, but will not be used during evaluation or prediction. This behavior is built in to Keras."]},{"cell_type":"code","metadata":{"colab_type":"code","id":"OgNNUkvUh7sX","colab":{}},"source":["# Cell 4.5\n","\n","def model():\n"," model = keras.Sequential([\n"," layers.Reshape(\n"," target_shape=[28, 28, 1],\n"," input_shape=(28, 28,)),\n"," \n"," layers.Conv2D(filters=32,kernel_size=(5,5), padding='same', activation='relu'),\n"," layers.MaxPooling2D(pool_size=(2,2), strides=(2,2), padding='same'),\n"," layers.Conv2D(filters=64,kernel_size=(5,5), padding='same', activation='relu'),\n"," layers.MaxPooling2D(pool_size=(2,2), strides=(2,2), padding='same'),\n"," layers.Flatten(),\n"," layers.Dense(HIDDEN_SIZE, activation='relu'),\n"," layers.Dropout(rate=0.4),\n"," layers.Dense(NUM_CLASSES, activation='softmax')\n"," ])\n"," return model\n","\n","print ('cell finished')\n"],"execution_count":0,"outputs":[]},{"cell_type":"markdown","metadata":{"colab_type":"text","id":"2SphimLph7sY"},"source":["Just as in previous networks the model function will create the model. Notice the summary shows a much bigger model than any used previously"]},{"cell_type":"code","metadata":{"colab_type":"code","id":"It-9txwLh7sZ","colab":{}},"source":["# Cell 4.6\n","\n","conv_model = model() # calls the layer function\n","\n","conv_model.summary()\n","\n","print ('cell finished')"],"execution_count":0,"outputs":[]},{"cell_type":"markdown","metadata":{"colab_type":"text","id":"S6kyyGAlh7sb"},"source":["Notice the the compilation parameters are exactly as used before, even though the model is much larger."]},{"cell_type":"code","metadata":{"colab_type":"code","id":"bw963Ffyh7sb","colab":{}},"source":["# Cell 4.7\n","\n","conv_model.compile(optimizer='adam', \n"," loss='sparse_categorical_crossentropy',\n"," metrics=['accuracy'])\n","\n","print ('cell finished')\n"],"execution_count":0,"outputs":[]},{"cell_type":"markdown","metadata":{"colab_type":"text","id":"0sbjw8mwh7se"},"source":["The next cell will train or fit the model. It will use the training dataset train_ds, run 50 times or 50 epochs through the data, with 600 minibatches per epoch. If you take the steps_per_epoch * the mini-batch size, it should be greater than or equal to the size of the training set. \n","\n","Since this is a much larger model than previously used, it will take much longer to train on a CPU. A GPU is needed for reasonable performance in class."]},{"cell_type":"code","metadata":{"colab_type":"code","id":"pQgTVWGxh7se","colab":{}},"source":["# Cell 4.8\n","\n","history = conv_model.fit(\n"," train_ds,\n"," epochs=50, \n"," steps_per_epoch=600,verbose=2)\n","\n","\n","print ('cell finished')\n"],"execution_count":0,"outputs":[]},{"cell_type":"markdown","metadata":{"colab_type":"text","id":"K8RYJ0H2h7sg"},"source":["As in previous models the accuracy of the model is plot as before. The accuracy should be better, even though this is a tiny dataset. "]},{"cell_type":"code","metadata":{"colab_type":"code","id":"t8KUIZyAh7sh","colab":{}},"source":["# Cell 4.9\n","\n","pylab.plot(history.history['accuracy'],'b')\n","pylab.title('Model accuracy')\n","pylab.ylabel('Accuracy')\n","pylab.xlabel('Epoch')\n","pylab.legend(['Train'], loc='upper left')\n","pylab.show()\n","\n","# Plot training & validation loss values\n","pylab.plot(history.history['loss'], 'r')\n","#plt.plot(history.history['val_loss'])\n","pylab.title('Model loss')\n","pylab.ylabel('Loss')\n","pylab.xlabel('Epoch')\n","pylab.legend(['Train'], loc='upper left')\n","pylab.show()\n","\n","print ('cell finished')\n"],"execution_count":0,"outputs":[]},{"cell_type":"markdown","metadata":{"colab_type":"text","id":"g9eJXUeCh7sj"},"source":["Just like before this will check the model on the test data"]},{"cell_type":"code","metadata":{"colab_type":"code","id":"3MZ-vE3Qh7sj","colab":{}},"source":["# Cell 4.10\n","\n","results=conv_model.evaluate(\n"," test_ds, steps=100)\n","\n","print(\"loss = {}\".format(results[0]))\n","print(\"accuracy = {}\".format(results[1]))\n","\n","print ('cell finished')\n"],"execution_count":0,"outputs":[]},{"cell_type":"markdown","metadata":{"colab_type":"text","collapsed":true,"id":"35VNik4Lh7sm"},"source":["When working with images even a less than 1 percent difference can make a big difference in the utility of the classifier. Notice that the accuracy achieved with the convolutional model is significantly better than even the deep network from the last exercise. "]},{"cell_type":"code","metadata":{"colab_type":"code","id":"RO_0AZruh7sm","colab":{}},"source":[""],"execution_count":0,"outputs":[]}]} 2 | -------------------------------------------------------------------------------- /05_data_prep.ipynb: -------------------------------------------------------------------------------- 1 | {"nbformat":4,"nbformat_minor":0,"metadata":{"accelerator":"GPU","colab":{"name":"05_data_prep.ipynb","provenance":[{"file_id":"https://github.com/edge-ai-vision/dlcvtf2/blob/master/05_data_prep.ipynb","timestamp":1591899020705}]},"kernelspec":{"name":"python3","display_name":"Python 3"}},"cells":[{"cell_type":"markdown","metadata":{"colab_type":"text","id":"I-S9EwVjCGwX"},"source":["# Deep Learning for Computer Vision with TensorFlow 2.0\n","\n","Table of contents:\n","[Lab 0](https://colab.research.google.com/github/edge-ai-vision/dlcvtf2/blob/master/00_test_install.ipynb) | \n","[Lab 1](https://colab.research.google.com/github/edge-ai-vision/dlcvtf2/blob/master/01_linear_regression.ipynb) | \n","[Lab 2](https://colab.research.google.com/github/edge-ai-vision/dlcvtf2/blob/master/02_tensorflow_logistic_regression.ipynb) | \n","[Lab 3a](https://colab.research.google.com/github/edge-ai-vision/dlcvtf2/blob/master/03a_tensorflow_deep_network.ipynb) | \n","[Lab 3b](https://colab.research.google.com/github/edge-ai-vision/dlcvtf2/blob/master/03b_deep_mnist_visualize.ipynb) | \n","[Lab 4](https://colab.research.google.com/github/edge-ai-vision/dlcvtf2/blob/master/04_mnist_cnn.ipynb) | \n","[Lab 5](https://colab.research.google.com/github/edge-ai-vision/dlcvtf2/blob/master/05_data_prep.ipynb) | \n","[Lab 6](https://colab.research.google.com/github/edge-ai-vision/dlcvtf2/blob/master/06_transfer_learning.ipynb) | \n","\n","# Lab 5: Data Prep\n","\n","If you ever want to start over just go to the menu at the top and select Runtime -> Restart runtime. (Please do not use Runtime -> Reset all runtimes.)"]},{"cell_type":"markdown","metadata":{"colab_type":"text","id":"aOhB_5YVirXW"},"source":["In this notebook we will delve further into dataset creation. The MNIST dataset used previously was extremly simple and this notebook will give you a better idea of how this is done for typical applications. \n"]},{"cell_type":"code","metadata":{"colab_type":"code","id":"M5YWNi68irXW","colab":{}},"source":["# Cell 5.1\n","%tensorflow_version 2.x\n","import tensorflow as tf\n","\n","from __future__ import absolute_import, division, print_function\n","\n","import os\n","\n","from tensorflow import keras\n","print(\"TensorFlow version is \", tf.__version__)\n","\n","import numpy as np\n","\n","import matplotlib.pyplot as plt\n","import matplotlib.image as mpimg\n","import matplotlib.transforms as trans\n","\n","from PIL import Image, ImageEnhance, ImageFilter\n","\n","%pylab inline"],"execution_count":0,"outputs":[]},{"cell_type":"markdown","metadata":{"colab_type":"text","id":"xtExOylGirXZ"},"source":["The next cell will download a cat and dog dataset so that the contents can be examined in more detail. The idea is to show how this process is typically completed.\n","\n","The `base_dir` variable is set to the top of the directories to be examined. "]},{"cell_type":"code","metadata":{"colab_type":"code","id":"uOvVfxfzirXa","colab":{}},"source":["# Cell 5.2\n","\n","zip_file = tf.keras.utils.get_file(origin=\"https://storage.googleapis.com/mledu-datasets/cats_and_dogs_filtered.zip\", \n"," cache_dir='.',fname=\"cats_and_dogs_filtered.zip\", extract=True)\n","\n","base_dir, _ = os.path.splitext(zip_file)\n","print(base_dir)"],"execution_count":0,"outputs":[]},{"cell_type":"markdown","metadata":{"colab_type":"text","id":"4ybj1_3dirXc"},"source":["Show the directory structure\n"]},{"cell_type":"code","metadata":{"colab_type":"code","id":"PDUykgXairXc","colab":{}},"source":["# Cell 5.3\n","\n","# code to show directory structure\n","print(os.listdir('./datasets'))\n","\n","print(os.listdir('./datasets/cats_and_dogs_filtered'))\n","\n","print(os.listdir('./datasets/cats_and_dogs_filtered/train'))\n","\n","dog_list = os.listdir('./datasets/cats_and_dogs_filtered/train/dogs')\n","print (dog_list[0:5])\n","print (len(dog_list))\n","\n","cat_list = os.listdir('./datasets/cats_and_dogs_filtered/train/cats')\n","print (cat_list[0:5])\n","print (len(cat_list))"],"execution_count":0,"outputs":[]},{"cell_type":"markdown","metadata":{"colab_type":"text","id":"F8a4hhSkirXe"},"source":["Explore table of cat pics"]},{"cell_type":"code","metadata":{"colab_type":"code","id":"PJR5uubdirXe","colab":{}},"source":["# Cell 5.4\n","\n","# code to show table of pics from cats\n","\n","# update offset by values of 9 to see the next set of pictures\n","offset = 9\n","\n","plt.figure(figsize=(10,10))\n","for i in range(9):\n"," plt.subplot(330 + 1 + i)\n"," img_file = plt.imread('./datasets/cats_and_dogs_filtered/train/cats/' + cat_list[i+ offset])\n"," #img_file = plt.resize()\n"," plt.imshow(img_file)\n","\n","plt.show()"],"execution_count":0,"outputs":[]},{"cell_type":"markdown","metadata":{"colab_type":"text","id":"X8OpL1wiirXg"},"source":["Explore table of dog pics"]},{"cell_type":"code","metadata":{"colab_type":"code","id":"pbmGif25irXh","colab":{}},"source":["# Cell 5.5\n","\n","# code to show table of pics from dogs\n","\n","# update offset by values of 9 to see the next set of pictures\n","offset = 9\n","\n","plt.figure(figsize=(10,10))\n","for i in range(9):\n"," plt.subplot(330 + 1 + i)\n"," img_file = plt.imread('./datasets/cats_and_dogs_filtered/train/dogs/' + dog_list[i+offset])\n"," #img_file = plt.resize()\n"," plt.imshow(img_file)\n","plt.show()"],"execution_count":0,"outputs":[]},{"cell_type":"markdown","metadata":{"colab_type":"text","id":"IeNx4ugxirXl"},"source":["The next cell shows how the training and validation directories are setup for the training process. These paths are used to create the datasets that will be used during training and validation. "]},{"cell_type":"code","metadata":{"colab_type":"code","id":"OeeaCSzYirXl","colab":{}},"source":["# Cell 5.6\n","\n","train_dir = os.path.join(base_dir, 'train')\n","validation_dir = os.path.join(base_dir, 'validation')\n","\n","# Directory with our training cat pictures\n","train_cats_dir = os.path.join(train_dir, 'cats')\n","print ('Total training cat images:', len(os.listdir(train_cats_dir)))\n","\n","# Directory with our training dog pictures\n","train_dogs_dir = os.path.join(train_dir, 'dogs')\n","print ('Total training dog images:', len(os.listdir(train_dogs_dir)))\n","\n","# Directory with our validation cat pictures\n","validation_cats_dir = os.path.join(validation_dir, 'cats')\n","print ('Total validation cat images:', len(os.listdir(validation_cats_dir)))\n","\n","# Directory with our validation dog pictures\n","validation_dogs_dir = os.path.join(validation_dir, 'dogs')\n","print ('Total validation dog images:', len(os.listdir(validation_dogs_dir)))"],"execution_count":0,"outputs":[]},{"cell_type":"markdown","metadata":{"colab_type":"text","id":"GqjjKTyFirXo"},"source":["Show function call for the Image Data Generator class and some common arguments"]},{"cell_type":"code","metadata":{"colab_type":"code","id":"b0giOm8LirXp","colab":{}},"source":["# Cell 5.7\n","\n","gen =keras.preprocessing.image.ImageDataGenerator(\n"," rotation_range=20,\n"," width_shift_range=0.2,\n"," height_shift_range=0.2,\n"," shear_range=10,\n"," zoom_range=0.2,\n"," channel_shift_range=0.2,\n"," horizontal_flip=True,\n"," rescale=1./255)\n","\n","gimg = plt.imread('./datasets/cats_and_dogs_filtered/train/cats/cat.952.jpg')\n","ximg = np.expand_dims(gimg,0)\n","\n","img_iter = gen.flow(ximg)\n","\n","#aug_imgs=[next(img_iter)[0].astype(np.uint8) for i in range(9)]\n","aug_imgs=[next(img_iter)[0] for i in range(9)]\n","\n","plt.figure(figsize=(10,10))\n","for i in range(9):\n"," plt.subplot(330 + 1 + i)\n"," img_file = aug_imgs[i]\n"," plt.imshow(img_file)\n","plt.show()"],"execution_count":0,"outputs":[]},{"cell_type":"markdown","metadata":{"colab_type":"text","id":"c143kLEnirXr"},"source":["Using the directories provided above two ImageDataGenerators can be created, one for training, the other for validation. These generators will apply augmentations to the images as they are pulled from the training and validation directories. "]},{"cell_type":"code","metadata":{"colab_type":"code","id":"z2INI9s6irXr","colab":{}},"source":["# Cell 5.8\n","\n","image_size = 160 # All images will be resized to 160x160\n","batch_size = 9\n","\n","# Rescale all images by 1./255 and apply image augmentation\n","train_datagen = keras.preprocessing.image.ImageDataGenerator(\n"," rescale=1./255)\n","\n","validation_datagen = keras.preprocessing.image.ImageDataGenerator(rescale=1./255)\n","\n","# Flow training images in batches of 20 using train_datagen generator\n","train_generator = train_datagen.flow_from_directory(\n"," train_dir, # Source directory for the training images\n"," target_size=(image_size, image_size), \n"," batch_size=batch_size,\n"," # Since we use binary_crossentropy loss, we need binary labels\n"," class_mode='binary')\n","\n","# Flow validation images in batches of 20 using test_datagen generator\n","validation_generator = validation_datagen.flow_from_directory(\n"," validation_dir, # Source directory for the validation images\n"," target_size=(image_size, image_size),\n"," batch_size=batch_size,\n"," class_mode='binary')\n","\n","t_imgs = iter(train_generator)\n","\n","plt.figure(figsize=(10,10))\n","for i in range(9):\n"," plt.subplot(330 + 1 + i)\n"," img_file = t_imgs[0][0][i]\n"," plt.imshow(img_file)\n","plt.show()\n","\n","print ('Now the validation images')\n","print ('')\n","print ('')\n","\n","\n","\n","v_imgs = iter(validation_generator)\n","\n","plt.figure(figsize=(10,10))\n","for i in range(9):\n"," plt.subplot(330 + 1 + i)\n"," img_file = v_imgs[0][0][i]\n"," plt.imshow(img_file)\n","plt.show()"],"execution_count":0,"outputs":[]},{"cell_type":"markdown","metadata":{"colab_type":"text","id":"Quh0cCuEirXt"},"source":[""]},{"cell_type":"code","metadata":{"colab_type":"code","id":"s7Jc9tZdirXu","colab":{}},"source":[""],"execution_count":0,"outputs":[]}]} 2 | -------------------------------------------------------------------------------- /06_transfer_learning.ipynb: -------------------------------------------------------------------------------- 1 | {"nbformat":4,"nbformat_minor":0,"metadata":{"colab":{"name":"transfer_learning.ipynb","provenance":[{"file_id":"https://github.com/edge-ai-vision/dlcvtf2/blob/master/06_transfer_learning.ipynb","timestamp":1591899129657}],"private_outputs":true,"collapsed_sections":[]},"kernelspec":{"name":"python3","display_name":"Python 3"},"accelerator":"GPU"},"cells":[{"cell_type":"markdown","metadata":{"accelerator":"GPU","colab_type":"text","id":"77gENRVX40S7"},"source":["##### Copyright 2019 The TensorFlow Authors."]},{"cell_type":"markdown","metadata":{"id":"wguY0pZPvvzt","colab_type":"text"},"source":[""]},{"cell_type":"code","metadata":{"cellView":"form","colab_type":"code","id":"d8jyt37T42Vf","colab":{}},"source":["#@title Licensed under the Apache License, Version 2.0 (the \"License\");\n","# you may not use this file except in compliance with the License.\n","# You may obtain a copy of the License at\n","#\n","# https://www.apache.org/licenses/LICENSE-2.0\n","#\n","# Unless required by applicable law or agreed to in writing, software\n","# distributed under the License is distributed on an \"AS IS\" BASIS,\n","# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n","# See the License for the specific language governing permissions and\n","# limitations under the License."],"execution_count":0,"outputs":[]},{"cell_type":"code","metadata":{"cellView":"form","colab_type":"code","id":"aPxHdjwW5P2j","colab":{}},"source":["#@title MIT License\n","#\n","# Copyright (c) 2017 François Chollet # IGNORE_COPYRIGHT: cleared by OSS licensing\n","#\n","# Permission is hereby granted, free of charge, to any person obtaining a\n","# copy of this software and associated documentation files (the \"Software\"),\n","# to deal in the Software without restriction, including without limitation\n","# the rights to use, copy, modify, merge, publish, distribute, sublicense,\n","# and/or sell copies of the Software, and to permit persons to whom the\n","# Software is furnished to do so, subject to the following conditions:\n","#\n","# The above copyright notice and this permission notice shall be included in\n","# all copies or substantial portions of the Software.\n","#\n","# THE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR\n","# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,\n","# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL\n","# THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER\n","# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING\n","# FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER\n","# DEALINGS IN THE SOFTWARE."],"execution_count":0,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"aZQcPlosr4zq","colab_type":"text"},"source":["(This exercise is part of the TensorFlow Core training materials and has been slightly modified to update it to TensorFlow 2.0.) "]},{"cell_type":"markdown","metadata":{"colab_type":"text","id":"eBkwdlalvvK6"},"source":["\n"," \n"," \n"," \n","
\n"," View on TensorFlow.org\n"," \n"," Run in Google Colab\n"," \n"," View source on GitHub\n","
"]},{"cell_type":"markdown","metadata":{"colab_type":"text","id":"hRTa3Ee15WsJ"},"source":["# Transfer Learning Using Pretrained ConvNets"]},{"cell_type":"markdown","metadata":{"id":"03Di_fd8vadr","colab_type":"text"},"source":["Table of contents:\n","[Lab 0](https://colab.research.google.com/github/edge-ai-vision/dlcvtf2/blob/master/00_test_install.ipynb) | \n","[Lab 1](https://colab.research.google.com/github/edge-ai-vision/dlcvtf2/blob/master/01_linear_regression.ipynb) | \n","[Lab 2](https://colab.research.google.com/github/edge-ai-vision/dlcvtf2/blob/master/02_tensorflow_logistic_regression.ipynb) | \n","[Lab 3a](https://colab.research.google.com/github/edge-ai-vision/dlcvtf2/blob/master/03a_tensorflow_deep_network.ipynb) | \n","[Lab 3b](https://colab.research.google.com/github/edge-ai-vision/dlcvtf2/blob/master/03b_deep_mnist_visualize.ipynb) | \n","[Lab 4](https://colab.research.google.com/github/edge-ai-vision/dlcvtf2/blob/master/04_mnist_cnn.ipynb) | \n","[Lab 5](https://colab.research.google.com/github/edge-ai-vision/dlcvtf2/blob/master/05_data_prep.ipynb) | \n","[Lab 6](https://colab.research.google.com/github/edge-ai-vision/dlcvtf2/blob/master/06_transfer_learning.ipynb) | "]},{"cell_type":"markdown","metadata":{"colab_type":"text","id":"2X4KyhORdSeO"},"source":["In this tutorial we will discuss how to classify cats vs dogs images by using transfer learning from a pre-trained network. This will allows us to get higher accuracies than we saw by training our network from scratch.\n","\n","A **pre-trained model** is a saved network that was previously trained on a large dataset, typically on a large-scale image-classification task. We can either use the pretrained model as it is or transfer learning using the pretrained convents. The intuition behind **transfer learning** is that if this model trained on a large and general enough dataset, this model will effectively serve as a generic model of the visual world. We can leverage these learned feature maps without having to train a large model on a large dataset by using these models as the basis of our own model specific to our task. There are 2 scenarios of transfer learning using a pretrained model:\n","\n","1. **Feature Extraction** - use the representations of learned by a previous network to extract meaningful features from new samples. We simply add a new classifier, which will be trained from scratch, on top of the pretrained model so that we can repurpose the feature maps learned previously for our dataset. **Do we use the entire pretrained model or just the convolutional base?** - We use the feature extraction portion of these pretrained convnets (convolutional base) since they are likely to be generic features and learned concepts over a picture. However, the classification part of the pretrained model is often specific to original classification task, and subsequently specific to the set of classes on which the model was trained. \n","2. **Fine-Tuning** - unfreezing a few of the top layers of a frozen model base used for feature extraction, and jointly training both the newly added classifier layers as well as the last layers of the frozen model. This allows us to \"fine tune\" the higher order feature representations in addition to our final classifier in order to make them more relevant for the specific task involved. \n","\n","**We will follow the general machine learning workflow:**\n","1. Examine and understand data\n","2. Build an input pipeline - using Keras ImageDataGenerator as we did in the image classification tutorial\n","3. Compose our model \n"," * Load in our pretrained model (and pretrained weights)\n"," * Stack our classification layers on top\n","4. Train our model\n","5. Evaluate model\n","\n","We will see an example of using the pre-trained convnet as the feature extraction and then fine-tune to train the last few layers of the base model.\n","\n","**Audience:** This post is geared towards beginners with some Keras API and ML background. To get the most out of this post, you should have some basic ML background, know what CNNs are, and be familiar with the Keras Sequential API. \n","\n","**Time Estimated**: 30 minutes"]},{"cell_type":"code","metadata":{"colab_type":"code","id":"iBMcobPHdD8O","colab":{}},"source":["# Cell 6.1\n","from __future__ import absolute_import, division, print_function\n","\n","%tensorflow_version 2.x\n","import tensorflow as tf\n","\n","import os\n","\n","from tensorflow import keras\n","print(\"TensorFlow version is \", tf.__version__)\n","\n","import numpy as np\n","\n","import matplotlib.pyplot as plt\n","import matplotlib.image as mpimg"],"execution_count":0,"outputs":[]},{"cell_type":"markdown","metadata":{"colab_type":"text","id":"v77rlkCKW0IJ"},"source":["## Data preprocessing"]},{"cell_type":"markdown","metadata":{"colab_type":"text","id":"aXzwKdouXf1h"},"source":["### Download data - cats_and_dogs_filtered.zip\n","We will download a filtered version of Kaggle's [Dogs vs Cats](https://www.kaggle.com/c/dogs-vs-cats/data) dataset. Then store the downloaded zip file to the \"/tmp/\" directory."]},{"cell_type":"code","metadata":{"colab_type":"code","id":"nRnO59Kr6enO","colab":{}},"source":["# Cell 6.2\n","zip_file = tf.keras.utils.get_file(origin=\"https://storage.googleapis.com/mledu-datasets/cats_and_dogs_filtered.zip\", \n"," fname=\"cats_and_dogs_filtered.zip\", extract=True)\n","base_dir, _ = os.path.splitext(zip_file)\n","print ('Cell finished.')"],"execution_count":0,"outputs":[]},{"cell_type":"markdown","metadata":{"colab_type":"text","id":"9_6h-c5EXN91"},"source":["### Prepare training and validation cats and dogs datasets\n","Create the training and validation directories for cats datasets and dog datasets."]},{"cell_type":"code","metadata":{"colab_type":"code","id":"RWcldM4TXLen","colab":{}},"source":["# Cell 6.3\n","train_dir = os.path.join(base_dir, 'train')\n","validation_dir = os.path.join(base_dir, 'validation')\n","\n","# Directory with our training cat pictures\n","train_cats_dir = os.path.join(train_dir, 'cats')\n","print ('Total training cat images:', len(os.listdir(train_cats_dir)))\n","\n","# Directory with our training dog pictures\n","train_dogs_dir = os.path.join(train_dir, 'dogs')\n","print ('Total training dog images:', len(os.listdir(train_dogs_dir)))\n","\n","# Directory with our validation cat pictures\n","validation_cats_dir = os.path.join(validation_dir, 'cats')\n","print ('Total validation cat images:', len(os.listdir(validation_cats_dir)))\n","\n","# Directory with our validation dog pictures\n","validation_dogs_dir = os.path.join(validation_dir, 'dogs')\n","print ('Total validation dog images:', len(os.listdir(validation_dogs_dir)))\n","\n","print ('Cell finished.')"],"execution_count":0,"outputs":[]},{"cell_type":"markdown","metadata":{"colab_type":"text","id":"wvidPx6jeFzf"},"source":["### Create Image Data Generator with Image Augmentation\n","\n","We will use ImageDataGenerator to rescale the images. \n","\n","Create the train generator and specify where the train dataset directory, image size, batch size and it's a binary classification.\n","\n","Create the validation generator with similar approach as the train generator with the flow_from_directory() method."]},{"cell_type":"code","metadata":{"colab_type":"code","id":"y3PM6GVHcC31","colab":{}},"source":["# Cell 6.4\n","image_size = 160 # All images will be resized to 160x160\n","batch_size = 32\n","\n","# Rescale all images by 1./255 and apply image augmentation\n","train_datagen = keras.preprocessing.image.ImageDataGenerator(\n"," rescale=1./255)\n","\n","validation_datagen = keras.preprocessing.image.ImageDataGenerator(rescale=1./255)\n","\n","# Flow training images in batches of 20 using train_datagen generator\n","train_generator = train_datagen.flow_from_directory(\n"," train_dir, # Source directory for the training images\n"," target_size=(image_size, image_size), \n"," batch_size=batch_size,\n"," # Since we use binary_crossentropy loss, we need binary labels\n"," class_mode='binary')\n","\n","# Flow validation images in batches of 20 using test_datagen generator\n","validation_generator = validation_datagen.flow_from_directory(\n"," validation_dir, # Source directory for the validation images\n"," target_size=(image_size, image_size),\n"," batch_size=batch_size,\n"," class_mode='binary')\n","print ('Cell finished.')"],"execution_count":0,"outputs":[]},{"cell_type":"markdown","metadata":{"colab_type":"text","id":"OkH-kazQecHB"},"source":["## Create the base model from the pre-trained convnets\n","We will create the base model from the **MobileNet V2** model developed at Google, and pre-trained on the ImageNet dataset, a large dataset of 1.4M images and 1000 classes of web images. This is a powerful model. Let's see what the features that it has learned can do for our cat vs. dog problem.\n","\n","First, we need to pick which intermediate layer of MobileNet V2 we will use for feature extraction. A common practice is to use the output of the very last layer before the flatten operation, the so-called \"bottleneck layer\". The reasoning here is that the following fully-connected layers will be too specialized to the task the network was trained on, and thus the features learned by these layers won't be very useful for a new task. The bottleneck features, however, retain much generality.\n","\n","Let's instantiate an MobileNet V2 model pre-loaded with weights trained on ImageNet. By specifying the **include_top=False** argument, we load a network that doesn't include the classification layers at the top, which is ideal for feature extraction."]},{"cell_type":"code","metadata":{"colab_type":"code","id":"19IQ2gqneqmS","colab":{}},"source":["# Cell 6.5\n","IMG_SHAPE = (image_size, image_size, 3)\n","\n","# Create the base model from the pre-trained model MobileNet V2\n","base_model = tf.keras.applications.MobileNetV2(input_shape=IMG_SHAPE,\n"," include_top=False, \n"," weights='imagenet')\n","print ('Cell finished.')"],"execution_count":0,"outputs":[]},{"cell_type":"markdown","metadata":{"colab_type":"text","id":"rlx56nQtfe8Y"},"source":["## Feature extraction\n","We will freeze the convolutional base created from the previous step and use that as a feature extractor, add a classifier on top of it and train the top-level classifier."]},{"cell_type":"markdown","metadata":{"colab_type":"text","id":"CnMLieHBCwil"},"source":["### Freeze the convolutional base\n","It's important to freeze the convolutional based before we compile and train the model. By freezing (or setting `layer.trainable = False`), we prevent the weights in these layers from being updated during training."]},{"cell_type":"code","metadata":{"colab_type":"code","id":"OTCJH4bphOeo","colab":{}},"source":["# Cell 6.6\n","base_model.trainable = False\n","print ('Cell finished.')"],"execution_count":0,"outputs":[]},{"cell_type":"code","metadata":{"colab_type":"code","id":"KpbzSmPkDa-N","colab":{}},"source":["# Cell 6.7\n","# Let's take a look at the base model architecture\n","base_model.summary()\n","print ('Cell finished.')"],"execution_count":0,"outputs":[]},{"cell_type":"markdown","metadata":{"colab_type":"text","id":"wdMRM8YModbk"},"source":["#### Add a classification head"]},{"cell_type":"markdown","metadata":{"colab_type":"text","id":"0iqnBeZrfoIc"},"source":["Now let's add a few layers on top of the base model:"]},{"cell_type":"code","metadata":{"colab_type":"code","id":"eApvroIyn1K0","colab":{}},"source":["# Cell 6.7\n","model = tf.keras.Sequential([\n"," base_model,\n"," keras.layers.GlobalAveragePooling2D(),\n"," keras.layers.Dense(1, activation='sigmoid')\n","])\n","print ('Cell finished.')"],"execution_count":0,"outputs":[]},{"cell_type":"markdown","metadata":{"colab_type":"text","id":"g0ylJXE_kRLi"},"source":["### Compile the model\n","\n","You must compile the model before training it. "]},{"cell_type":"code","metadata":{"colab_type":"code","id":"RpR8HdyMhukJ","colab":{}},"source":["# Cell 6.8\n","model.compile(optimizer=tf.keras.optimizers.RMSprop(lr=0.0001), \n"," loss='binary_crossentropy', \n"," metrics=['accuracy'])\n","print ('Cell finished.')"],"execution_count":0,"outputs":[]},{"cell_type":"code","metadata":{"colab_type":"code","id":"I8ARiyMFsgbH","colab":{}},"source":["# Cell 6.9\n","model.summary()\n","print ('Cell finished.')"],"execution_count":0,"outputs":[]},{"cell_type":"markdown","metadata":{"colab_type":"text","id":"lxOcmVr0ydFZ"},"source":["These 1.2K trainable parameters are divided among 2 TensorFlow `Variable` objects, the weights and biases of the two dense layers: "]},{"cell_type":"code","metadata":{"colab_type":"code","id":"krvBumovycVA","colab":{}},"source":["# Cell 6.10\n","print (len(model.trainable_variables))\n","print ('Cell finished.')"],"execution_count":0,"outputs":[]},{"cell_type":"markdown","metadata":{"colab_type":"text","id":"RxvgOYTDSWTx"},"source":["### Train the model\n","\n","After training for 10 epochs, we are able to get ~94% accuracy. \n","\n","If you have more time, train it to convergence (50 epochs, ~96% accuracy)\n"]},{"cell_type":"code","metadata":{"colab_type":"code","id":"Om4O3EESkab1","colab":{}},"source":["# Cell 6.11\n","epochs = 10\n","steps_per_epoch = train_generator.n // batch_size\n","validation_steps = validation_generator.n // batch_size\n","\n","history = model.fit_generator(train_generator, \n"," steps_per_epoch = steps_per_epoch,\n"," epochs=epochs, \n"," workers=4,\n"," validation_data=validation_generator, \n"," validation_steps=validation_steps)\n","print ('Cell finished.')"],"execution_count":0,"outputs":[]},{"cell_type":"markdown","metadata":{"colab_type":"text","id":"Hd94CKImf8vi"},"source":["### Learning curves\n","\n","Let's take a look at the learning curves of the training and validation accuracy / loss, when using the MobileNet V2 base model as a fixed feature extractor. "]},{"cell_type":"markdown","metadata":{"colab_type":"text","id":"l7HOsQTPNgO9"},"source":["If you train to convergence (`epochs=50`) the resulting graph should look like this:\n","\n","![Before fine tuning, the model reaches 96% accuracy](https://github.com/tensorflow/docs/blob/master/site/en/tutorials/images/images/before_fine_tuning.png?raw=1)"]},{"cell_type":"code","metadata":{"colab_type":"code","id":"53OTCh3jnbwV","colab":{}},"source":["# Cell 6.12\n","acc = history.history['accuracy']\n","val_acc = history.history['val_accuracy']\n","\n","loss = history.history['loss']\n","val_loss = history.history['val_loss']\n","\n","plt.figure(figsize=(8, 8))\n","plt.subplot(2, 1, 1)\n","plt.plot(acc, label='Training Accuracy')\n","plt.plot(val_acc, label='Validation Accuracy')\n","plt.legend(loc='lower right')\n","plt.ylabel('Accuracy')\n","plt.ylim([min(plt.ylim()),1])\n","plt.title('Training and Validation Accuracy')\n","\n","plt.subplot(2, 1, 2)\n","plt.plot(loss, label='Training Loss')\n","plt.plot(val_loss, label='Validation Loss')\n","plt.legend(loc='upper right')\n","plt.ylabel('Cross Entropy')\n","plt.ylim([0,max(plt.ylim())])\n","plt.title('Training and Validation Loss')\n","plt.show()\n","print ('Cell finished.')"],"execution_count":0,"outputs":[]},{"cell_type":"markdown","metadata":{"colab_type":"text","id":"CqwV-CRdS6Nv"},"source":["## Fine tuning\n","In our feature extraction experiment, we were only training a few layers on top of an MobileNet V2 base model. The weights of the pre-trained network were **not** updated during training. One way to increase performance even further is to \"fine-tune\" the weights of the top layers of the pre-trained model alongside the training of the top-level classifier. The training process will force the weights to be tuned from generic features maps to features associated specifically to our dataset.\n","\n","Note: this should only be attempted after you have trained the top-level classifier with the pre-trained model set to non-trainable. If you add a randomly initialized classifier on top of a pre-trained model and attempt to train all layers jointly, the magnitude of the gradient updates will be too large (due to the random weights from the classifier) and your pre-trained model will just forget everything it has learned.\n","\n","Additionally, the reasoning behind fine-tuning the top layers of the pre-trained model rather than all layers of the pre-trained model is the following: in a convnet, the higher up a layer is, the more specialized it is. The first few layers in a convnet learned very simple and generic features, which generalize to almost all types of images. But as you go higher up, the features are increasingly more specific to the dataset that the model was trained on. The goal of fine-tuning is to adapt these specialized features to work with the new dataset."]},{"cell_type":"markdown","metadata":{"colab_type":"text","id":"CPXnzUK0QonF"},"source":["### Un-freeze the top layers of the model\n"]},{"cell_type":"markdown","metadata":{"colab_type":"text","id":"rfxv_ifotQak"},"source":["All we need to do is unfreeze the `base_model`, and set the bottom layers be un-trainable. Then, recompile the model (necessary for these changes to take effect), and resume training. "]},{"cell_type":"code","metadata":{"colab_type":"code","id":"4nzcagVitLQm","colab":{}},"source":["# Cell 6.13\n","base_model.trainable = True\n","print ('Cell finished.')"],"execution_count":0,"outputs":[]},{"cell_type":"code","metadata":{"colab_type":"code","id":"-4HgVAacRs5v","colab":{}},"source":["# Cell 6.14\n","# Let's take a look to see how many layers are in the base model\n","print(\"Number of layers in the base model: \", len(base_model.layers))\n","\n","# Fine tune from this layer onwards\n","fine_tune_at = 100\n","\n","# Freeze all the layers before the `fine_tune_at` layer\n","for layer in base_model.layers[:fine_tune_at]:\n"," layer.trainable = False\n","\n","print ('Cell finished.')"],"execution_count":0,"outputs":[]},{"cell_type":"markdown","metadata":{"colab_type":"text","id":"4Uk1dgsxT0IS"},"source":["### Compile the model\n","\n","Compile the model using a much-lower training rate."]},{"cell_type":"code","metadata":{"colab_type":"code","id":"NtUnaz0WUDva","colab":{}},"source":["# Cell 6.15\n","\n","model.compile(loss='binary_crossentropy',\n"," optimizer = tf.keras.optimizers.RMSprop(lr=2e-5),\n"," metrics=['accuracy'])\n","print ('Cell finished.')"],"execution_count":0,"outputs":[]},{"cell_type":"code","metadata":{"colab_type":"code","id":"WwBWy7J2kZvA","colab":{}},"source":["# Cell 6.16\n","model.summary()\n","print ('Cell finished.')"],"execution_count":0,"outputs":[]},{"cell_type":"code","metadata":{"colab_type":"code","id":"bNXelbMQtonr","colab":{}},"source":["# Cell 6.17\n","len(model.trainable_variables)\n","print ('Cell finished.')"],"execution_count":0,"outputs":[]},{"cell_type":"markdown","metadata":{"colab_type":"text","id":"4G5O4jd6TuAG"},"source":["### Continue Train the model"]},{"cell_type":"markdown","metadata":{"colab_type":"text","id":"0foWUN-yDLo_"},"source":["If you trained to convergence earlier, this will get you a few percent more accuracy."]},{"cell_type":"code","metadata":{"colab_type":"code","id":"ECQLkAsFTlun","colab":{}},"source":["# Cell 6.18\n","history_fine = model.fit_generator(train_generator, \n"," steps_per_epoch = steps_per_epoch,\n"," epochs=epochs, \n"," workers=4,\n"," validation_data=validation_generator, \n"," validation_steps=validation_steps)\n","print ('Cell finished.')"],"execution_count":0,"outputs":[]},{"cell_type":"markdown","metadata":{"colab_type":"text","id":"TfXEmsxQf6eP"},"source":["### Learning curves\n","\n","Let's take a look at the learning curves of the training and validation accuracy / loss, when fine tuning the last few layers of the MobileNet V2 base model, as well as the classifier on top of it. Note the validation loss much higher than the training loss which means there maybe some overfitting. \n","\n","**Note**: the training dataset is fairly small, and is similar to the original datasets that MobileNet V2 was trained on, so fine-tuning may result in overfitting.\n"]},{"cell_type":"markdown","metadata":{"colab_type":"text","id":"DNtfNZKlInGT"},"source":["If you train to convergence (`epochs=50`) the resulting graph should look like this:\n","\n","![After fine tuning the model nearly reaches 98% accuracy](https://github.com/tensorflow/docs/blob/master/site/en/tutorials/images/images/fine_tuning.png?raw=1)"]},{"cell_type":"code","metadata":{"colab_type":"code","id":"PpA8PlpQKygw","colab":{}},"source":["# Cell 6.19\n","acc += history_fine.history['accuracy']\n","val_acc += history_fine.history['val_accuracy']\n","\n","loss += history_fine.history['loss']\n","val_loss += history_fine.history['val_loss']\n","print ('Cell finished.')"],"execution_count":0,"outputs":[]},{"cell_type":"code","metadata":{"colab_type":"code","id":"chW103JUItdk","colab":{}},"source":["# Cell 6.20\n","plt.figure(figsize=(8, 8))\n","plt.subplot(2, 1, 1)\n","plt.plot(acc, label='Training Accuracy')\n","plt.plot(val_acc, label='Validation Accuracy')\n","plt.ylim([0.9, 1])\n","plt.plot([epochs-1,epochs-1], plt.ylim(), label='Start Fine Tuning')\n","plt.legend(loc='lower right')\n","plt.title('Training and Validation Accuracy')\n","\n","plt.subplot(2, 1, 2)\n","plt.plot(loss, label='Training Loss')\n","plt.plot(val_loss, label='Validation Loss')\n","plt.ylim([0, 0.2])\n","plt.plot([epochs-1,epochs-1], plt.ylim(), label='Start Fine Tuning')\n","plt.legend(loc='upper right')\n","plt.title('Training and Validation Loss')\n","plt.show()\n","print ('Cell finished.')"],"execution_count":0,"outputs":[]},{"cell_type":"markdown","metadata":{"colab_type":"text","id":"_TZTwG7nhm0C"},"source":["# Key takeaways\n","In summary here is what we covered in this tutorial on how to do transfer learning using a pre-trained model to improve accuracy:\n","* Using a pre-trained model for **feature extraction** - when working with a small dataset, it is common to leverage the features learned by a model trained on a larger dataset in the same domain. This is done by instantiating the pre-trained model and adding a fully connected classifier on top. The pre-trained model is \"frozen\" and only the weights of the classifier are updated during training.\n","In this case, the convolutional base extracts all the features associated with each image and we train a classifier that determines, given these set of features to which class it belongs.\n","* **Fine-tuning** a pre-trained model - to further improve performance, one might want to repurpose the top-level layers of the pre-trained models to the new dataset via fine-tuning.\n","In this case, we tune our weights such that we learn highly specified and high level features specific to our dataset. This only make sense when the training dataset is large and very similar to the orginial dataset that the pre-trained model was trained on.\n"]}]} 2 | -------------------------------------------------------------------------------- /startup/startup.sh: -------------------------------------------------------------------------------- 1 | iptables -t nat -I PREROUTING -p tcp --dport 80 -j REDIRECT --to-ports 8080 2 | iptables -t nat -I OUTPUT -p tcp -o lo --dport 80 -j REDIRECT --to-ports 8080 3 | cd ~jupyter 4 | git clone https://github.com/embedded-vision/dlcvtf2 5 | chown -R jupyter dlcvtf2 6 | --------------------------------------------------------------------------------