![\"Open](\"https://colab.research.google.com/assets/colab-badge.svg\"){kind=icon}
"
10 | ]
11 | },
12 | {
13 | "cell_type": "markdown",
14 | "metadata": {
15 | "id": "qnyTxjK_GbOD"
16 | },
17 | "source": [
18 | "# Ungraded Lab: Beyond Hello World, A Computer Vision Example\n",
19 | "In the previous exercise, you saw how to create a neural network that figured out the problem you were trying to solve. This gave an explicit example of learned behavior. Of course, in that instance, it was a bit of overkill because it would have been easier to write the function `y=2x-1` directly instead of bothering with using machine learning to learn the relationship between `x` and `y`.\n",
20 | "\n",
21 | "But what about a scenario where writing rules like that is much more difficult -- for example a computer vision problem? Let's take a look at a scenario where you will build a neural network to recognize different items of clothing, trained from a dataset containing 10 different types."
22 | ]
23 | },
24 | {
25 | "cell_type": "markdown",
26 | "metadata": {
27 | "id": "H41FYgtlHPjW"
28 | },
29 | "source": [
30 | "## Start Coding\n",
31 | "\n",
32 | "Let's start with our import of TensorFlow."
33 | ]
34 | },
35 | {
36 | "cell_type": "code",
37 | "execution_count": null,
38 | "metadata": {
39 | "id": "q3KzJyjv3rnA"
40 | },
41 | "outputs": [],
42 | "source": [
43 | "import tensorflow as tf\n",
44 | "\n",
45 | "print(tf.__version__)"
46 | ]
47 | },
48 | {
49 | "cell_type": "markdown",
50 | "metadata": {
51 | "id": "n_n1U5do3u_F"
52 | },
53 | "source": [
54 | "The [Fashion MNIST dataset](https://github.com/zalandoresearch/fashion-mnist) is a collection of grayscale 28x28 pixel clothing images. Each image is associated with a label as shown in this table⁉\n",
55 | "\n",
56 | "| Label | Description |\n",
57 | "| --- | --- |\n",
58 | "| 0 | T-shirt/top |\n",
59 | "| 1 | Trouser |\n",
60 | "| 2 | Pullover |\n",
61 | "| 3 | Dress |\n",
62 | "| 4 | Coat |\n",
63 | "| 5 | Sandal |\n",
64 | "| 6 | Shirt |\n",
65 | "| 7 | Sneaker |\n",
66 | "| 8 | Bag |\n",
67 | "| 9 | Ankle boot |\n",
68 | "\n",
69 | "This dataset is available directly in the [tf.keras.datasets](https://www.tensorflow.org/api_docs/python/tf/keras/datasets) API and you load it like this:"
70 | ]
71 | },
72 | {
73 | "cell_type": "code",
74 | "execution_count": null,
75 | "metadata": {
76 | "id": "PmxkHFpt31bM"
77 | },
78 | "outputs": [],
79 | "source": [
80 | "# Load the Fashion MNIST dataset\n",
81 | "fmnist = tf.keras.datasets.fashion_mnist"
82 | ]
83 | },
84 | {
85 | "cell_type": "markdown",
86 | "metadata": {
87 | "id": "GuoLQQBT4E-_"
88 | },
89 | "source": [
90 | "Calling `load_data()` on this object will give you two tuples with two lists each. These will be the training and testing values for the graphics that contain the clothing items and their labels.\n"
91 | ]
92 | },
93 | {
94 | "cell_type": "code",
95 | "execution_count": null,
96 | "metadata": {
97 | "id": "BTdRgExe4TRB"
98 | },
99 | "outputs": [],
100 | "source": [
101 | "# Load the training and test split of the Fashion MNIST dataset\n",
102 | "(training_images, training_labels), (test_images, test_labels) = fmnist.load_data()"
103 | ]
104 | },
105 | {
106 | "cell_type": "markdown",
107 | "metadata": {
108 | "id": "rw395ROx4f5Q"
109 | },
110 | "source": [
111 | "What does these values look like? Let's print a training image (both as an image and a numpy array), and a training label to see. Experiment with different indices in the array. For example, also take a look at index `42`. That's a different boot than the one at index `0`.\n"
112 | ]
113 | },
114 | {
115 | "cell_type": "code",
116 | "execution_count": null,
117 | "metadata": {
118 | "id": "FPc9d3gJ3jWF"
119 | },
120 | "outputs": [],
121 | "source": [
122 | "import numpy as np\n",
123 | "import matplotlib.pyplot as plt\n",
124 | "\n",
125 | "# You can put between 0 to 59999 here\n",
126 | "index = 0\n",
127 | "\n",
128 | "# Set number of characters per row when printing\n",
129 | "np.set_printoptions(linewidth=320)\n",
130 | "\n",
131 | "# Print the label and image\n",
132 | "print(f'LABEL: {training_labels[index]}')\n",
133 | "print(f'\\nIMAGE PIXEL ARRAY:\\n {training_images[index]}')\n",
134 | "\n",
135 | "# Visualize the image\n",
136 | "plt.imshow(training_images[index])"
137 | ]
138 | },
139 | {
140 | "cell_type": "markdown",
141 | "metadata": {
142 | "id": "3cbrdH225_nH"
143 | },
144 | "source": [
145 | "You'll notice that all of the values in the number are between 0 and 255. If you are training a neural network especially in image processing, for various reasons it will usually learn better if you scale all values to between 0 and 1. It's a process called _normalization_ and fortunately in Python, it's easy to normalize an array without looping. You do it like this:"
146 | ]
147 | },
148 | {
149 | "cell_type": "code",
150 | "execution_count": null,
151 | "metadata": {
152 | "id": "kRH19pWs6ZDn"
153 | },
154 | "outputs": [],
155 | "source": [
156 | "# Normalize the pixel values of the train and test images\n",
157 | "training_images = training_images / 255.0\n",
158 | "test_images = test_images / 255.0"
159 | ]
160 | },
161 | {
162 | "cell_type": "markdown",
163 | "metadata": {
164 | "id": "3DkO0As46lRn"
165 | },
166 | "source": [
167 | "Now you might be wondering why the dataset is split into two: training and testing? Remember we spoke about this in the intro? The idea is to have 1 set of data for training, and then another set of data that the model hasn't yet seen. This will be used to evaluate how good it would be at classifying values."
168 | ]
169 | },
170 | {
171 | "cell_type": "markdown",
172 | "metadata": {
173 | "id": "dIn7S9gf62ie"
174 | },
175 | "source": [
176 | "Let's now design the model. There's quite a few new concepts here. But don't worry, you'll get the hang of them. "
177 | ]
178 | },
179 | {
180 | "cell_type": "code",
181 | "execution_count": null,
182 | "metadata": {
183 | "id": "7mAyndG3kVlK"
184 | },
185 | "outputs": [],
186 | "source": [
187 | "# Build the classification model\n",
188 | "model = tf.keras.models.Sequential([tf.keras.layers.Flatten(), \n",
189 | " tf.keras.layers.Dense(128, activation=tf.nn.relu), \n",
190 | " tf.keras.layers.Dense(10, activation=tf.nn.softmax)])"
191 | ]
192 | },
193 | {
194 | "cell_type": "markdown",
195 | "metadata": {
196 | "id": "-lUcWaiX7MFj"
197 | },
198 | "source": [
199 | "[Sequential](https://keras.io/api/models/sequential/): That defines a sequence of layers in the neural network.\n",
200 | "\n",
201 | "[Flatten](https://keras.io/api/layers/reshaping_layers/flatten/): Remember earlier where our images were a 28x28 pixel matrix when you printed them out? Flatten just takes that square and turns it into a 1-dimensional array.\n",
202 | "\n",
203 | "[Dense](https://keras.io/api/layers/core_layers/dense/): Adds a layer of neurons\n",
204 | "\n",
205 | "Each layer of neurons need an [activation function](https://keras.io/api/layers/activations/) to tell them what to do. There are a lot of options, but just use these for now: \n",
206 | "\n",
207 | "[ReLU](https://keras.io/api/layers/activations/#relu-function) effectively means:\n",
208 | "\n",
209 | "```\n",
210 | "if x > 0: \n",
211 | " return x\n",
212 | "\n",
213 | "else: \n",
214 | " return 0\n",
215 | "```\n",
216 | "\n",
217 | "In other words, it only passes values 0 or greater to the next layer in the network.\n",
218 | "\n",
219 | "[Softmax](https://keras.io/api/layers/activations/#softmax-function) takes a list of values and scales these so the sum of all elements will be equal to 1. When applied to model outputs, you can think of the scaled values as the probability for that class. For example, in your classification model which has 10 units in the output dense layer, having the highest value at `index = 4` means that the model is most confident that the input clothing image is a coat. If it is at index = 5, then it is a sandal, and so forth. See the short code block below which demonstrates these concepts. You can also watch this [lecture](https://www.youtube.com/watch?v=LLux1SW--oM&ab_channel=DeepLearningAI) if you want to know more about the Softmax function and how the values are computed.\n"
220 | ]
221 | },
222 | {
223 | "cell_type": "code",
224 | "execution_count": null,
225 | "metadata": {
226 | "id": "Dk1hzzpDoGPI"
227 | },
228 | "outputs": [],
229 | "source": [
230 | "# Declare sample inputs and convert to a tensor\n",
231 | "inputs = np.array([[1.0, 3.0, 4.0, 2.0]])\n",
232 | "inputs = tf.convert_to_tensor(inputs)\n",
233 | "print(f'input to softmax function: {inputs.numpy()}')\n",
234 | "\n",
235 | "# Feed the inputs to a softmax activation function\n",
236 | "outputs = tf.keras.activations.softmax(inputs)\n",
237 | "print(f'output of softmax function: {outputs.numpy()}')\n",
238 | "\n",
239 | "# Get the sum of all values after the softmax\n",
240 | "sum = tf.reduce_sum(outputs)\n",
241 | "print(f'sum of outputs: {sum}')\n",
242 | "\n",
243 | "# Get the index with highest value\n",
244 | "prediction = np.argmax(outputs)\n",
245 | "print(f'class with highest probability: {prediction}')"
246 | ]
247 | },
248 | {
249 | "cell_type": "markdown",
250 | "metadata": {
251 | "id": "c8vbMCqb9Mh6"
252 | },
253 | "source": [
254 | "The next thing to do, now that the model is defined, is to actually build it. You do this by compiling it with an optimizer and loss function as before -- and then you train it by calling `model.fit()` asking it to fit your training data to your training labels. It will figure out the relationship between the training data and its actual labels so in the future if you have inputs that looks like the training data, then it can predict what the label for that input is."
255 | ]
256 | },
257 | {
258 | "cell_type": "code",
259 | "execution_count": null,
260 | "metadata": {
261 | "id": "BLMdl9aP8nQ0"
262 | },
263 | "outputs": [],
264 | "source": [
265 | "model.compile(optimizer = tf.optimizers.Adam(),\n",
266 | " loss = 'sparse_categorical_crossentropy',\n",
267 | " metrics=['accuracy'])\n",
268 | "\n",
269 | "model.fit(training_images, training_labels, epochs=5)"
270 | ]
271 | },
272 | {
273 | "cell_type": "markdown",
274 | "metadata": {
275 | "id": "-JJMsvSB-1UY"
276 | },
277 | "source": [
278 | "Once it's done training -- you should see an accuracy value at the end of the final epoch. It might look something like `0.9098`. This tells you that your neural network is about 91% accurate in classifying the training data. That is, it figured out a pattern match between the image and the labels that worked 91% of the time. Not great, but not bad considering it was only trained for 5 epochs and done quite quickly.\n",
279 | "\n",
280 | "But how would it work with unseen data? That's why we have the test images and labels. We can call [`model.evaluate()`](https://keras.io/api/models/model_training_apis/#evaluate-method) with this test dataset as inputs and it will report back the loss and accuracy of the model. Let's give it a try:"
281 | ]
282 | },
283 | {
284 | "cell_type": "code",
285 | "execution_count": null,
286 | "metadata": {
287 | "id": "WzlqsEzX9s5P"
288 | },
289 | "outputs": [],
290 | "source": [
291 | "# Evaluate the model on unseen data\n",
292 | "model.evaluate(test_images, test_labels)"
293 | ]
294 | },
295 | {
296 | "cell_type": "markdown",
297 | "metadata": {
298 | "id": "6tki-Aro_Uax"
299 | },
300 | "source": [
301 | "You can expect the accuracy here to be about `0.88` which means it was 88% accurate on the entire test set. As expected, it probably would not do as well with *unseen* data as it did with data it was trained on! As you go through this course, you'll look at ways to improve this. "
302 | ]
303 | },
304 | {
305 | "cell_type": "markdown",
306 | "metadata": {
307 | "id": "htldZNWcIPSN"
308 | },
309 | "source": [
310 | "# Exploration Exercises\n",
311 | "\n",
312 | "To explore further and deepen your understanding, try the below exercises:"
313 | ]
314 | },
315 | {
316 | "cell_type": "markdown",
317 | "metadata": {
318 | "id": "rquQqIx4AaGR"
319 | },
320 | "source": [
321 | "### Exercise 1:\n",
322 | "For this first exercise run the below code: It creates a set of classifications for each of the test images, and then prints the first entry in the classifications. The output, after you run it is a list of numbers. Why do you think this is, and what do those numbers represent? "
323 | ]
324 | },
325 | {
326 | "cell_type": "code",
327 | "execution_count": null,
328 | "metadata": {
329 | "id": "RyEIki0z_hAD"
330 | },
331 | "outputs": [],
332 | "source": [
333 | "classifications = model.predict(test_images)\n",
334 | "\n",
335 | "print(classifications[0])"
336 | ]
337 | },
338 | {
339 | "cell_type": "markdown",
340 | "metadata": {
341 | "id": "MdzqbQhRArzm"
342 | },
343 | "source": [
344 | "**Hint:** try running `print(test_labels[0])` -- and you'll get a `9`. Does that help you understand why this list looks the way it does? "
345 | ]
346 | },
347 | {
348 | "cell_type": "code",
349 | "execution_count": null,
350 | "metadata": {
351 | "id": "WnBGOrMiA1n5"
352 | },
353 | "outputs": [],
354 | "source": [
355 | "print(test_labels[0])"
356 | ]
357 | },
358 | {
359 | "cell_type": "markdown",
360 | "metadata": {
361 | "id": "uUs7eqr7uSvs"
362 | },
363 | "source": [
364 | "### E1Q1: What does this list represent?\n",
365 | "\n",
366 | "\n",
367 | "1. It's 10 random meaningless values\n",
368 | "2. It's the first 10 classifications that the computer made\n",
369 | "3. It's the probability that this item is each of the 10 classes\n"
370 | ]
371 | },
372 | {
373 | "cell_type": "markdown",
374 | "metadata": {
375 | "id": "wAbr92RTA67u"
376 | },
377 | "source": [
378 | "\n", 380 | "\n", 381 | "#### Answer: \n", 382 | "The correct answer is (3)\n", 383 | "\n", 384 | "The output of the model is a list of 10 numbers. These numbers are a probability that the value being classified is the corresponding value (https://github.com/zalandoresearch/fashion-mnist#labels), i.e. the first value in the list is the probability that the image is of a '0' (T-shirt/top), the next is a '1' (Trouser) etc. Notice that they are all VERY LOW probabilities.\n", 385 | "\n", 386 | "For index 9 (Ankle boot), the probability was in the 90's, i.e. the neural network is telling us that the image is most likely an ankle boot.\n", 387 | "\n", 388 | "
\n", 389 | "\n", 414 | "\n", 415 | "#### Answer\n", 416 | "The correct answer is (2). Both the list and the labels are 0 based, so the ankle boot having label 9 means that it is the 10th of the 10 classes. The list having the 10th element being the highest value means that the Neural Network has predicted that the item it is classifying is most likely an ankle boot\n", 417 | "\n", 418 | "
\n", 419 | "\n", 485 | "\n", 486 | "#### Answer\n", 487 | "The correct answer is (1) by adding more Neurons we have to do more calculations, slowing down the process, but in this case they have a good impact -- we do get more accurate. That doesn't mean it's always a case of 'more is better', you can hit the law of diminishing returns very quickly!\n", 488 | "\n", 489 | "
\n", 490 | "\n", 505 | "\n", 506 | "#### Answer\n", 507 | "You get an error about the shape of the data. It may seem vague right now, but it reinforces the rule of thumb that the first layer in your network should be the same shape as your data. Right now our data is 28x28 images, and 28 layers of 28 neurons would be infeasible, so it makes more sense to 'flatten' that 28,28 into a 784x1. Instead of writng all the code to handle that ourselves, we add the Flatten() layer at the begining, and when the arrays are loaded into the model later, they'll automatically be flattened for us.\n", 508 | "\n", 509 | "
\n", 510 | "\n", 557 | "\n", 558 | "#### Answer\n", 559 | "You get an error as soon as it finds an unexpected value. Another rule of thumb -- the number of neurons in the last layer should match the number of classes you are classifying for. In this case it's the digits 0-9, so there are 10 of them, hence you should have 10 neurons in your final layer.\n", 560 | "\n", 561 | "
\n", 562 | "\n", 610 | "\n", 611 | "#### Answer \n", 612 | "There isn't a significant impact -- because this is relatively simple data. For far more complex data (including color images to be classified as flowers that you'll see in the next lesson), extra layers are often necessary. \n", 613 | "\n", 614 | "
\n", 615 | "