├── .idea └── simpleTensorFlowClassificationExample.iml ├── readme.md ├── data.csv ├── Basic Classification Example with TensorFlow.ipynb └── Exemple de base de classification avec TensorFlow.ipynb /.idea/simpleTensorFlowClassificationExample.iml: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | -------------------------------------------------------------------------------- /readme.md: -------------------------------------------------------------------------------- 1 | Companion notebook and dataset to [A Visual and Interactive Guide to the Basics of Neural Networks](https://jalammar.github.io/visual-interactive-guide-basics-neural-networks/) 2 | 3 | Dataset source: Coursera's [Machine Learning](https://www.coursera.org/learn/machine-learning) course. 4 | 5 | --- 6 | 7 | Carnet de notes et ensemble de données compagnon de [Un guide visuel et interactif des bases des réseaux neuronaux](https://rr0.org/people/a/AlammarJay/visual-interactive-guide-basics-neural-networks/index_fr.html) 8 | 9 | Source de l'ensemble de données : Cours de [Machine Learning](https://www.coursera.org/learn/machine-learning) de Coursera. 10 | -------------------------------------------------------------------------------- /data.csv: -------------------------------------------------------------------------------- 1 | index,area,bathrooms,price,sq_price 2 | 0,2104.0,3.0,399900.0,190.066539924 3 | 1,1600.0,3.0,329900.0,206.1875 4 | 2,2400.0,3.0,369000.0,153.75 5 | 3,1416.0,2.0,232000.0,163.84180791 6 | 4,3000.0,4.0,539900.0,179.966666667 7 | 5,1985.0,4.0,299900.0,151.083123426 8 | 6,1534.0,3.0,314900.0,205.280312907 9 | 7,1427.0,3.0,198999.0,139.452697968 10 | 8,1380.0,3.0,212000.0,153.623188406 11 | 9,1494.0,3.0,242500.0,162.315930388 12 | 10,1940.0,4.0,239999.0,123.710824742 13 | 11,2000.0,3.0,347000.0,173.5 14 | 12,1890.0,3.0,329999.0,174.602645503 15 | 13,4478.0,5.0,699900.0,156.297454221 16 | 14,1268.0,3.0,259900.0,204.968454259 17 | 15,2300.0,4.0,449900.0,195.608695652 18 | 16,1320.0,2.0,299900.0,227.196969697 19 | 17,1236.0,3.0,199900.0,161.731391586 20 | 18,2609.0,4.0,499998.0,191.643541587 21 | 19,3031.0,4.0,599000.0,197.624546354 22 | 20,1767.0,3.0,252900.0,143.123938879 23 | 21,1888.0,2.0,255000.0,135.063559322 24 | 22,1604.0,3.0,242900.0,151.433915212 25 | 23,1962.0,4.0,259900.0,132.46687054 26 | 24,3890.0,3.0,573900.0,147.532133676 27 | 25,1100.0,3.0,249900.0,227.181818182 28 | 26,1458.0,3.0,464500.0,318.587105624 29 | 27,2526.0,3.0,469000.0,185.669041964 30 | 28,2200.0,3.0,475000.0,215.909090909 31 | 29,2637.0,3.0,299900.0,113.727720895 32 | 30,1839.0,2.0,349900.0,190.266449157 33 | 31,1000.0,1.0,169900.0,169.9 34 | 32,2040.0,4.0,314900.0,154.362745098 35 | 33,3137.0,3.0,579900.0,184.858144724 36 | 34,1811.0,4.0,285900.0,157.868580895 37 | 35,1437.0,3.0,249900.0,173.903966597 38 | 36,1239.0,3.0,229900.0,185.552865214 39 | 37,2132.0,4.0,345000.0,161.81988743 40 | 38,4215.0,4.0,549000.0,130.24911032 41 | 39,2162.0,4.0,287000.0,132.747456059 42 | 40,1664.0,2.0,368500.0,221.454326923 43 | 41,2238.0,3.0,329900.0,147.408400357 44 | 42,2567.0,4.0,314000.0,122.321776393 45 | 43,1200.0,3.0,299000.0,249.166666667 46 | 44,852.0,2.0,179900.0,211.150234742 47 | 45,1852.0,4.0,299900.0,161.933045356 48 | 46,1203.0,3.0,239500.0,199.085619285 49 | -------------------------------------------------------------------------------- /Basic Classification Example with TensorFlow.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "# Basic Classification Example with TensorFlow\n", 8 | "\n", 9 | "This notebook is a companion of [A Visual and Interactive Guide to the Basics of Neural Networks](https://jalammar.github.io/visual-interactive-guide-basics-neural-networks/).\n", 10 | "\n", 11 | "This is an example of how to do classification on a simple dataset in TensorFlow. Basically, we're building a model to help a friend choose a house to buy. She has given us the table below of houses and whether she likes them or not. We're to build a model that takes a house area and number of bathrooms as input, and outputs a prediction of whether she would like the house or not.\n", 12 | "\n", 13 | "| Area (sq ft) (x1) | Bathrooms (x2) | Label (y) |\n", 14 | " | --- | --- | --- |\n", 15 | " | 2,104 | 3 | Good |\n", 16 | " | 1,600 | 3 | Good |\n", 17 | " | 2,400 | 3 | Good |\n", 18 | " | 1,416 | \t2 | Bad |\n", 19 | " | 3,000 | \t4 | Bad |\n", 20 | " | 1,985 | \t4 | Good |\n", 21 | " | 1,534 | \t3 | Bad |\n", 22 | " | 1,427 | \t3 | Good |\n", 23 | " | 1,380 | \t3 | Good |\n", 24 | " | 1,494 | \t3 | Good |\n", 25 | " \n", 26 | " \n", 27 | " \n", 28 | " We'll start by loading our favorite libraries" 29 | ] 30 | }, 31 | { 32 | "cell_type": "code", 33 | "execution_count": 1, 34 | "metadata": { 35 | "collapsed": false 36 | }, 37 | "outputs": [], 38 | "source": [ 39 | "import pandas as pd # A beautiful library to help us work with data as tables\n", 40 | "import tensorflow as tf # Fire from the gods\n" 41 | ] 42 | }, 43 | { 44 | "cell_type": "markdown", 45 | "metadata": {}, 46 | "source": [ 47 | "We'll then load the house data CSV.\n", 48 | "Pandas is an incredible library that gives us great flexibility in dealing with table-like data.\n", 49 | "We load tables (or csv files, or excel sheets) into a \"data frame\", and process it however we like.\n", 50 | "You can think of it as a programmatic way to do a lot of the things you previously did with Excel." 51 | ] 52 | }, 53 | { 54 | "cell_type": "code", 55 | "execution_count": 2, 56 | "metadata": { 57 | "collapsed": false, 58 | "scrolled": true 59 | }, 60 | "outputs": [ 61 | { 62 | "data": { 63 | "text/plain": " area bathrooms\n0 2104.0 3.0\n1 1600.0 3.0\n2 2400.0 3.0\n3 1416.0 2.0\n4 3000.0 4.0\n5 1985.0 4.0\n6 1534.0 3.0\n7 1427.0 3.0\n8 1380.0 3.0\n9 1494.0 3.0", 64 | "text/html": "
\n\n\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
areabathrooms
02104.03.0
11600.03.0
22400.03.0
31416.02.0
43000.04.0
51985.04.0
61534.03.0
71427.03.0
81380.03.0
91494.03.0
\n
" 65 | }, 66 | "execution_count": 2, 67 | "metadata": {}, 68 | "output_type": "execute_result" 69 | } 70 | ], 71 | "source": [ 72 | "dataframe = pd.read_csv(\"data.csv\") # Let's have Pandas load our dataset as a dataframe\n", 73 | "dataframe = dataframe.drop([\"index\", \"price\", \"sq_price\"], axis=1) # Remove columns we don't care about\n", 74 | "dataframe = dataframe[0:10] # We'll only use the first 10 rows of the dataset in this example\n", 75 | "dataframe # Let's have the notebook show us how the dataframe looks now" 76 | ] 77 | }, 78 | { 79 | "cell_type": "markdown", 80 | "metadata": { 81 | "collapsed": false 82 | }, 83 | "source": [ 84 | "The dataframe now only has the features. Let's introduce the labels." 85 | ] 86 | }, 87 | { 88 | "cell_type": "code", 89 | "execution_count": 3, 90 | "metadata": { 91 | "collapsed": false 92 | }, 93 | "outputs": [ 94 | { 95 | "data": { 96 | "text/plain": " area bathrooms y1 y2\n0 2104.0 3.0 1 0\n1 1600.0 3.0 1 0\n2 2400.0 3.0 1 0\n3 1416.0 2.0 0 1\n4 3000.0 4.0 0 1\n5 1985.0 4.0 1 0\n6 1534.0 3.0 0 1\n7 1427.0 3.0 1 0\n8 1380.0 3.0 1 0\n9 1494.0 3.0 1 0", 97 | "text/html": "
\n\n\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
areabathroomsy1y2
02104.03.010
11600.03.010
22400.03.010
31416.02.001
43000.04.001
51985.04.010
61534.03.001
71427.03.010
81380.03.010
91494.03.010
\n
" 98 | }, 99 | "execution_count": 3, 100 | "metadata": {}, 101 | "output_type": "execute_result" 102 | } 103 | ], 104 | "source": [ 105 | "dataframe.loc[:, (\"y1\")] = [1, 1, 1, 0, 0, 1, 0, 1, 1, 1] # This is our friend's list of which houses she liked\n", 106 | " # 1 = good, 0 = bad\n", 107 | "dataframe.loc[:, (\"y2\")] = dataframe[\"y1\"] == 0 # y2 is the negation of y1\n", 108 | "dataframe.loc[:, (\"y2\")] = dataframe[\"y2\"].astype(int) # Turn TRUE/FALSE values into 1/0\n", 109 | "# y2 means we don't like a house\n", 110 | "# (Yes, it's redundant. But learning to do it this way opens the door to Multiclass classification)\n", 111 | "dataframe # How is our dataframe looking now?" 112 | ] 113 | }, 114 | { 115 | "cell_type": "markdown", 116 | "metadata": { 117 | "collapsed": false 118 | }, 119 | "source": [ 120 | "Now that we have all our data in the dataframe, we'll need to shape it in matrices to feed it to TensorFlow" 121 | ] 122 | }, 123 | { 124 | "cell_type": "code", 125 | "execution_count": 4, 126 | "metadata": { 127 | "collapsed": false, 128 | "scrolled": false 129 | }, 130 | "outputs": [], 131 | "source": [ 132 | "inputX = dataframe.loc[:, ['area', 'bathrooms']].values\n", 133 | "inputY = dataframe.loc[:, [\"y1\", \"y2\"]].values" 134 | ] 135 | }, 136 | { 137 | "cell_type": "markdown", 138 | "metadata": {}, 139 | "source": [ 140 | "So now our input matrix looks like this:" 141 | ] 142 | }, 143 | { 144 | "cell_type": "code", 145 | "execution_count": 5, 146 | "metadata": { 147 | "collapsed": false 148 | }, 149 | "outputs": [ 150 | { 151 | "data": { 152 | "text/plain": "array([[2.104e+03, 3.000e+00],\n [1.600e+03, 3.000e+00],\n [2.400e+03, 3.000e+00],\n [1.416e+03, 2.000e+00],\n [3.000e+03, 4.000e+00],\n [1.985e+03, 4.000e+00],\n [1.534e+03, 3.000e+00],\n [1.427e+03, 3.000e+00],\n [1.380e+03, 3.000e+00],\n [1.494e+03, 3.000e+00]])" 153 | }, 154 | "execution_count": 5, 155 | "metadata": {}, 156 | "output_type": "execute_result" 157 | } 158 | ], 159 | "source": [ 160 | "inputX" 161 | ] 162 | }, 163 | { 164 | "cell_type": "markdown", 165 | "metadata": {}, 166 | "source": [ 167 | "And our labels matrix looks like this:" 168 | ] 169 | }, 170 | { 171 | "cell_type": "code", 172 | "execution_count": 6, 173 | "metadata": { 174 | "collapsed": false 175 | }, 176 | "outputs": [ 177 | { 178 | "data": { 179 | "text/plain": "array([[1, 0],\n [1, 0],\n [1, 0],\n [0, 1],\n [0, 1],\n [1, 0],\n [0, 1],\n [1, 0],\n [1, 0],\n [1, 0]])" 180 | }, 181 | "execution_count": 6, 182 | "metadata": {}, 183 | "output_type": "execute_result" 184 | } 185 | ], 186 | "source": [ 187 | "inputY" 188 | ] 189 | }, 190 | { 191 | "cell_type": "markdown", 192 | "metadata": {}, 193 | "source": [ 194 | "Let's prepare some parameters for the training process" 195 | ] 196 | }, 197 | { 198 | "cell_type": "code", 199 | "execution_count": 7, 200 | "metadata": { 201 | "collapsed": false 202 | }, 203 | "outputs": [], 204 | "source": [ 205 | "# Parameters\n", 206 | "learning_rate = 0.000001\n", 207 | "training_epochs = 2000\n", 208 | "display_step = 50\n", 209 | "n_samples = inputY.size" 210 | ] 211 | }, 212 | { 213 | "cell_type": "markdown", 214 | "metadata": {}, 215 | "source": [ 216 | "And now to define the TensorFlow operations. Notice that this is a declaration step where we tell TensorFlow how the prediction is calculated. If we execute it, no calculation would be made. It would just acknowledge that it now knows how to do the operation." 217 | ] 218 | }, 219 | { 220 | "cell_type": "code", 221 | "execution_count": 8, 222 | "metadata": { 223 | "collapsed": true 224 | }, 225 | "outputs": [], 226 | "source": [ 227 | "x = tf.placeholder(tf.float32, [None, 2]) # Okay TensorFlow, we'll feed you an array of examples. Each example will\n", 228 | " # be an array of two float values (area, and number of bathrooms).\n", 229 | " # \"None\" means we can feed you any number of examples\n", 230 | " # Notice we haven't fed it the values yet\n", 231 | " \n", 232 | "W = tf.Variable(tf.zeros([2, 2])) # Maintain a 2 x 2 float matrix for the weights that we'll keep updating \n", 233 | " # through the training process (make them all zero to begin with)\n", 234 | " \n", 235 | "b = tf.Variable(tf.zeros([2])) # Also maintain two bias values\n", 236 | "\n", 237 | "y_values = tf.add(tf.matmul(x, W), b) # The first step in calculating the prediction would be to multiply\n", 238 | " # the inputs matrix by the weights matrix then add the biases\n", 239 | " \n", 240 | "y = tf.nn.softmax(y_values) # Then we use softmax as an \"activation function\" that translates the\n", 241 | " # numbers outputted by the previous layer into probability form\n", 242 | " \n", 243 | "y_ = tf.placeholder(tf.float32, [None,2]) # For training purposes, we'll also feed you a matrix of labels" 244 | ] 245 | }, 246 | { 247 | "cell_type": "markdown", 248 | "metadata": {}, 249 | "source": [ 250 | "Let's specify our cost function and use Gradient Descent" 251 | ] 252 | }, 253 | { 254 | "cell_type": "code", 255 | "execution_count": 9, 256 | "metadata": { 257 | "collapsed": true 258 | }, 259 | "outputs": [ 260 | { 261 | "name": "stdout", 262 | "output_type": "stream", 263 | "text": [ 264 | "WARNING:tensorflow:From /Users/javarome/anaconda3/envs/zelros-cs-nlp/lib/python3.7/site-packages/tensorflow/python/ops/math_grad.py:1205: add_dispatch_support..wrapper (from tensorflow.python.ops.array_ops) is deprecated and will be removed in a future version.\n", 265 | "Instructions for updating:\n", 266 | "Use tf.where in 2.0, which has the same broadcast rule as np.where\n" 267 | ] 268 | } 269 | ], 270 | "source": [ 271 | "\n", 272 | "# Cost function: Mean squared error\n", 273 | "cost = tf.reduce_sum(tf.pow(y_ - y, 2))/(2*n_samples)\n", 274 | "# Gradient descent\n", 275 | "optimizer = tf.train.GradientDescentOptimizer(learning_rate).minimize(cost)\n" 276 | ] 277 | }, 278 | { 279 | "cell_type": "code", 280 | "execution_count": 10, 281 | "metadata": { 282 | "collapsed": true 283 | }, 284 | "outputs": [ 285 | { 286 | "name": "stdout", 287 | "output_type": "stream", 288 | "text": [ 289 | "WARNING:tensorflow:From /Users/javarome/anaconda3/envs/zelros-cs-nlp/lib/python3.7/site-packages/tensorflow/python/util/tf_should_use.py:193: initialize_all_variables (from tensorflow.python.ops.variables) is deprecated and will be removed after 2017-03-02.\n", 290 | "Instructions for updating:\n", 291 | "Use `tf.global_variables_initializer` instead.\n" 292 | ] 293 | } 294 | ], 295 | "source": [ 296 | "# Initialize variabls and tensorflow session\n", 297 | "init = tf.initialize_all_variables()\n", 298 | "sess = tf.Session()\n", 299 | "sess.run(init)" 300 | ] 301 | }, 302 | { 303 | "cell_type": "markdown", 304 | "metadata": {}, 305 | "source": [ 306 | "*Drum roll*\n", 307 | "\n", 308 | "And now for the actual training" 309 | ] 310 | }, 311 | { 312 | "cell_type": "code", 313 | "execution_count": 11, 314 | "metadata": { 315 | "collapsed": false, 316 | "scrolled": false 317 | }, 318 | "outputs": [ 319 | { 320 | "name": "stdout", 321 | "output_type": "stream", 322 | "text": [ 323 | "Training step: 0000 cost= 0.114958666\n", 324 | "Training step: 0050 cost= 0.109539941\n", 325 | "Training step: 0100 cost= 0.109539881\n", 326 | "Training step: 0150 cost= 0.109539807\n", 327 | "Training step: 0200 cost= 0.109539740\n", 328 | "Training step: 0250 cost= 0.109539680\n", 329 | "Training step: 0300 cost= 0.109539606\n", 330 | "Training step: 0350 cost= 0.109539531\n", 331 | "Training step: 0400 cost= 0.109539464\n", 332 | "Training step: 0450 cost= 0.109539390\n", 333 | "Training step: 0500 cost= 0.109539323\n", 334 | "Training step: 0550 cost= 0.109539248\n", 335 | "Training step: 0600 cost= 0.109539203\n", 336 | "Training step: 0650 cost= 0.109539129\n", 337 | "Training step: 0700 cost= 0.109539054\n", 338 | "Training step: 0750 cost= 0.109538987\n", 339 | "Training step: 0800 cost= 0.109538913\n", 340 | "Training step: 0850 cost= 0.109538853\n", 341 | "Training step: 0900 cost= 0.109538786\n", 342 | "Training step: 0950 cost= 0.109538712\n", 343 | "Training step: 1000 cost= 0.109538652\n", 344 | "Training step: 1050 cost= 0.109538577\n", 345 | "Training step: 1100 cost= 0.109538510\n", 346 | "Training step: 1150 cost= 0.109538436\n", 347 | "Training step: 1200 cost= 0.109538369\n", 348 | "Training step: 1250 cost= 0.109538309\n", 349 | "Training step: 1300 cost= 0.109538250\n", 350 | "Training step: 1350 cost= 0.109538175\n", 351 | "Training step: 1400 cost= 0.109538101\n", 352 | "Training step: 1450 cost= 0.109538034\n", 353 | "Training step: 1500 cost= 0.109537959\n", 354 | "Training step: 1550 cost= 0.109537899\n", 355 | "Training step: 1600 cost= 0.109537832\n", 356 | "Training step: 1650 cost= 0.109537773\n", 357 | "Training step: 1700 cost= 0.109537683\n", 358 | "Training step: 1750 cost= 0.109537624\n", 359 | "Training step: 1800 cost= 0.109537557\n", 360 | "Training step: 1850 cost= 0.109537482\n", 361 | "Training step: 1900 cost= 0.109537423\n", 362 | "Training step: 1950 cost= 0.109537356\n", 363 | "Optimization Finished!\n", 364 | "Training cost= 0.10953728 W= [[ 2.1414938e-04 -2.1415012e-04]\n", 365 | " [ 5.1274808e-05 -5.1274790e-05]] b= [ 1.19155184e-05 -1.19155275e-05] \n", 366 | "\n" 367 | ] 368 | } 369 | ], 370 | "source": [ 371 | "for i in range(training_epochs): \n", 372 | " sess.run(optimizer, feed_dict={x: inputX, y_: inputY}) # Take a gradient descent step using our inputs and labels\n", 373 | "\n", 374 | " # That's all! The rest of the cell just outputs debug messages. \n", 375 | " # Display logs per epoch step\n", 376 | " if (i) % display_step == 0:\n", 377 | " cc = sess.run(cost, feed_dict={x: inputX, y_:inputY})\n", 378 | " print(\"Training step:\", '%04d' % (i), \"cost=\", \"{:.9f}\".format(cc)) #, \\\"W=\", sess.run(W), \"b=\", sess.run(b)\n", 379 | "\n", 380 | "print(\"Optimization Finished!\")\n", 381 | "training_cost = sess.run(cost, feed_dict={x: inputX, y_: inputY})\n", 382 | "print(\"Training cost=\", training_cost, \"W=\", sess.run(W), \"b=\", sess.run(b), '\\n')\n" 383 | ] 384 | }, 385 | { 386 | "cell_type": "markdown", 387 | "metadata": { 388 | "collapsed": true 389 | }, 390 | "source": [ 391 | "Now the training is done. TensorFlow is now holding on to our trained model (Which is basically just the defined operations, plus the variables W and b that resulted from the training process).\n", 392 | "\n", 393 | "Is a cost value of 0.109537 good or bad? I have no idea. At least it's better than the first cost value of 0.114958666. Let's use the model on our dataset to see how it does, though:" 394 | ] 395 | }, 396 | { 397 | "cell_type": "code", 398 | "execution_count": 12, 399 | "metadata": { 400 | "collapsed": false 401 | }, 402 | "outputs": [ 403 | { 404 | "data": { 405 | "text/plain": "array([[0.7112522 , 0.28874776],\n [0.66498977, 0.33501023],\n [0.73657656, 0.26342347],\n [0.6471879 , 0.3528121 ],\n [0.78335613, 0.2166439 ],\n [0.7006948 , 0.29930523],\n [0.6586633 , 0.34133676],\n [0.6482863 , 0.35171375],\n [0.6436828 , 0.35631716],\n [0.65480113, 0.3451989 ]], dtype=float32)" 406 | }, 407 | "execution_count": 12, 408 | "metadata": {}, 409 | "output_type": "execute_result" 410 | } 411 | ], 412 | "source": [ 413 | "sess.run(y, feed_dict={x: inputX })" 414 | ] 415 | }, 416 | { 417 | "cell_type": "markdown", 418 | "metadata": {}, 419 | "source": [ 420 | "So It's guessing they're all good houses. That makes it get 7/10 correct. Not terribly impressive. A model with a hidden layer should do better, I guess." 421 | ] 422 | }, 423 | { 424 | "cell_type": "markdown", 425 | "metadata": { 426 | "collapsed": false 427 | }, 428 | "source": [ 429 | "Btw, this is how I calculated the softmax values in the post:" 430 | ] 431 | }, 432 | { 433 | "cell_type": "code", 434 | "execution_count": 13, 435 | "metadata": { 436 | "collapsed": false 437 | }, 438 | "outputs": [ 439 | { 440 | "data": { 441 | "text/plain": "array([0.26894143, 0.7310586 ], dtype=float32)" 442 | }, 443 | "execution_count": 13, 444 | "metadata": {}, 445 | "output_type": "execute_result" 446 | } 447 | ], 448 | "source": [ 449 | "sess.run(tf.nn.softmax([1., 2.]))" 450 | ] 451 | } 452 | ], 453 | "metadata": { 454 | "anaconda-cloud": {}, 455 | "kernelspec": { 456 | "name": "python3", 457 | "language": "python", 458 | "display_name": "Python 3" 459 | }, 460 | "language_info": { 461 | "codemirror_mode": { 462 | "name": "ipython", 463 | "version": 2 464 | }, 465 | "file_extension": ".py", 466 | "mimetype": "text/x-python", 467 | "name": "python", 468 | "nbconvert_exporter": "python", 469 | "pygments_lexer": "ipython2", 470 | "version": "2.7.12" 471 | } 472 | }, 473 | "nbformat": 4, 474 | "nbformat_minor": 0 475 | } -------------------------------------------------------------------------------- /Exemple de base de classification avec TensorFlow.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "# Exemple de base de classification avec TensorFlow\n", 8 | "\n", 9 | "Ce bloc-notes est un compagnon de [Un guide visuel et interactif des bases des réseaux neuronaux](https://rr0.org/people/a/AlammarJay/visual-interactive-guide-basics-neural-networks/index_fr.html).\n", 10 | "\n", 11 | "Voici un exemple de comment faire un classement sur un ensemble de données simple dans TensorFlow.\n", 12 | "Pour résumer, on construit un modèle pour aider une amie à choisir une maison à acheter.\n", 13 | "Elle nous a donné le tableau ci-dessous de maisons et si elle les aime ou non.\n", 14 | "Nous allons élaborer un modèle qui prend une surface de maison et un nombre de salles de bain en entrée,\n", 15 | "et sort une prédiction indiquant si elle aimera la maison ou non.\n", 16 | "\n", 17 | "| Surface (pieds²) (x1) | Salles de bain (x2) | Étiquette (y) |\n", 18 | " | --- | --- | --- |\n", 19 | " | 2104 | 3 | Bon |\n", 20 | " | 1600 | 3 | Bon |\n", 21 | " | 2400 | 3 | Bon |\n", 22 | " | 1416 | \t2 | Mauvais |\n", 23 | " | 3000 | \t4 | Mauvais |\n", 24 | " | 1985 | \t4 | Bon |\n", 25 | " | 1534 | \t3 | Mauvais |\n", 26 | " | 1427 | \t3 | Bon |\n", 27 | " | 1380 | \t3 | Bon |\n", 28 | " | 1494 | \t3 | Bon |\n", 29 | "\n", 30 | "Nous allons commencer en chargeant nos bibliothèques favorites" 31 | ] 32 | }, 33 | { 34 | "cell_type": "code", 35 | "execution_count": 27, 36 | "metadata": { 37 | "collapsed": false 38 | }, 39 | "outputs": [], 40 | "source": [ 41 | "import pandas as pd # Une belle bibliothèque pour nous aider à travailler les données sous forme tabulaire\n", 42 | "import tensorflow as tf # Le feu des dieux\n" 43 | ] 44 | }, 45 | { 46 | "cell_type": "markdown", 47 | "metadata": {}, 48 | "source": [ 49 | "Nous allons ensuite charger le CSV des données de maisons.\n", 50 | "Pandas est une bibliothèque incroyable offrant une grande souplesse dans le traitement des données tabulaires.\n", 51 | "On charge les tableaux (ou des fichiers CSV, ou des feuilles Excel) dans une \"data frame\", et on les traite comme on veut.\n", 52 | "Vous pouvez voir ça comme une manière programmatique de faire beaucoup des choses que vous avez faites avant avec Excel." 53 | ] 54 | }, 55 | { 56 | "cell_type": "code", 57 | "execution_count": 28, 58 | "metadata": { 59 | "collapsed": false, 60 | "scrolled": true 61 | }, 62 | "outputs": [ 63 | { 64 | "data": { 65 | "text/plain": " area bathrooms\n0 2104.0 3.0\n1 1600.0 3.0\n2 2400.0 3.0\n3 1416.0 2.0\n4 3000.0 4.0\n5 1985.0 4.0\n6 1534.0 3.0\n7 1427.0 3.0\n8 1380.0 3.0\n9 1494.0 3.0", 66 | "text/html": "
\n\n\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
areabathrooms
02104.03.0
11600.03.0
22400.03.0
31416.02.0
43000.04.0
51985.04.0
61534.03.0
71427.03.0
81380.03.0
91494.03.0
\n
" 67 | }, 68 | "execution_count": 28, 69 | "metadata": {}, 70 | "output_type": "execute_result" 71 | } 72 | ], 73 | "source": [ 74 | "dataframe = pd.read_csv(\"data.csv\") # Demandons à Pandas de charger notre ensemble de données comme une dataframe\n", 75 | "dataframe = dataframe.drop([\"index\", \"price\", \"sq_price\"], axis=1) # Enlever les colonnes qui ne nous intéressent pas\n", 76 | "dataframe = dataframe[0:10] # Nous n'allons utiliser que les 10 premières lignes de l'ensemble de données dans cet exemple\n", 77 | "dataframe # Demandons au bloc-notes de nous montrer à quoi ressemble la dataframe maintenant" 78 | ] 79 | }, 80 | { 81 | "cell_type": "markdown", 82 | "metadata": { 83 | "collapsed": false 84 | }, 85 | "source": [ 86 | "La dataframe n'a pour l'instant que les caractéristiques. Ajoutons les étiquettes." 87 | ] 88 | }, 89 | { 90 | "cell_type": "code", 91 | "execution_count": 29, 92 | "metadata": { 93 | "collapsed": false 94 | }, 95 | "outputs": [ 96 | { 97 | "data": { 98 | "text/plain": " area bathrooms y1 y2\n0 2104.0 3.0 1 0\n1 1600.0 3.0 1 0\n2 2400.0 3.0 1 0\n3 1416.0 2.0 0 1\n4 3000.0 4.0 0 1\n5 1985.0 4.0 1 0\n6 1534.0 3.0 0 1\n7 1427.0 3.0 1 0\n8 1380.0 3.0 1 0\n9 1494.0 3.0 1 0", 99 | "text/html": "
\n\n\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
areabathroomsy1y2
02104.03.010
11600.03.010
22400.03.010
31416.02.001
43000.04.001
51985.04.010
61534.03.001
71427.03.010
81380.03.010
91494.03.010
\n
" 100 | }, 101 | "execution_count": 29, 102 | "metadata": {}, 103 | "output_type": "execute_result" 104 | } 105 | ], 106 | "source": [ 107 | "dataframe.loc[:, (\"y1\")] = [1, 1, 1, 0, 0, 1, 0, 1, 1, 1] # Ceci est la liste de notre amie des maisons qu'elle a aimées\n", 108 | " # 1 = bon, 0 = mauvais\n", 109 | "dataframe.loc[:, (\"y2\")] = dataframe[\"y1\"] == 0 # y2 est la négation de y1\n", 110 | "dataframe.loc[:, (\"y2\")] = dataframe[\"y2\"].astype(int) # Transforme les valeurs TRUE/FALSE en 1/0\n", 111 | "# y2 signifie que nous n'aimons pas une maison\n", 112 | "# (Oui, c'est redondant. Mais apprendre à le faire de cette manière ouvre la porte au classement multi-classes)\n", 113 | "dataframe # À quoi ressemble notre dataframe maintenant ?" 114 | ] 115 | }, 116 | { 117 | "cell_type": "markdown", 118 | "metadata": { 119 | "collapsed": false 120 | }, 121 | "source": [ 122 | "Maintenant que nous avons toutes nos données dans la dataframe,\n", 123 | "nous devons les arranger sous forme de matrices avant de les donner à TensorFlow" 124 | ] 125 | }, 126 | { 127 | "cell_type": "code", 128 | "execution_count": 30, 129 | "metadata": { 130 | "collapsed": false, 131 | "scrolled": false 132 | }, 133 | "outputs": [], 134 | "source": [ 135 | "inputX = dataframe.loc[:, ['area', 'bathrooms']].values\n", 136 | "inputY = dataframe.loc[:, [\"y1\", \"y2\"]].values" 137 | ] 138 | }, 139 | { 140 | "cell_type": "markdown", 141 | "metadata": {}, 142 | "source": [ 143 | "Maintenant notre matrice d'entrée ressemble à ceci:" 144 | ] 145 | }, 146 | { 147 | "cell_type": "code", 148 | "execution_count": 31, 149 | "metadata": { 150 | "collapsed": false 151 | }, 152 | "outputs": [ 153 | { 154 | "data": { 155 | "text/plain": "array([[2.104e+03, 3.000e+00],\n [1.600e+03, 3.000e+00],\n [2.400e+03, 3.000e+00],\n [1.416e+03, 2.000e+00],\n [3.000e+03, 4.000e+00],\n [1.985e+03, 4.000e+00],\n [1.534e+03, 3.000e+00],\n [1.427e+03, 3.000e+00],\n [1.380e+03, 3.000e+00],\n [1.494e+03, 3.000e+00]])" 156 | }, 157 | "execution_count": 31, 158 | "metadata": {}, 159 | "output_type": "execute_result" 160 | } 161 | ], 162 | "source": [ 163 | "inputX" 164 | ] 165 | }, 166 | { 167 | "cell_type": "markdown", 168 | "metadata": {}, 169 | "source": [ 170 | "Et notre matrice d'étiquettes à ceci :" 171 | ] 172 | }, 173 | { 174 | "cell_type": "code", 175 | "execution_count": 32, 176 | "metadata": { 177 | "collapsed": false 178 | }, 179 | "outputs": [ 180 | { 181 | "data": { 182 | "text/plain": "array([[1, 0],\n [1, 0],\n [1, 0],\n [0, 1],\n [0, 1],\n [1, 0],\n [0, 1],\n [1, 0],\n [1, 0],\n [1, 0]])" 183 | }, 184 | "execution_count": 32, 185 | "metadata": {}, 186 | "output_type": "execute_result" 187 | } 188 | ], 189 | "source": [ 190 | "inputY" 191 | ] 192 | }, 193 | { 194 | "cell_type": "markdown", 195 | "metadata": {}, 196 | "source": [ 197 | "Préparons des paramètres pour le processus d'entraînement" 198 | ] 199 | }, 200 | { 201 | "cell_type": "code", 202 | "execution_count": 33, 203 | "metadata": { 204 | "collapsed": false 205 | }, 206 | "outputs": [], 207 | "source": [ 208 | "# Paramètres\n", 209 | "learning_rate = 0.000001\n", 210 | "training_epochs = 2000\n", 211 | "display_step = 50\n", 212 | "n_samples = inputY.size" 213 | ] 214 | }, 215 | { 216 | "cell_type": "markdown", 217 | "metadata": {}, 218 | "source": [ 219 | "Et maintenant définissons les opérations TensorFlow.\n", 220 | "Notez qu'il s'agit d'une étape de déclaration où nous indiquons à TensorFlow comment est calculée la prédiction.\n", 221 | "Si nous l'exécutons, aucun calcul ne sera effectué.\n", 222 | "Il répondra juste qu'il sait maintenant comment faire l'opération." 223 | ] 224 | }, 225 | { 226 | "cell_type": "code", 227 | "execution_count": 34, 228 | "metadata": { 229 | "collapsed": true 230 | }, 231 | "outputs": [], 232 | "source": [ 233 | "x = tf.placeholder(tf.float32, [None, 2]) # Ok TensorFlow, on te donnera un tableau d'exemples. Chaque exemple sera\n", 234 | " # un tableau de 2 valeurs décimales (surface, et nombre de salles de bain).\n", 235 | " # \"None\" veut dire qu'on peut te donner n'importe quel nombre d'exemples\n", 236 | " # Notez que nous ne lui avons pas encore fourni les valeurs\n", 237 | " \n", 238 | "W = tf.Variable(tf.zeros([2, 2])) # Maintiens un matrice de nombre décimaux de 2 x 2 pour les poids que nous mettrons à jour constamment\n", 239 | " # lors du processus d'entraînement (on les met tous à zéro pour commencer)\n", 240 | " \n", 241 | "b = tf.Variable(tf.zeros([2])) # Maintiens aussi 2 valeurs de biais\n", 242 | "\n", 243 | "y_values = tf.add(tf.matmul(x, W), b) # La 1ère étape pour calculer la prédiction sera de multiplier\n", 244 | " # la matrice des entrées par la matrice des poids puis d'ajouter les biais\n", 245 | " \n", 246 | "y = tf.nn.softmax(y_values) # Puis utiliser le softmax comme \"fonction d'activation\" qui traduit les\n", 247 | " # nombres en sortie de la couche précédente sous forme de probabilités\n", 248 | " \n", 249 | "y_ = tf.placeholder(tf.float32, [None,2]) # Pour l'entraînement, on te fournira aussi une matrice d'étiquettes" 250 | ] 251 | }, 252 | { 253 | "cell_type": "markdown", 254 | "metadata": {}, 255 | "source": [ 256 | "Spécifions notre fonction de coût et utilisons la Descente de Gradient" 257 | ] 258 | }, 259 | { 260 | "cell_type": "code", 261 | "execution_count": 35, 262 | "metadata": { 263 | "collapsed": true 264 | }, 265 | "outputs": [], 266 | "source": [ 267 | "\n", 268 | "# Fonction de coût : Erreur quadratique moyenne\n", 269 | "cost = tf.reduce_sum(tf.pow(y_ - y, 2))/(2*n_samples)\n", 270 | "# Descente de gradient\n", 271 | "optimizer = tf.train.GradientDescentOptimizer(learning_rate).minimize(cost)\n" 272 | ] 273 | }, 274 | { 275 | "cell_type": "code", 276 | "execution_count": 36, 277 | "metadata": { 278 | "collapsed": true 279 | }, 280 | "outputs": [], 281 | "source": [ 282 | "# Initialise les variables et la session tensorflow\n", 283 | "init = tf.initialize_all_variables()\n", 284 | "sess = tf.Session()\n", 285 | "sess.run(init)" 286 | ] 287 | }, 288 | { 289 | "cell_type": "markdown", 290 | "metadata": {}, 291 | "source": [ 292 | "*Roulement de tambours*\n", 293 | "\n", 294 | "Et maintenant l'entraînement à proprement parler" 295 | ] 296 | }, 297 | { 298 | "cell_type": "code", 299 | "execution_count": 37, 300 | "metadata": { 301 | "collapsed": false, 302 | "scrolled": false 303 | }, 304 | "outputs": [ 305 | { 306 | "name": "stdout", 307 | "output_type": "stream", 308 | "text": [ 309 | "Etape d entraînement : 0000 coût= 0.114958666\n", 310 | "Etape d entraînement : 0050 coût= 0.109539941\n", 311 | "Etape d entraînement : 0100 coût= 0.109539881\n", 312 | "Etape d entraînement : 0150 coût= 0.109539807\n", 313 | "Etape d entraînement : 0200 coût= 0.109539740\n", 314 | "Etape d entraînement : 0250 coût= 0.109539680\n", 315 | "Etape d entraînement : 0300 coût= 0.109539606\n", 316 | "Etape d entraînement : 0350 coût= 0.109539531\n", 317 | "Etape d entraînement : 0400 coût= 0.109539464\n", 318 | "Etape d entraînement : 0450 coût= 0.109539390\n", 319 | "Etape d entraînement : 0500 coût= 0.109539323\n", 320 | "Etape d entraînement : 0550 coût= 0.109539248\n", 321 | "Etape d entraînement : 0600 coût= 0.109539203\n", 322 | "Etape d entraînement : 0650 coût= 0.109539129\n", 323 | "Etape d entraînement : 0700 coût= 0.109539054\n", 324 | "Etape d entraînement : 0750 coût= 0.109538987\n", 325 | "Etape d entraînement : 0800 coût= 0.109538913\n", 326 | "Etape d entraînement : 0850 coût= 0.109538853\n", 327 | "Etape d entraînement : 0900 coût= 0.109538786\n", 328 | "Etape d entraînement : 0950 coût= 0.109538712\n", 329 | "Etape d entraînement : 1000 coût= 0.109538652\n", 330 | "Etape d entraînement : 1050 coût= 0.109538577\n", 331 | "Etape d entraînement : 1100 coût= 0.109538510\n", 332 | "Etape d entraînement : 1150 coût= 0.109538436\n", 333 | "Etape d entraînement : 1200 coût= 0.109538369\n", 334 | "Etape d entraînement : 1250 coût= 0.109538309\n", 335 | "Etape d entraînement : 1300 coût= 0.109538250\n", 336 | "Etape d entraînement : 1350 coût= 0.109538175\n", 337 | "Etape d entraînement : 1400 coût= 0.109538101\n", 338 | "Etape d entraînement : 1450 coût= 0.109538034\n", 339 | "Etape d entraînement : 1500 coût= 0.109537959\n", 340 | "Etape d entraînement : 1550 coût= 0.109537899\n", 341 | "Etape d entraînement : 1600 coût= 0.109537832\n", 342 | "Etape d entraînement : 1650 coût= 0.109537773\n", 343 | "Etape d entraînement : 1700 coût= 0.109537683\n", 344 | "Etape d entraînement : 1750 coût= 0.109537624\n", 345 | "Etape d entraînement : 1800 coût= 0.109537557\n", 346 | "Etape d entraînement : 1850 coût= 0.109537482\n", 347 | "Etape d entraînement : 1900 coût= 0.109537423\n", 348 | "Etape d entraînement : 1950 coût= 0.109537356\n", 349 | "Optimisation terminée !\n", 350 | "Coût d'entraînement= 0.10953728 W= [[ 2.1414938e-04 -2.1415012e-04]\n", 351 | " [ 5.1274808e-05 -5.1274790e-05]] b= [ 1.19155184e-05 -1.19155275e-05] \n", 352 | "\n" 353 | ] 354 | } 355 | ], 356 | "source": [ 357 | "for i in range(training_epochs): \n", 358 | " sess.run(optimizer, feed_dict={x: inputX, y_: inputY}) # Itère dans la descente de gradient en utilisant nos entrées et étiquettes\n", 359 | "\n", 360 | " # C'est tout ! Le reste de la cellule ne fait qu'afficher des messages de deboguage.\n", 361 | " # Affiche les logs par étape d'époque\n", 362 | " if (i) % display_step == 0:\n", 363 | " cc = sess.run(cost, feed_dict={x: inputX, y_:inputY})\n", 364 | " print(\"Etape d entraînement :\", '%04d' % (i), \"coût=\", \"{:.9f}\".format(cc)) #, \\\"W=\", sess.run(W), \"b=\", sess.run(b)\n", 365 | "\n", 366 | "print(\"Optimisation terminée !\")\n", 367 | "training_cost = sess.run(cost, feed_dict={x: inputX, y_: inputY})\n", 368 | "print(\"Coût d'entraînement=\", training_cost, \"W=\", sess.run(W), \"b=\", sess.run(b), '\\n')\n" 369 | ] 370 | }, 371 | { 372 | "cell_type": "markdown", 373 | "metadata": { 374 | "collapsed": true 375 | }, 376 | "source": [ 377 | "L'entraînement est maintenant terminé.\n", 378 | "TensorFlow garde maintenant notre modèle entraîné\n", 379 | "(qui se résume aux opérations définies,\n", 380 | "plus les variables W et b qui ont été trouvées par le processus d'entraînement).\n", 381 | "\n", 382 | "Une valeur de coût de 0.109537 est-elle bonne ou mauvaise ? Aucune idée.\n", 383 | "Au moins c'est mieux que la 1ère valeur de coût de 0.114958666.\n", 384 | "Utilisons le modèle sur notre dataset pour voir comment il se comporte, quand même :" 385 | ] 386 | }, 387 | { 388 | "cell_type": "code", 389 | "execution_count": 38, 390 | "metadata": { 391 | "collapsed": false 392 | }, 393 | "outputs": [ 394 | { 395 | "data": { 396 | "text/plain": "array([[0.7112522 , 0.28874776],\n [0.66498977, 0.33501023],\n [0.73657656, 0.26342347],\n [0.6471879 , 0.3528121 ],\n [0.78335613, 0.2166439 ],\n [0.7006948 , 0.29930523],\n [0.6586633 , 0.34133676],\n [0.6482863 , 0.35171375],\n [0.6436828 , 0.35631716],\n [0.65480113, 0.3451989 ]], dtype=float32)" 397 | }, 398 | "execution_count": 38, 399 | "metadata": {}, 400 | "output_type": "execute_result" 401 | } 402 | ], 403 | "source": [ 404 | "sess.run(y, feed_dict={x: inputX })" 405 | ] 406 | }, 407 | { 408 | "cell_type": "markdown", 409 | "metadata": {}, 410 | "source": [ 411 | "Donc il pense que toutes sont des bonnes maisons.\n", 412 | "Ca fait 7 bonnes réponses sur 10. Pas super impressionnant.\n", 413 | "Un modèle avec une couche cachée devrait être meilleur, je suppose." 414 | ] 415 | }, 416 | { 417 | "cell_type": "markdown", 418 | "metadata": { 419 | "collapsed": false 420 | }, 421 | "source": [ 422 | "En passant, voici comment j'ai calculé les valeurs de softmax dans l'article :" 423 | ] 424 | }, 425 | { 426 | "cell_type": "code", 427 | "execution_count": 39, 428 | "metadata": { 429 | "collapsed": false 430 | }, 431 | "outputs": [ 432 | { 433 | "data": { 434 | "text/plain": "array([0.26894143, 0.7310586 ], dtype=float32)" 435 | }, 436 | "execution_count": 39, 437 | "metadata": {}, 438 | "output_type": "execute_result" 439 | } 440 | ], 441 | "source": [ 442 | "sess.run(tf.nn.softmax([1., 2.]))" 443 | ] 444 | } 445 | ], 446 | "metadata": { 447 | "anaconda-cloud": {}, 448 | "kernelspec": { 449 | "name": "python3", 450 | "language": "python", 451 | "display_name": "Python 3" 452 | }, 453 | "language_info": { 454 | "codemirror_mode": { 455 | "name": "ipython", 456 | "version": 2 457 | }, 458 | "file_extension": ".py", 459 | "mimetype": "text/x-python", 460 | "name": "python", 461 | "nbconvert_exporter": "python", 462 | "pygments_lexer": "ipython2", 463 | "version": "2.7.12" 464 | } 465 | }, 466 | "nbformat": 4, 467 | "nbformat_minor": 0 468 | } --------------------------------------------------------------------------------