├── img1_TFsite.jpg ├── img2_error.jpg ├── README.md └── 0-TFData-For-Keras-Estimator-gcloud.ipynb /img1_TFsite.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/terryum/tf.data-for-keras-and-tensorflow-estimator/HEAD/img1_TFsite.jpg -------------------------------------------------------------------------------- /img2_error.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/terryum/tf.data-for-keras-and-tensorflow-estimator/HEAD/img2_error.jpg -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # tf.data examples for keras and estimator models 2 | 3 | 4 | **Please refer to [the jupyter notebook file](https://github.com/terryum/tf.data-for-keras-and-tensorflow-estimator/blob/master/0-TFData-For-Keras-Estimator-gcloud.ipynb) for details.** 5 | 6 |
7 | 8 | ## Motivation 9 | The official [Tensorflow guide](https://www.tensorflow.org/guide/keras#input_tfdata_datasets) says that tf.data.Dataset can be directly fed into model.fit in Keras since TF 1.09. However, it is an absolute lie at the moment of TF 1.10, Sept 2018, as you can see from the following images. 10 | 11 | ![Tensorflow site](https://github.com/terryum/tf.data-for-keras-and-tensorflow-estimator/blob/master/img1_TFsite.jpg) 12 | ![Error](https://github.com/terryum/tf.data-for-keras-and-tensorflow-estimator/blob/master/img2_error.jpg) 13 | 14 | I have spent three days to figure out what the problem is, but it turns out that **IT IS JUST NOT WORKING**. Instead, it is working with **TF-Nightly** (TF 1.12-dev at this moment) rather than TF 1.10, as discussed [here](https://github.com/tensorflow/tensorflow/issues/21894) and [here](https://medium.com/tensorflow/training-and-serving-ml-models-with-tf-keras-fd975cc0fa27). 15 | 16 | For those who may suffer from the similar problems that I had, I made [this tutorial code](https://github.com/terryum/tf.data-for-keras-and-tensorflow-estimator/blob/master/0-TFData-For-Keras-Estimator-gcloud.ipynb) to show how you can incorporate tf.data into your Keras model. 17 | 18 |
19 | 20 | ## Contents 21 | 22 | **[IMPORTANT] You should first install `tf-nightly` or `tf-nightly-gpu` via `pip install tf-nightly` or `pip install tf-nightly-gpu` if you have intalled a tensorflow under 1.11 version 23 | 24 | * [This code](https://github.com/terryum/tf.data-for-keras-and-tensorflow-estimator/blob/master/0-TFData-For-Keras-Estimator-gcloud.ipynb) consists of four parts as followings: 25 | 26 | #### 0. Data & Model preparation 27 | Load MNIST dataset and build a simple 2-layer MLP model (784-40-40-10) using Keras. 28 | 29 | #### 1. Train the model usgin a tf.data.Dataset loaded from numpy data on the memory. 30 | Assumption: The amount of your data is enough small, thus you can load them on the memory. 31 | Solution: Make a tf.data.Dataset and train the model using it. 32 | 33 | #### 2. Train the model usgin a tf.data.Dataset loaded from a TF record file. 34 | Assumption: The amount of your data is to large to load on the memory. 35 | Solution: Create a TFRecord file first, and make a tf.data.Dataset from it. 36 | 37 | * Estimator is a standard ML model for real-world products which is ready to deploy on google cloud. 38 | 39 | #### 3. Use a pre-made estimator and tf.data made from a TF record file. 40 | This code first shows how tf.data can be naturally incorporated into a pre-made estimator provided by Tensorflow APIs. 41 | 42 | #### 4. Use a custom estimator created by using Keras 43 | You may want to build a custom model using Keras and train it using your large-scale data. One of the easiest ways to do it is (i) to write a TF record file, (ii) make a tf.data.Dataset from it, (iii) buile a model using Keras, (iv) convert the model to an estimator using `model_to_estimator`, (v) train the model like the previous example. 44 | 45 | #### 5. Access the data on google cloud 46 | This code explain how to upload your data from local to a google storage (bucket), get the list of files on the bucket, and train the model using the data on the bucket. 47 | 48 |
49 | 50 | ## Thanks 51 | I hope this tutorial could save your time. Feel free to use or change this code for your own purpose. If you are intereted in deep learning research, you can follow me on [twitter](https://twitter.com/TerryUm_ML) and have interesting discussions. 52 | -------------------------------------------------------------------------------- /0-TFData-For-Keras-Estimator-gcloud.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "## Tutorial for using TF.Data, Keras, Estimator, and Google storage" 8 | ] 9 | }, 10 | { 11 | "cell_type": "code", 12 | "execution_count": 1, 13 | "metadata": {}, 14 | "outputs": [], 15 | "source": [ 16 | "import numpy as np\n", 17 | "import matplotlib.pyplot as plt" 18 | ] 19 | }, 20 | { 21 | "cell_type": "markdown", 22 | "metadata": {}, 23 | "source": [ 24 | "Make sure you have TF >= 1.11" 25 | ] 26 | }, 27 | { 28 | "cell_type": "markdown", 29 | "metadata": {}, 30 | "source": [ 31 | "Please install tf-nightly-gpu by ```pip install --upgrade tf-nightly``` if you have TF < 1.11" 32 | ] 33 | }, 34 | { 35 | "cell_type": "code", 36 | "execution_count": 2, 37 | "metadata": {}, 38 | "outputs": [ 39 | { 40 | "name": "stdout", 41 | "output_type": "stream", 42 | "text": [ 43 | "1.11.0\n" 44 | ] 45 | } 46 | ], 47 | "source": [ 48 | "from tensorflow import keras\n", 49 | "import tensorflow as tf\n", 50 | "print(tf.__version__)" 51 | ] 52 | }, 53 | { 54 | "cell_type": "markdown", 55 | "metadata": {}, 56 | "source": [ 57 | "### 0. Data & Model preparation" 58 | ] 59 | }, 60 | { 61 | "cell_type": "markdown", 62 | "metadata": {}, 63 | "source": [ 64 | "Load MNIST dataset" 65 | ] 66 | }, 67 | { 68 | "cell_type": "code", 69 | "execution_count": 3, 70 | "metadata": {}, 71 | "outputs": [ 72 | { 73 | "name": "stdout", 74 | "output_type": "stream", 75 | "text": [ 76 | "(60000, 28, 28) (10000, 28, 28) (60000,) (10000,)\n" 77 | ] 78 | } 79 | ], 80 | "source": [ 81 | "(x_train,y_train),(x_test,y_test) = keras.datasets.mnist.load_data()\n", 82 | "print(x_train.shape, x_test.shape, y_train.shape, y_test.shape)" 83 | ] 84 | }, 85 | { 86 | "cell_type": "markdown", 87 | "metadata": {}, 88 | "source": [ 89 | "X and Y are usually prepared in float32 & int32" 90 | ] 91 | }, 92 | { 93 | "cell_type": "code", 94 | "execution_count": 4, 95 | "metadata": {}, 96 | "outputs": [], 97 | "source": [ 98 | "x_train = x_train.astype(np.float32)\n", 99 | "x_test = x_test.astype(np.float32)\n", 100 | "y_train = y_train.astype(np.int32)\n", 101 | "y_test = y_test.astype(np.int32)" 102 | ] 103 | }, 104 | { 105 | "cell_type": "code", 106 | "execution_count": 5, 107 | "metadata": {}, 108 | "outputs": [ 109 | { 110 | "name": "stdout", 111 | "output_type": "stream", 112 | "text": [ 113 | "60000 10000 784 10\n" 114 | ] 115 | } 116 | ], 117 | "source": [ 118 | "nClass = 10\n", 119 | "nData_train = x_train.shape[0]\n", 120 | "nData_test = x_test.shape[0]\n", 121 | "nDimIn = x_train.shape[1]*x_train.shape[2]\n", 122 | "nDimOut = nClass\n", 123 | "print(nData_train, nData_test, nDimIn, nDimOut) " 124 | ] 125 | }, 126 | { 127 | "cell_type": "markdown", 128 | "metadata": {}, 129 | "source": [ 130 | "My Keras Model" 131 | ] 132 | }, 133 | { 134 | "cell_type": "code", 135 | "execution_count": 6, 136 | "metadata": {}, 137 | "outputs": [], 138 | "source": [ 139 | "def myModel():\n", 140 | " dataIn = keras.Input(shape=(nDimIn,), name='X')\n", 141 | " fc1 = keras.layers.Dense(40, activation='relu', name='fc1')(dataIn)\n", 142 | " fc2 = keras.layers.Dense(40, activation='relu', name='fc2')(fc1)\n", 143 | " dataOut = keras.layers.Dense(nDimOut, activation='softmax', name='dataOut')(fc2)\n", 144 | " model = keras.Model(inputs=dataIn, outputs=dataOut, name='Y')\n", 145 | " model.compile(optimizer=tf.train.AdamOptimizer(0.001),loss='categorical_crossentropy',metrics=['accuracy'])\n", 146 | " return model\n", 147 | "batchSize = 64\n", 148 | "nEpoch = 5\n", 149 | "nSteps = (int)(nData_train/batchSize)" 150 | ] 151 | }, 152 | { 153 | "cell_type": "markdown", 154 | "metadata": {}, 155 | "source": [ 156 | "number of steps for training & testing each epoch" 157 | ] 158 | }, 159 | { 160 | "cell_type": "code", 161 | "execution_count": 7, 162 | "metadata": {}, 163 | "outputs": [], 164 | "source": [ 165 | "nStep_train = (int)(nData_train/batchSize)\n", 166 | "nStep_test = (int)(nData_test/batchSize)" 167 | ] 168 | }, 169 | { 170 | "cell_type": "code", 171 | "execution_count": 8, 172 | "metadata": {}, 173 | "outputs": [ 174 | { 175 | "name": "stdout", 176 | "output_type": "stream", 177 | "text": [ 178 | "_________________________________________________________________\n", 179 | "Layer (type) Output Shape Param # \n", 180 | "=================================================================\n", 181 | "X (InputLayer) (None, 784) 0 \n", 182 | "_________________________________________________________________\n", 183 | "fc1 (Dense) (None, 40) 31400 \n", 184 | "_________________________________________________________________\n", 185 | "fc2 (Dense) (None, 40) 1640 \n", 186 | "_________________________________________________________________\n", 187 | "dataOut (Dense) (None, 10) 410 \n", 188 | "=================================================================\n", 189 | "Total params: 33,450\n", 190 | "Trainable params: 33,450\n", 191 | "Non-trainable params: 0\n", 192 | "_________________________________________________________________\n" 193 | ] 194 | } 195 | ], 196 | "source": [ 197 | "model = myModel()\n", 198 | "model.summary()\n", 199 | "del model\n", 200 | "keras.backend.clear_session()" 201 | ] 202 | }, 203 | { 204 | "cell_type": "markdown", 205 | "metadata": {}, 206 | "source": [ 207 | "### 1. Using Keras with tf.data from numpy data on the memory" 208 | ] 209 | }, 210 | { 211 | "cell_type": "markdown", 212 | "metadata": {}, 213 | "source": [ 214 | "Data preprocessing" 215 | ] 216 | }, 217 | { 218 | "cell_type": "code", 219 | "execution_count": 9, 220 | "metadata": {}, 221 | "outputs": [], 222 | "source": [ 223 | "def myMapFunc_npy(xx, yy):\n", 224 | " norm = tf.constant(255, dtype=tf.float32, shape=(nDimIn,))\n", 225 | " xx = tf.div(tf.reshape(xx, [-1]), norm)\n", 226 | " yy = tf.one_hot(yy, nClass)\n", 227 | " return xx, yy" 228 | ] 229 | }, 230 | { 231 | "cell_type": "markdown", 232 | "metadata": {}, 233 | "source": [ 234 | "Prepare train and test datasets using tf.data" 235 | ] 236 | }, 237 | { 238 | "cell_type": "markdown", 239 | "metadata": {}, 240 | "source": [ 241 | "shuffle 10000 instances -> data transformation (e.g. normalization) -> minibatches" 242 | ] 243 | }, 244 | { 245 | "cell_type": "code", 246 | "execution_count": 10, 247 | "metadata": { 248 | "scrolled": true 249 | }, 250 | "outputs": [], 251 | "source": [ 252 | "data_train = tf.data.Dataset.from_tensor_slices((x_train, y_train))\n", 253 | "data_train = data_train.shuffle(10000).map(myMapFunc_npy).batch(batchSize).repeat()\n", 254 | "data_test = tf.data.Dataset.from_tensor_slices((x_test, y_test))\n", 255 | "data_test = data_test.shuffle(10000).map(myMapFunc_npy).batch(batchSize).repeat()" 256 | ] 257 | }, 258 | { 259 | "cell_type": "code", 260 | "execution_count": 11, 261 | "metadata": {}, 262 | "outputs": [], 263 | "source": [ 264 | "model = myModel()" 265 | ] 266 | }, 267 | { 268 | "cell_type": "code", 269 | "execution_count": 12, 270 | "metadata": { 271 | "scrolled": false 272 | }, 273 | "outputs": [ 274 | { 275 | "name": "stdout", 276 | "output_type": "stream", 277 | "text": [ 278 | "Epoch 1/5\n", 279 | "937/937 [==============================] - 10s 11ms/step - loss: 0.3820 - acc: 0.8890 - val_loss: 0.2067 - val_acc: 0.9387\n", 280 | "Epoch 2/5\n", 281 | "937/937 [==============================] - 7s 7ms/step - loss: 0.1763 - acc: 0.9484 - val_loss: 0.1485 - val_acc: 0.9553\n", 282 | "Epoch 3/5\n", 283 | "937/937 [==============================] - 7s 7ms/step - loss: 0.1366 - acc: 0.9595 - val_loss: 0.1286 - val_acc: 0.9596\n", 284 | "Epoch 4/5\n", 285 | "937/937 [==============================] - 7s 7ms/step - loss: 0.1124 - acc: 0.9664 - val_loss: 0.1150 - val_acc: 0.9660\n", 286 | "Epoch 5/5\n", 287 | "937/937 [==============================] - 7s 7ms/step - loss: 0.0949 - acc: 0.9706 - val_loss: 0.1151 - val_acc: 0.9651\n" 288 | ] 289 | }, 290 | { 291 | "data": { 292 | "text/plain": [ 293 | "" 294 | ] 295 | }, 296 | "execution_count": 12, 297 | "metadata": {}, 298 | "output_type": "execute_result" 299 | } 300 | ], 301 | "source": [ 302 | "model.fit(data_train, epochs=nEpoch, validation_data=data_test, steps_per_epoch=nStep_train, validation_steps=nStep_test)" 303 | ] 304 | }, 305 | { 306 | "cell_type": "code", 307 | "execution_count": 13, 308 | "metadata": {}, 309 | "outputs": [], 310 | "source": [ 311 | "del data_train, data_test, model\n", 312 | "keras.backend.clear_session()" 313 | ] 314 | }, 315 | { 316 | "cell_type": "markdown", 317 | "metadata": {}, 318 | "source": [ 319 | "### 2. Using Keras with tf.data from TFRecord files" 320 | ] 321 | }, 322 | { 323 | "cell_type": "markdown", 324 | "metadata": {}, 325 | "source": [ 326 | "#### 2-1. Write a TFRecord file" 327 | ] 328 | }, 329 | { 330 | "cell_type": "markdown", 331 | "metadata": {}, 332 | "source": [ 333 | "Write binary files" 334 | ] 335 | }, 336 | { 337 | "cell_type": "code", 338 | "execution_count": 14, 339 | "metadata": {}, 340 | "outputs": [], 341 | "source": [ 342 | "def _int64_feature(value):\n", 343 | " return tf.train.Feature(int64_list=tf.train.Int64List(value=[value]))\n", 344 | "def _float_feature(array):\n", 345 | " return tf.train.Feature(float_list=tf.train.FloatList(value=array))\n", 346 | "def _bytes_feature(value):\n", 347 | " return tf.train.Feature(bytes_list=tf.train.BytesList(value=[value]))\n", 348 | "\n", 349 | "def myWriteTFRecord(filename, xx, yy):\n", 350 | " writer = tf.python_io.TFRecordWriter(filename)\n", 351 | " for ii in range(len(yy)):\n", 352 | " myFeat = tf.train.Features(feature={\n", 353 | " 'X': _float_feature(xx[ii]),\n", 354 | " 'Y': _int64_feature(yy[ii])})\n", 355 | " example = tf.train.Example(features=myFeat)\n", 356 | " writer.write(example.SerializeToString())\n", 357 | " writer.close()" 358 | ] 359 | }, 360 | { 361 | "cell_type": "markdown", 362 | "metadata": {}, 363 | "source": [ 364 | "Data should have 2-dim, (N*D)" 365 | ] 366 | }, 367 | { 368 | "cell_type": "code", 369 | "execution_count": 15, 370 | "metadata": {}, 371 | "outputs": [], 372 | "source": [ 373 | "x_train_vec = x_train.reshape([nData_train,-1])\n", 374 | "x_test_vec = x_test.reshape([nData_test,-1])" 375 | ] 376 | }, 377 | { 378 | "cell_type": "code", 379 | "execution_count": 16, 380 | "metadata": {}, 381 | "outputs": [], 382 | "source": [ 383 | "file_train = 'mnist_train.tfrecords'\n", 384 | "file_test = 'mnist_test.tfrecords'" 385 | ] 386 | }, 387 | { 388 | "cell_type": "code", 389 | "execution_count": 17, 390 | "metadata": {}, 391 | "outputs": [], 392 | "source": [ 393 | "myWriteTFRecord(file_train, x_train_vec, y_train)\n", 394 | "myWriteTFRecord(file_test, x_test_vec, y_test)" 395 | ] 396 | }, 397 | { 398 | "cell_type": "markdown", 399 | "metadata": {}, 400 | "source": [ 401 | "#### 2-2. Training from the TFRecord files" 402 | ] 403 | }, 404 | { 405 | "cell_type": "markdown", 406 | "metadata": {}, 407 | "source": [ 408 | "Data preprocessing" 409 | ] 410 | }, 411 | { 412 | "cell_type": "code", 413 | "execution_count": 18, 414 | "metadata": {}, 415 | "outputs": [], 416 | "source": [ 417 | "def myMapFunc_onehot(example):\n", 418 | " feature_def = {'X': tf.FixedLenFeature(nDimIn, tf.float32),\n", 419 | " 'Y': tf.FixedLenFeature(1, tf.int64)}\n", 420 | " features = tf.parse_single_example(example, feature_def)\n", 421 | " norm = tf.constant(255, dtype=tf.float32, shape=(nDimIn,))\n", 422 | " xx = tf.div(features['X'], norm)\n", 423 | " yy = tf.reshape(tf.one_hot(features['Y'], nClass, dtype=tf.float32), [-1])\n", 424 | " return xx, yy" 425 | ] 426 | }, 427 | { 428 | "cell_type": "markdown", 429 | "metadata": {}, 430 | "source": [ 431 | "Prepare train and test datasets using tf.data" 432 | ] 433 | }, 434 | { 435 | "cell_type": "code", 436 | "execution_count": 19, 437 | "metadata": {}, 438 | "outputs": [], 439 | "source": [ 440 | "data_train = tf.data.TFRecordDataset(file_train)\n", 441 | "data_train = data_train.shuffle(nData_train).map(myMapFunc_onehot).batch(batchSize).repeat()\n", 442 | "data_test = tf.data.TFRecordDataset(file_test)\n", 443 | "data_test = data_test.shuffle(nData_test).map(myMapFunc_onehot).batch(batchSize).repeat()" 444 | ] 445 | }, 446 | { 447 | "cell_type": "code", 448 | "execution_count": 20, 449 | "metadata": {}, 450 | "outputs": [], 451 | "source": [ 452 | "model = myModel()" 453 | ] 454 | }, 455 | { 456 | "cell_type": "code", 457 | "execution_count": 21, 458 | "metadata": {}, 459 | "outputs": [ 460 | { 461 | "name": "stdout", 462 | "output_type": "stream", 463 | "text": [ 464 | "Epoch 1/5\n", 465 | "937/937 [==============================] - 8s 8ms/step - loss: 0.3822 - acc: 0.8900 - val_loss: 0.2071 - val_acc: 0.9393\n", 466 | "Epoch 2/5\n", 467 | "937/937 [==============================] - 7s 8ms/step - loss: 0.1816 - acc: 0.9478 - val_loss: 0.1566 - val_acc: 0.9508\n", 468 | "Epoch 3/5\n", 469 | "937/937 [==============================] - 8s 8ms/step - loss: 0.1345 - acc: 0.9604 - val_loss: 0.1308 - val_acc: 0.9619\n", 470 | "Epoch 4/5\n", 471 | "937/937 [==============================] - 8s 8ms/step - loss: 0.1096 - acc: 0.9673 - val_loss: 0.1208 - val_acc: 0.9643\n", 472 | "Epoch 5/5\n", 473 | "937/937 [==============================] - 8s 8ms/step - loss: 0.0929 - acc: 0.9723 - val_loss: 0.1053 - val_acc: 0.9696\n" 474 | ] 475 | }, 476 | { 477 | "data": { 478 | "text/plain": [ 479 | "" 480 | ] 481 | }, 482 | "execution_count": 21, 483 | "metadata": {}, 484 | "output_type": "execute_result" 485 | } 486 | ], 487 | "source": [ 488 | "model.fit(data_train, epochs=nEpoch, validation_data=data_test, steps_per_epoch=nStep_train, validation_steps=nStep_test)" 489 | ] 490 | }, 491 | { 492 | "cell_type": "code", 493 | "execution_count": 22, 494 | "metadata": {}, 495 | "outputs": [], 496 | "source": [ 497 | "del data_train, data_test, model\n", 498 | "keras.backend.clear_session()" 499 | ] 500 | }, 501 | { 502 | "cell_type": "code", 503 | "execution_count": null, 504 | "metadata": {}, 505 | "outputs": [], 506 | "source": [] 507 | }, 508 | { 509 | "cell_type": "markdown", 510 | "metadata": {}, 511 | "source": [ 512 | "### 3. Using a pre-made estimator with tf.data from TFRecord files" 513 | ] 514 | }, 515 | { 516 | "cell_type": "markdown", 517 | "metadata": {}, 518 | "source": [ 519 | "Data preprocessing" 520 | ] 521 | }, 522 | { 523 | "cell_type": "code", 524 | "execution_count": 23, 525 | "metadata": {}, 526 | "outputs": [], 527 | "source": [ 528 | "def myMapFunc_scalar(example):\n", 529 | " feature_def = {'X': tf.FixedLenFeature(nDimIn, tf.float32),\n", 530 | " 'Y': tf.FixedLenFeature(1, tf.int64)}\n", 531 | " features = tf.parse_single_example(example, feature_def)\n", 532 | " norm = tf.constant(255, dtype=tf.float32, shape=(nDimIn,))\n", 533 | " xx = tf.div(features['X'], norm)\n", 534 | " yy = features['Y']\n", 535 | " return {'X': xx}, yy" 536 | ] 537 | }, 538 | { 539 | "cell_type": "markdown", 540 | "metadata": {}, 541 | "source": [ 542 | "TF Estimator require an input function" 543 | ] 544 | }, 545 | { 546 | "cell_type": "code", 547 | "execution_count": 24, 548 | "metadata": {}, 549 | "outputs": [], 550 | "source": [ 551 | "def myInputFunc_scalar(filename,numData):\n", 552 | " data_temp = tf.data.TFRecordDataset(filename)\n", 553 | " data_temp = data_temp.shuffle(buffer_size=numData).map(myMapFunc_scalar).batch(batchSize).repeat()\n", 554 | " return data_temp" 555 | ] 556 | }, 557 | { 558 | "cell_type": "markdown", 559 | "metadata": {}, 560 | "source": [ 561 | "Create an estimator" 562 | ] 563 | }, 564 | { 565 | "cell_type": "code", 566 | "execution_count": 25, 567 | "metadata": { 568 | "scrolled": true 569 | }, 570 | "outputs": [ 571 | { 572 | "name": "stdout", 573 | "output_type": "stream", 574 | "text": [ 575 | "INFO:tensorflow:Using default config.\n", 576 | "WARNING:tensorflow:Using temporary folder as model directory: /tmp/tmpcj7___ya\n", 577 | "INFO:tensorflow:Using config: {'_keep_checkpoint_max': 5, '_num_ps_replicas': 0, '_eval_distribute': None, '_log_step_count_steps': 100, '_device_fn': None, '_cluster_spec': , '_model_dir': '/tmp/tmpcj7___ya', '_session_config': allow_soft_placement: true\n", 578 | "graph_options {\n", 579 | " rewrite_options {\n", 580 | " meta_optimizer_iterations: ONE\n", 581 | " }\n", 582 | "}\n", 583 | ", '_train_distribute': None, '_protocol': None, '_evaluation_master': '', '_save_checkpoints_steps': None, '_service': None, '_num_worker_replicas': 1, '_save_summary_steps': 100, '_is_chief': True, '_save_checkpoints_secs': 600, '_master': '', '_tf_random_seed': None, '_task_id': 0, '_keep_checkpoint_every_n_hours': 10000, '_task_type': 'worker', '_experimental_distribute': None, '_global_id_in_cluster': 0}\n" 584 | ] 585 | } 586 | ], 587 | "source": [ 588 | "feature_columns = [tf.feature_column.numeric_column('X', shape=[784,])]\n", 589 | "estimator = tf.estimator.DNNClassifier(\n", 590 | " feature_columns=feature_columns,\n", 591 | " hidden_units=[40, 40],\n", 592 | " optimizer=tf.train.AdamOptimizer(0.001),\n", 593 | " n_classes=10,\n", 594 | ")" 595 | ] 596 | }, 597 | { 598 | "cell_type": "markdown", 599 | "metadata": {}, 600 | "source": [ 601 | "Training" 602 | ] 603 | }, 604 | { 605 | "cell_type": "code", 606 | "execution_count": 26, 607 | "metadata": { 608 | "scrolled": true 609 | }, 610 | "outputs": [ 611 | { 612 | "name": "stdout", 613 | "output_type": "stream", 614 | "text": [ 615 | "INFO:tensorflow:Calling model_fn.\n", 616 | "INFO:tensorflow:Done calling model_fn.\n", 617 | "INFO:tensorflow:Create CheckpointSaverHook.\n", 618 | "INFO:tensorflow:Graph was finalized.\n", 619 | "INFO:tensorflow:Running local_init_op.\n", 620 | "INFO:tensorflow:Done running local_init_op.\n", 621 | "INFO:tensorflow:Saving checkpoints for 0 into /tmp/tmpcj7___ya/model.ckpt.\n", 622 | "INFO:tensorflow:loss = 149.6716, step = 0\n", 623 | "INFO:tensorflow:global_step/sec: 149.576\n", 624 | "INFO:tensorflow:loss = 30.462986, step = 100 (0.670 sec)\n", 625 | "INFO:tensorflow:global_step/sec: 160.669\n", 626 | "INFO:tensorflow:loss = 15.723204, step = 200 (0.622 sec)\n", 627 | "INFO:tensorflow:global_step/sec: 162.861\n", 628 | "INFO:tensorflow:loss = 25.859535, step = 300 (0.614 sec)\n", 629 | "INFO:tensorflow:global_step/sec: 158.054\n", 630 | "INFO:tensorflow:loss = 28.355984, step = 400 (0.633 sec)\n", 631 | "INFO:tensorflow:global_step/sec: 162.849\n", 632 | "INFO:tensorflow:loss = 14.388325, step = 500 (0.615 sec)\n", 633 | "INFO:tensorflow:global_step/sec: 162.55\n", 634 | "INFO:tensorflow:loss = 10.755209, step = 600 (0.615 sec)\n", 635 | "INFO:tensorflow:global_step/sec: 163.504\n", 636 | "INFO:tensorflow:loss = 15.831234, step = 700 (0.610 sec)\n", 637 | "INFO:tensorflow:global_step/sec: 166.786\n", 638 | "INFO:tensorflow:loss = 12.951447, step = 800 (0.599 sec)\n", 639 | "INFO:tensorflow:global_step/sec: 164.997\n", 640 | "INFO:tensorflow:loss = 12.315826, step = 900 (0.606 sec)\n", 641 | "INFO:tensorflow:global_step/sec: 129.125\n", 642 | "INFO:tensorflow:loss = 9.19237, step = 1000 (0.774 sec)\n", 643 | "INFO:tensorflow:global_step/sec: 211.461\n", 644 | "INFO:tensorflow:loss = 7.662009, step = 1100 (0.473 sec)\n", 645 | "INFO:tensorflow:global_step/sec: 211.654\n", 646 | "INFO:tensorflow:loss = 7.3218513, step = 1200 (0.474 sec)\n", 647 | "INFO:tensorflow:global_step/sec: 208.061\n", 648 | "INFO:tensorflow:loss = 12.865033, step = 1300 (0.479 sec)\n", 649 | "INFO:tensorflow:global_step/sec: 210.549\n", 650 | "INFO:tensorflow:loss = 6.351084, step = 1400 (0.475 sec)\n", 651 | "INFO:tensorflow:global_step/sec: 205.525\n", 652 | "INFO:tensorflow:loss = 11.366968, step = 1500 (0.486 sec)\n", 653 | "INFO:tensorflow:global_step/sec: 211.033\n", 654 | "INFO:tensorflow:loss = 2.6814349, step = 1600 (0.474 sec)\n", 655 | "INFO:tensorflow:global_step/sec: 209.859\n", 656 | "INFO:tensorflow:loss = 9.275259, step = 1700 (0.477 sec)\n", 657 | "INFO:tensorflow:global_step/sec: 209.801\n", 658 | "INFO:tensorflow:loss = 12.292616, step = 1800 (0.477 sec)\n", 659 | "INFO:tensorflow:global_step/sec: 125.172\n", 660 | "INFO:tensorflow:loss = 10.119383, step = 1900 (0.799 sec)\n", 661 | "INFO:tensorflow:global_step/sec: 213.364\n", 662 | "INFO:tensorflow:loss = 17.484518, step = 2000 (0.469 sec)\n", 663 | "INFO:tensorflow:global_step/sec: 211.072\n", 664 | "INFO:tensorflow:loss = 15.2436075, step = 2100 (0.474 sec)\n", 665 | "INFO:tensorflow:global_step/sec: 210.062\n", 666 | "INFO:tensorflow:loss = 9.485586, step = 2200 (0.477 sec)\n", 667 | "INFO:tensorflow:global_step/sec: 206.902\n", 668 | "INFO:tensorflow:loss = 1.5582359, step = 2300 (0.482 sec)\n", 669 | "INFO:tensorflow:global_step/sec: 208.006\n", 670 | "INFO:tensorflow:loss = 6.7894545, step = 2400 (0.481 sec)\n", 671 | "INFO:tensorflow:global_step/sec: 205.531\n", 672 | "INFO:tensorflow:loss = 3.0862002, step = 2500 (0.486 sec)\n", 673 | "INFO:tensorflow:global_step/sec: 206.22\n", 674 | "INFO:tensorflow:loss = 8.876631, step = 2600 (0.485 sec)\n", 675 | "INFO:tensorflow:global_step/sec: 208.961\n", 676 | "INFO:tensorflow:loss = 3.3888192, step = 2700 (0.479 sec)\n", 677 | "INFO:tensorflow:global_step/sec: 210.948\n", 678 | "INFO:tensorflow:loss = 2.971579, step = 2800 (0.474 sec)\n", 679 | "INFO:tensorflow:global_step/sec: 126.357\n", 680 | "INFO:tensorflow:loss = 1.4283128, step = 2900 (0.791 sec)\n", 681 | "INFO:tensorflow:global_step/sec: 212.384\n", 682 | "INFO:tensorflow:loss = 4.1364064, step = 3000 (0.471 sec)\n", 683 | "INFO:tensorflow:global_step/sec: 213.83\n", 684 | "INFO:tensorflow:loss = 8.534002, step = 3100 (0.468 sec)\n", 685 | "INFO:tensorflow:global_step/sec: 215.58\n", 686 | "INFO:tensorflow:loss = 10.292244, step = 3200 (0.464 sec)\n", 687 | "INFO:tensorflow:global_step/sec: 214.233\n", 688 | "INFO:tensorflow:loss = 7.7006454, step = 3300 (0.467 sec)\n", 689 | "INFO:tensorflow:global_step/sec: 214.121\n", 690 | "INFO:tensorflow:loss = 18.63194, step = 3400 (0.467 sec)\n", 691 | "INFO:tensorflow:global_step/sec: 210.957\n", 692 | "INFO:tensorflow:loss = 6.8191833, step = 3500 (0.474 sec)\n", 693 | "INFO:tensorflow:global_step/sec: 209.575\n", 694 | "INFO:tensorflow:loss = 5.535833, step = 3600 (0.477 sec)\n", 695 | "INFO:tensorflow:global_step/sec: 208.832\n", 696 | "INFO:tensorflow:loss = 5.663759, step = 3700 (0.479 sec)\n", 697 | "INFO:tensorflow:global_step/sec: 125.327\n", 698 | "INFO:tensorflow:loss = 8.055286, step = 3800 (0.798 sec)\n", 699 | "INFO:tensorflow:global_step/sec: 211.721\n", 700 | "INFO:tensorflow:loss = 3.4869452, step = 3900 (0.472 sec)\n", 701 | "INFO:tensorflow:global_step/sec: 213.894\n", 702 | "INFO:tensorflow:loss = 16.778776, step = 4000 (0.468 sec)\n", 703 | "INFO:tensorflow:global_step/sec: 213.763\n", 704 | "INFO:tensorflow:loss = 2.628328, step = 4100 (0.469 sec)\n", 705 | "INFO:tensorflow:global_step/sec: 214.407\n", 706 | "INFO:tensorflow:loss = 11.726524, step = 4200 (0.465 sec)\n", 707 | "INFO:tensorflow:global_step/sec: 211.476\n", 708 | "INFO:tensorflow:loss = 7.5534616, step = 4300 (0.473 sec)\n", 709 | "INFO:tensorflow:global_step/sec: 211.84\n", 710 | "INFO:tensorflow:loss = 2.5335348, step = 4400 (0.472 sec)\n", 711 | "INFO:tensorflow:global_step/sec: 212.697\n", 712 | "INFO:tensorflow:loss = 14.222668, step = 4500 (0.470 sec)\n", 713 | "INFO:tensorflow:global_step/sec: 214.66\n", 714 | "INFO:tensorflow:loss = 0.4138863, step = 4600 (0.466 sec)\n", 715 | "INFO:tensorflow:Saving checkpoints for 4685 into /tmp/tmpcj7___ya/model.ckpt.\n", 716 | "INFO:tensorflow:Loss for final step: 5.200771.\n" 717 | ] 718 | }, 719 | { 720 | "data": { 721 | "text/plain": [ 722 | "" 723 | ] 724 | }, 725 | "execution_count": 26, 726 | "metadata": {}, 727 | "output_type": "execute_result" 728 | } 729 | ], 730 | "source": [ 731 | "estimator.train(input_fn=lambda:myInputFunc_scalar(file_train,nData_train), steps=nStep_train*nEpoch)" 732 | ] 733 | }, 734 | { 735 | "cell_type": "markdown", 736 | "metadata": {}, 737 | "source": [ 738 | "Test" 739 | ] 740 | }, 741 | { 742 | "cell_type": "code", 743 | "execution_count": 27, 744 | "metadata": {}, 745 | "outputs": [ 746 | { 747 | "name": "stdout", 748 | "output_type": "stream", 749 | "text": [ 750 | "INFO:tensorflow:Calling model_fn.\n", 751 | "INFO:tensorflow:Done calling model_fn.\n", 752 | "INFO:tensorflow:Starting evaluation at 2018-10-02-20:32:36\n", 753 | "INFO:tensorflow:Graph was finalized.\n", 754 | "INFO:tensorflow:Restoring parameters from /tmp/tmpcj7___ya/model.ckpt-4685\n", 755 | "INFO:tensorflow:Running local_init_op.\n", 756 | "INFO:tensorflow:Done running local_init_op.\n", 757 | "INFO:tensorflow:Evaluation [15/156]\n", 758 | "INFO:tensorflow:Evaluation [30/156]\n", 759 | "INFO:tensorflow:Evaluation [45/156]\n", 760 | "INFO:tensorflow:Evaluation [60/156]\n", 761 | "INFO:tensorflow:Evaluation [75/156]\n", 762 | "INFO:tensorflow:Evaluation [90/156]\n", 763 | "INFO:tensorflow:Evaluation [105/156]\n", 764 | "INFO:tensorflow:Evaluation [120/156]\n", 765 | "INFO:tensorflow:Evaluation [135/156]\n", 766 | "INFO:tensorflow:Evaluation [150/156]\n", 767 | "INFO:tensorflow:Evaluation [156/156]\n", 768 | "INFO:tensorflow:Finished evaluation at 2018-10-02-20:32:37\n", 769 | "INFO:tensorflow:Saving dict for global step 4685: accuracy = 0.9672476, average_loss = 0.10643749, global_step = 4685, loss = 6.8119993\n", 770 | "INFO:tensorflow:Saving 'checkpoint_path' summary for global step 4685: /tmp/tmpcj7___ya/model.ckpt-4685\n" 771 | ] 772 | } 773 | ], 774 | "source": [ 775 | "eval_result_1 = estimator.evaluate(input_fn=lambda:myInputFunc_scalar(file_test,nData_test), steps=nStep_test)" 776 | ] 777 | }, 778 | { 779 | "cell_type": "code", 780 | "execution_count": 28, 781 | "metadata": {}, 782 | "outputs": [ 783 | { 784 | "name": "stdout", 785 | "output_type": "stream", 786 | "text": [ 787 | "{'global_step': 4685, 'loss': 6.8119993, 'accuracy': 0.9672476, 'average_loss': 0.10643749}\n" 788 | ] 789 | } 790 | ], 791 | "source": [ 792 | "print(eval_result_1)" 793 | ] 794 | }, 795 | { 796 | "cell_type": "code", 797 | "execution_count": null, 798 | "metadata": {}, 799 | "outputs": [], 800 | "source": [] 801 | }, 802 | { 803 | "cell_type": "markdown", 804 | "metadata": {}, 805 | "source": [ 806 | "### 4. Using an estimator from a Keras model with tf.data from TFRecord files" 807 | ] 808 | }, 809 | { 810 | "cell_type": "markdown", 811 | "metadata": {}, 812 | "source": [ 813 | "Data preprocessing" 814 | ] 815 | }, 816 | { 817 | "cell_type": "code", 818 | "execution_count": 29, 819 | "metadata": {}, 820 | "outputs": [], 821 | "source": [ 822 | "def myMapFunc_onehot(example):\n", 823 | " feature_def = {'X': tf.FixedLenFeature(nDimIn, tf.float32),\n", 824 | " 'Y': tf.FixedLenFeature(1, tf.int64)}\n", 825 | " features = tf.parse_single_example(example, feature_def)\n", 826 | " norm = tf.constant(255, dtype=tf.float32, shape=(nDimIn,))\n", 827 | " xx = tf.div(features['X'], norm)\n", 828 | " yy = tf.reshape(tf.one_hot(features['Y'], nClass, dtype=tf.float32), [-1])\n", 829 | " return xx, yy" 830 | ] 831 | }, 832 | { 833 | "cell_type": "markdown", 834 | "metadata": {}, 835 | "source": [ 836 | "Input function for the estimator" 837 | ] 838 | }, 839 | { 840 | "cell_type": "code", 841 | "execution_count": 30, 842 | "metadata": {}, 843 | "outputs": [], 844 | "source": [ 845 | "def myInputFunc_onehot(filename,numData):\n", 846 | " data_temp = tf.data.TFRecordDataset(filename)\n", 847 | " data_temp = data_temp.shuffle(buffer_size=numData).map(myMapFunc_onehot).batch(batchSize).repeat()\n", 848 | " return data_temp" 849 | ] 850 | }, 851 | { 852 | "cell_type": "markdown", 853 | "metadata": {}, 854 | "source": [ 855 | "Convert a Keras model to an estimator" 856 | ] 857 | }, 858 | { 859 | "cell_type": "code", 860 | "execution_count": 31, 861 | "metadata": {}, 862 | "outputs": [ 863 | { 864 | "name": "stdout", 865 | "output_type": "stream", 866 | "text": [ 867 | "INFO:tensorflow:Using the Keras model provided.\n", 868 | "INFO:tensorflow:Using default config.\n", 869 | "WARNING:tensorflow:Using temporary folder as model directory: /tmp/tmpzezltl9h\n", 870 | "INFO:tensorflow:Using config: {'_keep_checkpoint_max': 5, '_num_ps_replicas': 0, '_eval_distribute': None, '_log_step_count_steps': 100, '_device_fn': None, '_cluster_spec': , '_model_dir': '/tmp/tmpzezltl9h', '_session_config': allow_soft_placement: true\n", 871 | "graph_options {\n", 872 | " rewrite_options {\n", 873 | " meta_optimizer_iterations: ONE\n", 874 | " }\n", 875 | "}\n", 876 | ", '_train_distribute': None, '_protocol': None, '_evaluation_master': '', '_save_checkpoints_steps': None, '_service': None, '_num_worker_replicas': 1, '_save_summary_steps': 100, '_is_chief': True, '_save_checkpoints_secs': 600, '_master': '', '_tf_random_seed': None, '_task_id': 0, '_keep_checkpoint_every_n_hours': 10000, '_task_type': 'worker', '_experimental_distribute': None, '_global_id_in_cluster': 0}\n" 877 | ] 878 | } 879 | ], 880 | "source": [ 881 | "model = myModel()\n", 882 | "estimator = keras.estimator.model_to_estimator(model)" 883 | ] 884 | }, 885 | { 886 | "cell_type": "markdown", 887 | "metadata": {}, 888 | "source": [ 889 | "Training" 890 | ] 891 | }, 892 | { 893 | "cell_type": "code", 894 | "execution_count": 32, 895 | "metadata": { 896 | "scrolled": true 897 | }, 898 | "outputs": [ 899 | { 900 | "name": "stdout", 901 | "output_type": "stream", 902 | "text": [ 903 | "INFO:tensorflow:Calling model_fn.\n", 904 | "INFO:tensorflow:Done calling model_fn.\n", 905 | "INFO:tensorflow:Warm-starting with WarmStartSettings: WarmStartSettings(ckpt_to_initialize_from='/tmp/tmpzezltl9h/keras/keras_model.ckpt', vars_to_warm_start='.*', var_name_to_vocab_info={}, var_name_to_prev_var_name={})\n", 906 | "INFO:tensorflow:Warm-starting from: ('/tmp/tmpzezltl9h/keras/keras_model.ckpt',)\n", 907 | "INFO:tensorflow:Warm-starting variable: fc2/bias; prev_var_name: Unchanged\n", 908 | "INFO:tensorflow:Warm-starting variable: dataOut/bias; prev_var_name: Unchanged\n", 909 | "INFO:tensorflow:Warm-starting variable: dataOut/kernel; prev_var_name: Unchanged\n", 910 | "INFO:tensorflow:Warm-starting variable: fc2/kernel; prev_var_name: Unchanged\n", 911 | "INFO:tensorflow:Warm-starting variable: fc1/bias; prev_var_name: Unchanged\n", 912 | "INFO:tensorflow:Warm-starting variable: fc1/kernel; prev_var_name: Unchanged\n", 913 | "INFO:tensorflow:Create CheckpointSaverHook.\n", 914 | "INFO:tensorflow:Graph was finalized.\n", 915 | "INFO:tensorflow:Running local_init_op.\n", 916 | "INFO:tensorflow:Done running local_init_op.\n", 917 | "INFO:tensorflow:Saving checkpoints for 0 into /tmp/tmpzezltl9h/model.ckpt.\n", 918 | "INFO:tensorflow:loss = 2.343115, step = 0\n", 919 | "INFO:tensorflow:global_step/sec: 175.6\n", 920 | "INFO:tensorflow:loss = 0.38823017, step = 100 (0.571 sec)\n", 921 | "INFO:tensorflow:global_step/sec: 190.585\n", 922 | "INFO:tensorflow:loss = 0.30623454, step = 200 (0.526 sec)\n", 923 | "INFO:tensorflow:global_step/sec: 191.583\n", 924 | "INFO:tensorflow:loss = 0.27017453, step = 300 (0.521 sec)\n", 925 | "INFO:tensorflow:global_step/sec: 191.298\n", 926 | "INFO:tensorflow:loss = 0.44381276, step = 400 (0.523 sec)\n", 927 | "INFO:tensorflow:global_step/sec: 190.423\n", 928 | "INFO:tensorflow:loss = 0.22122486, step = 500 (0.527 sec)\n", 929 | "INFO:tensorflow:global_step/sec: 187.411\n", 930 | "INFO:tensorflow:loss = 0.23458631, step = 600 (0.532 sec)\n", 931 | "INFO:tensorflow:global_step/sec: 188.023\n", 932 | "INFO:tensorflow:loss = 0.2939775, step = 700 (0.532 sec)\n", 933 | "INFO:tensorflow:global_step/sec: 186.395\n", 934 | "INFO:tensorflow:loss = 0.25186622, step = 800 (0.537 sec)\n", 935 | "INFO:tensorflow:global_step/sec: 187.874\n", 936 | "INFO:tensorflow:loss = 0.08934962, step = 900 (0.532 sec)\n", 937 | "INFO:tensorflow:global_step/sec: 142.464\n", 938 | "INFO:tensorflow:loss = 0.1914472, step = 1000 (0.702 sec)\n", 939 | "INFO:tensorflow:global_step/sec: 191.316\n", 940 | "INFO:tensorflow:loss = 0.2799527, step = 1100 (0.523 sec)\n", 941 | "INFO:tensorflow:global_step/sec: 190.346\n", 942 | "INFO:tensorflow:loss = 0.23386763, step = 1200 (0.526 sec)\n", 943 | "INFO:tensorflow:global_step/sec: 191.2\n", 944 | "INFO:tensorflow:loss = 0.13022739, step = 1300 (0.524 sec)\n", 945 | "INFO:tensorflow:global_step/sec: 192.54\n", 946 | "INFO:tensorflow:loss = 0.2688165, step = 1400 (0.519 sec)\n", 947 | "INFO:tensorflow:global_step/sec: 189.973\n", 948 | "INFO:tensorflow:loss = 0.12757486, step = 1500 (0.528 sec)\n", 949 | "INFO:tensorflow:global_step/sec: 191.657\n", 950 | "INFO:tensorflow:loss = 0.10025652, step = 1600 (0.520 sec)\n", 951 | "INFO:tensorflow:global_step/sec: 189.079\n", 952 | "INFO:tensorflow:loss = 0.06178995, step = 1700 (0.529 sec)\n", 953 | "INFO:tensorflow:global_step/sec: 189.969\n", 954 | "INFO:tensorflow:loss = 0.38850045, step = 1800 (0.526 sec)\n", 955 | "INFO:tensorflow:global_step/sec: 118.53\n", 956 | "INFO:tensorflow:loss = 0.12653774, step = 1900 (0.844 sec)\n", 957 | "INFO:tensorflow:global_step/sec: 189.48\n", 958 | "INFO:tensorflow:loss = 0.1997881, step = 2000 (0.528 sec)\n", 959 | "INFO:tensorflow:global_step/sec: 189.424\n", 960 | "INFO:tensorflow:loss = 0.101094395, step = 2100 (0.529 sec)\n", 961 | "INFO:tensorflow:global_step/sec: 188.468\n", 962 | "INFO:tensorflow:loss = 0.08463234, step = 2200 (0.529 sec)\n", 963 | "INFO:tensorflow:global_step/sec: 191.43\n", 964 | "INFO:tensorflow:loss = 0.061152607, step = 2300 (0.522 sec)\n", 965 | "INFO:tensorflow:global_step/sec: 189.557\n", 966 | "INFO:tensorflow:loss = 0.19836038, step = 2400 (0.527 sec)\n", 967 | "INFO:tensorflow:global_step/sec: 188.568\n", 968 | "INFO:tensorflow:loss = 0.18402502, step = 2500 (0.530 sec)\n", 969 | "INFO:tensorflow:global_step/sec: 190.326\n", 970 | "INFO:tensorflow:loss = 0.19622898, step = 2600 (0.525 sec)\n", 971 | "INFO:tensorflow:global_step/sec: 189.639\n", 972 | "INFO:tensorflow:loss = 0.1950176, step = 2700 (0.528 sec)\n", 973 | "INFO:tensorflow:global_step/sec: 191.239\n", 974 | "INFO:tensorflow:loss = 0.14450148, step = 2800 (0.525 sec)\n", 975 | "INFO:tensorflow:global_step/sec: 117.451\n", 976 | "INFO:tensorflow:loss = 0.05950236, step = 2900 (0.850 sec)\n", 977 | "INFO:tensorflow:global_step/sec: 186.849\n", 978 | "INFO:tensorflow:loss = 0.16034009, step = 3000 (0.535 sec)\n", 979 | "INFO:tensorflow:global_step/sec: 187.936\n", 980 | "INFO:tensorflow:loss = 0.10459594, step = 3100 (0.532 sec)\n", 981 | "INFO:tensorflow:global_step/sec: 189.449\n", 982 | "INFO:tensorflow:loss = 0.102305606, step = 3200 (0.528 sec)\n", 983 | "INFO:tensorflow:global_step/sec: 184.032\n", 984 | "INFO:tensorflow:loss = 0.14368361, step = 3300 (0.543 sec)\n", 985 | "INFO:tensorflow:global_step/sec: 187.466\n", 986 | "INFO:tensorflow:loss = 0.059323546, step = 3400 (0.533 sec)\n", 987 | "INFO:tensorflow:global_step/sec: 186.053\n", 988 | "INFO:tensorflow:loss = 0.08452003, step = 3500 (0.538 sec)\n", 989 | "INFO:tensorflow:global_step/sec: 186.462\n", 990 | "INFO:tensorflow:loss = 0.11348133, step = 3600 (0.536 sec)\n", 991 | "INFO:tensorflow:global_step/sec: 185.874\n", 992 | "INFO:tensorflow:loss = 0.13121648, step = 3700 (0.538 sec)\n", 993 | "INFO:tensorflow:global_step/sec: 116.567\n", 994 | "INFO:tensorflow:loss = 0.08568603, step = 3800 (0.858 sec)\n", 995 | "INFO:tensorflow:global_step/sec: 189.5\n", 996 | "INFO:tensorflow:loss = 0.12811038, step = 3900 (0.528 sec)\n", 997 | "INFO:tensorflow:global_step/sec: 188.914\n", 998 | "INFO:tensorflow:loss = 0.12635428, step = 4000 (0.529 sec)\n", 999 | "INFO:tensorflow:global_step/sec: 188.904\n", 1000 | "INFO:tensorflow:loss = 0.048038885, step = 4100 (0.529 sec)\n", 1001 | "INFO:tensorflow:global_step/sec: 193.346\n", 1002 | "INFO:tensorflow:loss = 0.030349664, step = 4200 (0.518 sec)\n", 1003 | "INFO:tensorflow:global_step/sec: 189.306\n", 1004 | "INFO:tensorflow:loss = 0.14236182, step = 4300 (0.528 sec)\n", 1005 | "INFO:tensorflow:global_step/sec: 190.322\n", 1006 | "INFO:tensorflow:loss = 0.038686596, step = 4400 (0.526 sec)\n", 1007 | "INFO:tensorflow:global_step/sec: 187.357\n", 1008 | "INFO:tensorflow:loss = 0.019955378, step = 4500 (0.534 sec)\n", 1009 | "INFO:tensorflow:global_step/sec: 189.702\n", 1010 | "INFO:tensorflow:loss = 0.11344223, step = 4600 (0.527 sec)\n", 1011 | "INFO:tensorflow:Saving checkpoints for 4685 into /tmp/tmpzezltl9h/model.ckpt.\n", 1012 | "INFO:tensorflow:Loss for final step: 0.019161219.\n" 1013 | ] 1014 | }, 1015 | { 1016 | "data": { 1017 | "text/plain": [ 1018 | "" 1019 | ] 1020 | }, 1021 | "execution_count": 32, 1022 | "metadata": {}, 1023 | "output_type": "execute_result" 1024 | } 1025 | ], 1026 | "source": [ 1027 | "estimator.train(input_fn=lambda:myInputFunc_onehot(file_train,nData_train), steps=nStep_train*nEpoch)" 1028 | ] 1029 | }, 1030 | { 1031 | "cell_type": "markdown", 1032 | "metadata": {}, 1033 | "source": [ 1034 | "Test" 1035 | ] 1036 | }, 1037 | { 1038 | "cell_type": "code", 1039 | "execution_count": 33, 1040 | "metadata": { 1041 | "scrolled": true 1042 | }, 1043 | "outputs": [ 1044 | { 1045 | "name": "stdout", 1046 | "output_type": "stream", 1047 | "text": [ 1048 | "INFO:tensorflow:Calling model_fn.\n", 1049 | "INFO:tensorflow:Done calling model_fn.\n", 1050 | "INFO:tensorflow:Starting evaluation at 2018-10-02-20:33:05\n", 1051 | "INFO:tensorflow:Graph was finalized.\n", 1052 | "INFO:tensorflow:Restoring parameters from /tmp/tmpzezltl9h/model.ckpt-4685\n", 1053 | "INFO:tensorflow:Running local_init_op.\n", 1054 | "INFO:tensorflow:Done running local_init_op.\n", 1055 | "INFO:tensorflow:Evaluation [15/156]\n", 1056 | "INFO:tensorflow:Evaluation [30/156]\n", 1057 | "INFO:tensorflow:Evaluation [45/156]\n", 1058 | "INFO:tensorflow:Evaluation [60/156]\n", 1059 | "INFO:tensorflow:Evaluation [75/156]\n", 1060 | "INFO:tensorflow:Evaluation [90/156]\n", 1061 | "INFO:tensorflow:Evaluation [105/156]\n", 1062 | "INFO:tensorflow:Evaluation [120/156]\n", 1063 | "INFO:tensorflow:Evaluation [135/156]\n", 1064 | "INFO:tensorflow:Evaluation [150/156]\n", 1065 | "INFO:tensorflow:Evaluation [156/156]\n", 1066 | "INFO:tensorflow:Finished evaluation at 2018-10-02-20:33:06\n", 1067 | "INFO:tensorflow:Saving dict for global step 4685: accuracy = 0.96684694, global_step = 4685, loss = 0.11055692\n", 1068 | "INFO:tensorflow:Saving 'checkpoint_path' summary for global step 4685: /tmp/tmpzezltl9h/model.ckpt-4685\n" 1069 | ] 1070 | } 1071 | ], 1072 | "source": [ 1073 | "eval_result_2 = estimator.evaluate(input_fn=lambda:myInputFunc_onehot(file_test, nData_test), steps=nStep_test)" 1074 | ] 1075 | }, 1076 | { 1077 | "cell_type": "code", 1078 | "execution_count": 34, 1079 | "metadata": { 1080 | "scrolled": true 1081 | }, 1082 | "outputs": [ 1083 | { 1084 | "name": "stdout", 1085 | "output_type": "stream", 1086 | "text": [ 1087 | "{'global_step': 4685, 'accuracy': 0.96684694, 'loss': 0.11055692}\n" 1088 | ] 1089 | } 1090 | ], 1091 | "source": [ 1092 | "print(eval_result_2)" 1093 | ] 1094 | }, 1095 | { 1096 | "cell_type": "code", 1097 | "execution_count": null, 1098 | "metadata": {}, 1099 | "outputs": [], 1100 | "source": [] 1101 | }, 1102 | { 1103 | "cell_type": "markdown", 1104 | "metadata": {}, 1105 | "source": [ 1106 | "### 5. Accessing data on google cloud (google storage)" 1107 | ] 1108 | }, 1109 | { 1110 | "cell_type": "markdown", 1111 | "metadata": {}, 1112 | "source": [ 1113 | "- Before starting the tutorial, install google cloud storage by ```pip install google-cloud-storage```" 1114 | ] 1115 | }, 1116 | { 1117 | "cell_type": "markdown", 1118 | "metadata": {}, 1119 | "source": [ 1120 | "- [Important] You cannot write or upload a file on google cloud using a VM with the default setting. If you see a permission error (related to 403 POST), go to VM instance details and edit your \"Cloud API access scopes\" on the bottom from \"defalt\" to \"full access\" " 1121 | ] 1122 | }, 1123 | { 1124 | "cell_type": "code", 1125 | "execution_count": 35, 1126 | "metadata": {}, 1127 | "outputs": [], 1128 | "source": [ 1129 | "from google.cloud import storage\n", 1130 | "client = storage.Client()\n", 1131 | "bucketname = 'your-bucket-name'\n", 1132 | "bucket = client.get_bucket(bucketname)" 1133 | ] 1134 | }, 1135 | { 1136 | "cell_type": "markdown", 1137 | "metadata": {}, 1138 | "source": [ 1139 | "Upload files to 'gs://data-push-wearableband/temp/'" 1140 | ] 1141 | }, 1142 | { 1143 | "cell_type": "code", 1144 | "execution_count": 36, 1145 | "metadata": {}, 1146 | "outputs": [], 1147 | "source": [ 1148 | "target_folder = 'sub-folder-name/'" 1149 | ] 1150 | }, 1151 | { 1152 | "cell_type": "code", 1153 | "execution_count": 37, 1154 | "metadata": {}, 1155 | "outputs": [], 1156 | "source": [ 1157 | "blob_train = storage.Blob(target_folder+file_train, bucket) # destination\n", 1158 | "blob_train.upload_from_filename(file_train) # source file\n", 1159 | "blob_test = storage.Blob(target_folder+file_test, bucket) # destination\n", 1160 | "blob_test.upload_from_filename(file_test) # source file" 1161 | ] 1162 | }, 1163 | { 1164 | "cell_type": "markdown", 1165 | "metadata": {}, 1166 | "source": [ 1167 | "Get the list of files in the bucket" 1168 | ] 1169 | }, 1170 | { 1171 | "cell_type": "code", 1172 | "execution_count": 38, 1173 | "metadata": {}, 1174 | "outputs": [ 1175 | { 1176 | "name": "stdout", 1177 | "output_type": "stream", 1178 | "text": [ 1179 | "['temp/', 'temp/mnist_test.tfrecords', 'temp/mnist_train.tfrecords']\n" 1180 | ] 1181 | } 1182 | ], 1183 | "source": [ 1184 | "filelist = []\n", 1185 | "blobs = bucket.list_blobs(prefix=target_folder)\n", 1186 | "for blob in blobs:\n", 1187 | " filelist.append(blob.name) \n", 1188 | "print(filelist) \n", 1189 | "# Note that 'temp/' is included in the file list" 1190 | ] 1191 | }, 1192 | { 1193 | "cell_type": "markdown", 1194 | "metadata": {}, 1195 | "source": [ 1196 | "Train a model using the data on the cloud" 1197 | ] 1198 | }, 1199 | { 1200 | "cell_type": "code", 1201 | "execution_count": 39, 1202 | "metadata": {}, 1203 | "outputs": [ 1204 | { 1205 | "name": "stdout", 1206 | "output_type": "stream", 1207 | "text": [ 1208 | "gs://data-push-wearableband/temp/mnist_train.tfrecords\n", 1209 | "gs://data-push-wearableband/temp/mnist_test.tfrecords\n" 1210 | ] 1211 | } 1212 | ], 1213 | "source": [ 1214 | "fullpath_train = 'gs://'+bucketname+'/'+target_folder+file_train\n", 1215 | "fullpath_test = 'gs://'+bucketname+'/'+target_folder+file_test\n", 1216 | "print(fullpath_train)\n", 1217 | "print(fullpath_test)" 1218 | ] 1219 | }, 1220 | { 1221 | "cell_type": "markdown", 1222 | "metadata": {}, 1223 | "source": [ 1224 | "Convert a Keras model to an estimator" 1225 | ] 1226 | }, 1227 | { 1228 | "cell_type": "markdown", 1229 | "metadata": {}, 1230 | "source": [ 1231 | "Training" 1232 | ] 1233 | }, 1234 | { 1235 | "cell_type": "code", 1236 | "execution_count": 40, 1237 | "metadata": { 1238 | "scrolled": true 1239 | }, 1240 | "outputs": [ 1241 | { 1242 | "name": "stdout", 1243 | "output_type": "stream", 1244 | "text": [ 1245 | "INFO:tensorflow:Calling model_fn.\n", 1246 | "INFO:tensorflow:Done calling model_fn.\n", 1247 | "INFO:tensorflow:Warm-starting with WarmStartSettings: WarmStartSettings(ckpt_to_initialize_from='/tmp/tmpzezltl9h/keras/keras_model.ckpt', vars_to_warm_start='.*', var_name_to_vocab_info={}, var_name_to_prev_var_name={})\n", 1248 | "INFO:tensorflow:Warm-starting from: ('/tmp/tmpzezltl9h/keras/keras_model.ckpt',)\n", 1249 | "INFO:tensorflow:Warm-starting variable: fc2/bias; prev_var_name: Unchanged\n", 1250 | "INFO:tensorflow:Warm-starting variable: dataOut/bias; prev_var_name: Unchanged\n", 1251 | "INFO:tensorflow:Warm-starting variable: dataOut/kernel; prev_var_name: Unchanged\n", 1252 | "INFO:tensorflow:Warm-starting variable: fc2/kernel; prev_var_name: Unchanged\n", 1253 | "INFO:tensorflow:Warm-starting variable: fc1/bias; prev_var_name: Unchanged\n", 1254 | "INFO:tensorflow:Warm-starting variable: fc1/kernel; prev_var_name: Unchanged\n", 1255 | "INFO:tensorflow:Create CheckpointSaverHook.\n", 1256 | "INFO:tensorflow:Graph was finalized.\n", 1257 | "INFO:tensorflow:Restoring parameters from /tmp/tmpzezltl9h/model.ckpt-4685\n", 1258 | "INFO:tensorflow:Running local_init_op.\n", 1259 | "INFO:tensorflow:Done running local_init_op.\n", 1260 | "INFO:tensorflow:Saving checkpoints for 4685 into /tmp/tmpzezltl9h/model.ckpt.\n", 1261 | "INFO:tensorflow:loss = 0.26293677, step = 4685\n", 1262 | "INFO:tensorflow:global_step/sec: 165.869\n", 1263 | "INFO:tensorflow:loss = 0.04528652, step = 4785 (0.604 sec)\n", 1264 | "INFO:tensorflow:global_step/sec: 182.163\n", 1265 | "INFO:tensorflow:loss = 0.16166879, step = 4885 (0.550 sec)\n", 1266 | "INFO:tensorflow:global_step/sec: 181.656\n", 1267 | "INFO:tensorflow:loss = 0.089943334, step = 4985 (0.549 sec)\n", 1268 | "INFO:tensorflow:global_step/sec: 178.386\n", 1269 | "INFO:tensorflow:loss = 0.018729383, step = 5085 (0.561 sec)\n", 1270 | "INFO:tensorflow:global_step/sec: 183.32\n", 1271 | "INFO:tensorflow:loss = 0.1063158, step = 5185 (0.545 sec)\n", 1272 | "INFO:tensorflow:global_step/sec: 186.464\n", 1273 | "INFO:tensorflow:loss = 0.15742005, step = 5285 (0.536 sec)\n", 1274 | "INFO:tensorflow:global_step/sec: 181.458\n", 1275 | "INFO:tensorflow:loss = 0.14449483, step = 5385 (0.551 sec)\n", 1276 | "INFO:tensorflow:global_step/sec: 178.203\n", 1277 | "INFO:tensorflow:loss = 0.03829495, step = 5485 (0.561 sec)\n", 1278 | "INFO:tensorflow:global_step/sec: 183.695\n", 1279 | "INFO:tensorflow:loss = 0.13704526, step = 5585 (0.544 sec)\n", 1280 | "INFO:tensorflow:global_step/sec: 149.15\n", 1281 | "INFO:tensorflow:loss = 0.037466798, step = 5685 (0.671 sec)\n", 1282 | "INFO:tensorflow:global_step/sec: 188.641\n", 1283 | "INFO:tensorflow:loss = 0.015689494, step = 5785 (0.530 sec)\n", 1284 | "INFO:tensorflow:global_step/sec: 183.498\n", 1285 | "INFO:tensorflow:loss = 0.077800296, step = 5885 (0.545 sec)\n", 1286 | "INFO:tensorflow:global_step/sec: 188.772\n", 1287 | "INFO:tensorflow:loss = 0.08141988, step = 5985 (0.530 sec)\n", 1288 | "INFO:tensorflow:global_step/sec: 184.706\n", 1289 | "INFO:tensorflow:loss = 0.09549804, step = 6085 (0.541 sec)\n", 1290 | "INFO:tensorflow:global_step/sec: 187.783\n", 1291 | "INFO:tensorflow:loss = 0.10685474, step = 6185 (0.533 sec)\n", 1292 | "INFO:tensorflow:global_step/sec: 181.346\n", 1293 | "INFO:tensorflow:loss = 0.18335311, step = 6285 (0.551 sec)\n", 1294 | "INFO:tensorflow:global_step/sec: 181.126\n", 1295 | "INFO:tensorflow:loss = 0.0596697, step = 6385 (0.552 sec)\n", 1296 | "INFO:tensorflow:global_step/sec: 183.721\n", 1297 | "INFO:tensorflow:loss = 0.041592922, step = 6485 (0.544 sec)\n", 1298 | "INFO:tensorflow:global_step/sec: 114.153\n", 1299 | "INFO:tensorflow:loss = 0.03766963, step = 6585 (0.876 sec)\n", 1300 | "INFO:tensorflow:global_step/sec: 184.532\n", 1301 | "INFO:tensorflow:loss = 0.07494867, step = 6685 (0.542 sec)\n", 1302 | "INFO:tensorflow:global_step/sec: 182.859\n", 1303 | "INFO:tensorflow:loss = 0.047436465, step = 6785 (0.547 sec)\n", 1304 | "INFO:tensorflow:global_step/sec: 183.785\n", 1305 | "INFO:tensorflow:loss = 0.10758423, step = 6885 (0.544 sec)\n", 1306 | "INFO:tensorflow:global_step/sec: 186.557\n", 1307 | "INFO:tensorflow:loss = 0.09009351, step = 6985 (0.536 sec)\n", 1308 | "INFO:tensorflow:global_step/sec: 184.325\n", 1309 | "INFO:tensorflow:loss = 0.058775768, step = 7085 (0.542 sec)\n", 1310 | "INFO:tensorflow:global_step/sec: 186.574\n", 1311 | "INFO:tensorflow:loss = 0.07727949, step = 7185 (0.536 sec)\n", 1312 | "INFO:tensorflow:global_step/sec: 184.023\n", 1313 | "INFO:tensorflow:loss = 0.028509261, step = 7285 (0.543 sec)\n", 1314 | "INFO:tensorflow:global_step/sec: 184.929\n", 1315 | "INFO:tensorflow:loss = 0.0125967655, step = 7385 (0.541 sec)\n", 1316 | "INFO:tensorflow:global_step/sec: 185.106\n", 1317 | "INFO:tensorflow:loss = 0.071365595, step = 7485 (0.540 sec)\n", 1318 | "INFO:tensorflow:global_step/sec: 115.512\n", 1319 | "INFO:tensorflow:loss = 0.062833846, step = 7585 (0.866 sec)\n", 1320 | "INFO:tensorflow:global_step/sec: 188.068\n", 1321 | "INFO:tensorflow:loss = 0.07584804, step = 7685 (0.532 sec)\n", 1322 | "INFO:tensorflow:global_step/sec: 186.244\n", 1323 | "INFO:tensorflow:loss = 0.16068706, step = 7785 (0.537 sec)\n", 1324 | "INFO:tensorflow:global_step/sec: 187.305\n", 1325 | "INFO:tensorflow:loss = 0.05271759, step = 7885 (0.534 sec)\n", 1326 | "INFO:tensorflow:global_step/sec: 185.155\n", 1327 | "INFO:tensorflow:loss = 0.069579884, step = 7985 (0.540 sec)\n", 1328 | "INFO:tensorflow:global_step/sec: 186.201\n", 1329 | "INFO:tensorflow:loss = 0.07663117, step = 8085 (0.537 sec)\n", 1330 | "INFO:tensorflow:global_step/sec: 185.013\n", 1331 | "INFO:tensorflow:loss = 0.03538336, step = 8185 (0.540 sec)\n", 1332 | "INFO:tensorflow:global_step/sec: 185.323\n", 1333 | "INFO:tensorflow:loss = 0.025143351, step = 8285 (0.540 sec)\n", 1334 | "INFO:tensorflow:global_step/sec: 183.716\n", 1335 | "INFO:tensorflow:loss = 0.10386059, step = 8385 (0.544 sec)\n", 1336 | "INFO:tensorflow:global_step/sec: 110.331\n", 1337 | "INFO:tensorflow:loss = 0.03639785, step = 8485 (0.907 sec)\n", 1338 | "INFO:tensorflow:global_step/sec: 162.378\n", 1339 | "INFO:tensorflow:loss = 0.025539573, step = 8585 (0.616 sec)\n", 1340 | "INFO:tensorflow:global_step/sec: 161.973\n", 1341 | "INFO:tensorflow:loss = 0.11529465, step = 8685 (0.617 sec)\n", 1342 | "INFO:tensorflow:global_step/sec: 162.505\n", 1343 | "INFO:tensorflow:loss = 0.024646323, step = 8785 (0.615 sec)\n", 1344 | "INFO:tensorflow:global_step/sec: 162.294\n", 1345 | "INFO:tensorflow:loss = 0.017728638, step = 8885 (0.616 sec)\n", 1346 | "INFO:tensorflow:global_step/sec: 161.006\n", 1347 | "INFO:tensorflow:loss = 0.05596263, step = 8985 (0.621 sec)\n", 1348 | "INFO:tensorflow:global_step/sec: 161.408\n", 1349 | "INFO:tensorflow:loss = 0.026135616, step = 9085 (0.620 sec)\n", 1350 | "INFO:tensorflow:global_step/sec: 161.403\n", 1351 | "INFO:tensorflow:loss = 0.06356065, step = 9185 (0.619 sec)\n", 1352 | "INFO:tensorflow:global_step/sec: 161.996\n", 1353 | "INFO:tensorflow:loss = 0.014010385, step = 9285 (0.617 sec)\n", 1354 | "INFO:tensorflow:Saving checkpoints for 9370 into /tmp/tmpzezltl9h/model.ckpt.\n", 1355 | "INFO:tensorflow:Loss for final step: 0.053213447.\n" 1356 | ] 1357 | }, 1358 | { 1359 | "data": { 1360 | "text/plain": [ 1361 | "" 1362 | ] 1363 | }, 1364 | "execution_count": 40, 1365 | "metadata": {}, 1366 | "output_type": "execute_result" 1367 | } 1368 | ], 1369 | "source": [ 1370 | "estimator.train(input_fn=lambda:myInputFunc_onehot(file_train, nData_train), steps=nStep_train*nEpoch)" 1371 | ] 1372 | }, 1373 | { 1374 | "cell_type": "markdown", 1375 | "metadata": {}, 1376 | "source": [ 1377 | "Test" 1378 | ] 1379 | }, 1380 | { 1381 | "cell_type": "code", 1382 | "execution_count": 41, 1383 | "metadata": {}, 1384 | "outputs": [ 1385 | { 1386 | "name": "stdout", 1387 | "output_type": "stream", 1388 | "text": [ 1389 | "INFO:tensorflow:Calling model_fn.\n", 1390 | "INFO:tensorflow:Done calling model_fn.\n", 1391 | "INFO:tensorflow:Starting evaluation at 2018-10-02-20:33:38\n", 1392 | "INFO:tensorflow:Graph was finalized.\n", 1393 | "INFO:tensorflow:Restoring parameters from /tmp/tmpzezltl9h/model.ckpt-9370\n", 1394 | "INFO:tensorflow:Running local_init_op.\n", 1395 | "INFO:tensorflow:Done running local_init_op.\n", 1396 | "INFO:tensorflow:Evaluation [15/156]\n", 1397 | "INFO:tensorflow:Evaluation [30/156]\n", 1398 | "INFO:tensorflow:Evaluation [45/156]\n", 1399 | "INFO:tensorflow:Evaluation [60/156]\n", 1400 | "INFO:tensorflow:Evaluation [75/156]\n", 1401 | "INFO:tensorflow:Evaluation [90/156]\n", 1402 | "INFO:tensorflow:Evaluation [105/156]\n", 1403 | "INFO:tensorflow:Evaluation [120/156]\n", 1404 | "INFO:tensorflow:Evaluation [135/156]\n", 1405 | "INFO:tensorflow:Evaluation [150/156]\n", 1406 | "INFO:tensorflow:Evaluation [156/156]\n", 1407 | "INFO:tensorflow:Finished evaluation at 2018-10-02-20:33:40\n", 1408 | "INFO:tensorflow:Saving dict for global step 9370: accuracy = 0.96935093, global_step = 9370, loss = 0.10938837\n", 1409 | "INFO:tensorflow:Saving 'checkpoint_path' summary for global step 9370: /tmp/tmpzezltl9h/model.ckpt-9370\n" 1410 | ] 1411 | } 1412 | ], 1413 | "source": [ 1414 | "eval_result_3 = estimator.evaluate(input_fn=lambda:myInputFunc_onehot(fullpath_test, nData_test), steps=nStep_test)" 1415 | ] 1416 | }, 1417 | { 1418 | "cell_type": "code", 1419 | "execution_count": 42, 1420 | "metadata": { 1421 | "scrolled": true 1422 | }, 1423 | "outputs": [ 1424 | { 1425 | "name": "stdout", 1426 | "output_type": "stream", 1427 | "text": [ 1428 | "{'global_step': 9370, 'accuracy': 0.96935093, 'loss': 0.10938837}\n" 1429 | ] 1430 | } 1431 | ], 1432 | "source": [ 1433 | "print(eval_result_3)" 1434 | ] 1435 | } 1436 | ], 1437 | "metadata": { 1438 | "kernelspec": { 1439 | "display_name": "Python 3", 1440 | "language": "python", 1441 | "name": "python3" 1442 | }, 1443 | "language_info": { 1444 | "codemirror_mode": { 1445 | "name": "ipython", 1446 | "version": 3 1447 | }, 1448 | "file_extension": ".py", 1449 | "mimetype": "text/x-python", 1450 | "name": "python", 1451 | "nbconvert_exporter": "python", 1452 | "pygments_lexer": "ipython3", 1453 | "version": "3.5.3" 1454 | } 1455 | }, 1456 | "nbformat": 4, 1457 | "nbformat_minor": 2 1458 | } 1459 | --------------------------------------------------------------------------------