├── README.md ├── basic_op.py ├── linear_regression.py ├── logistic_regression.py ├── murder_rates_data.csv ├── output_linear_regression.PNG └── output_logistic_regression.PNG /README.md: -------------------------------------------------------------------------------- 1 | # Introduction to TensorFlow Basics 2 | 3 | The following files give a brief 101 intro to TensorFlow. These files were created together for my Introduction to TensorFlow lecture [VIDEO](https://youtu.be/I9vz34sY16I) | [SLIDES](https://drive.google.com/open?id=0B0UHx_6DZC6_U2NLTnhnZkZ0YUU) 4 | 5 | ## basic_op.py 6 | This file defines some basic operations, setting of constants, variables and operator nodes. It also introduces building a simple model and running a TensorFlow session to run the model. 7 | 8 | ## linear_regression.py 9 | Here we make a simple linear regression using the murder_rates_data.csv dataset to plot training and testing data on a linear regression. We also add TensorBoard summaries to the code which can then be run using `tensorboard --logdir="/logs"` 10 | ![Output of Linear Regression](https://raw.githubusercontent.com/jaungiers/TensorFlow-Intro/master/output_linear_regression.PNG "Output of Linear Regression") 11 | 12 | 13 | ## logistic_regression.py 14 | Here we expand on the linear regression model and build a logisitic (softmax) regression for the MNIST dataset. The output also includes a nice visual output from two test cases using matplotlib 15 | 16 | ![Output of Logistic Regression](https://raw.githubusercontent.com/jaungiers/TensorFlow-Intro/master/output_logistic_regression.PNG "Output of Logistic Regression") -------------------------------------------------------------------------------- /basic_op.py: -------------------------------------------------------------------------------- 1 | #print() is only available in python 3+ so we explicitly include print function 2 | from __future__ import print_function 3 | 4 | import tensorflow as tf 5 | import os 6 | 7 | # Hide TensorFlows warning messages 8 | os.environ['TF_CPP_MIN_LOG_LEVEL'] = '3' 9 | 10 | #Use constant function to define constanst node in tensorflow computation graph 11 | a = tf.constant(1) 12 | b = tf.constant(2) 13 | 14 | # Launch the default graph using Session() funuction. 15 | with tf.Session() as sess: 16 | print("a=1, b=2") 17 | print("Addition: %i" % sess.run(a+b)) 18 | print("Multiplication: %i" % sess.run(a*b)) 19 | 20 | 21 | # Basic Operations with variable as graph input 22 | # The value returned by the constructor represents the output 23 | # of the Variable op. (define as input when running session) 24 | # tf Graph input 25 | a = tf.placeholder(tf.int16) 26 | b = tf.placeholder(tf.int16) 27 | 28 | # Define some operations 29 | add = tf.add(a, b) 30 | mul = tf.multiply(a, b) 31 | 32 | # Launch the default graph. 33 | with tf.Session() as sess: 34 | # Run every operation with variable input 35 | print("Addition with variables: %i" % sess.run(add, feed_dict={a: 4, b: 8})) 36 | print("Multiplication with variables: %i" % sess.run(mul, feed_dict={a: 4, b: 8})) 37 | 38 | # Matrix multiplaction 39 | 40 | 41 | # Create a Constant op that produces a 1x2 matrix. The op is 42 | # added as a node to the default graph. 43 | # 44 | # The value returned by the constructor represents the output 45 | # of the Constant op. 46 | #Create matrix with 1x2 dim 47 | matrix1 = tf.constant([[1., 2.]]) 48 | 49 | # create matrix with 2x1 dim 50 | matrix2 = tf.constant([[3.],[4.]]) 51 | 52 | #create node "product" for result 53 | product = tf.matmul(matrix1, matrix2) 54 | 55 | # to compute the resultwe need to run the graph. 56 | # for that we create object of session and run it 57 | with tf.Session() as sess: 58 | result = sess.run(product) 59 | print(result) 60 | -------------------------------------------------------------------------------- /linear_regression.py: -------------------------------------------------------------------------------- 1 | from __future__ import print_function 2 | 3 | import tensorflow as tf 4 | import numpy as np 5 | import matplotlib.pyplot as plt 6 | import os 7 | 8 | # Hide TensorFlows warning messages 9 | os.environ['TF_CPP_MIN_LOG_LEVEL'] = '3' 10 | 11 | rng = np.random 12 | 13 | # Different parameters for learning 14 | learning_rate = 0.01 15 | training_epochs = 1000 16 | display_step = 100 17 | 18 | #Population, Percent with <$5000 income, Percent Unemployed, Murders per annum per 1mil population 19 | data = np.genfromtxt('murder_rates_data.csv', delimiter=',', skip_header=1) 20 | 21 | # Training Data 22 | train_test_split = int(len(data)*0.7) #70% training : 30% testing 23 | train_X = data[:, 2][:train_test_split] #Percent unemployed 24 | train_Y = data[:, 3][:train_test_split] #Murders per 1 million population per year 25 | n_samples = train_X.shape[0] 26 | 27 | # Create placeholder for providing inputs 28 | X = tf.placeholder("float") 29 | Y = tf.placeholder("float") 30 | 31 | # create weights and bias and initialize with random number 32 | W = tf.Variable(rng.randn(), name="weight") 33 | b = tf.Variable(rng.randn(), name="bias") 34 | 35 | with tf.name_scope('WX_b') as scope: 36 | # Construct a linear model using Y=WX+b 37 | pred = tf.add(tf.multiply(X, W), b) 38 | 39 | w_h = tf.summary.histogram('weights', W) 40 | b_h = tf.summary.histogram('biases', b) 41 | 42 | with tf.name_scope('cost_function') as scope: 43 | # Calculate Mean squared error 44 | cost = tf.reduce_sum(tf.pow(pred-Y, 2))/(2*n_samples) 45 | tf.summary.scalar('cost_function', cost) 46 | 47 | with tf.name_scope('train') as scope: 48 | # Gradient descent to minimize mean sequare error 49 | optimizer = tf.train.GradientDescentOptimizer(learning_rate).minimize(cost) 50 | 51 | # Initializing the variables 52 | init = tf.global_variables_initializer() 53 | 54 | marge_summary_op = tf.summary.merge_all() 55 | 56 | # Launch the graph 57 | with tf.Session() as sess: 58 | sess.run(init) 59 | 60 | summary_writer = tf.summary.FileWriter('/logs', graph_def=sess.graph_def) 61 | 62 | print(">>> Training started") 63 | 64 | # Fit all training data 65 | for epoch in range(training_epochs): 66 | for (x, y) in zip(train_X, train_Y): 67 | #create small batch of trining and testing data and feed it to model 68 | sess.run(optimizer, feed_dict={X: x, Y: y}) 69 | 70 | # Display training information after each N step 71 | if (epoch+1) % display_step == 0: 72 | c = sess.run(cost, feed_dict={X: train_X, Y:train_Y}) 73 | 74 | print(">>> Epoch:", '%04d' % (epoch+1), "cost=", "{:.9f}".format(c), "W=", sess.run(W), "b=", sess.run(b)) 75 | summary_str = sess.run(marge_summary_op, feed_dict={X: train_X, Y:train_Y}) 76 | summary_writer.add_summary(summary_str, epoch) 77 | 78 | print(">>> Training completed") 79 | training_cost = sess.run(cost, feed_dict={X: train_X, Y: train_Y}) 80 | print(">>> Training cost=", training_cost, "W=", sess.run(W), "b=", sess.run(b), '\n') 81 | 82 | # Testing 83 | print(">>> Testing started") 84 | test_X = data[:, 2][train_test_split:] #Percent unemployed 85 | test_Y = data[:, 3][train_test_split:] #Murders per 1 million population per year 86 | 87 | #Calculate Mean square error 88 | print(">>> Calculate Mean square error") 89 | testing_cost = sess.run( 90 | tf.reduce_sum(tf.pow(pred - Y, 2)) / (2 * test_X.shape[0]), 91 | feed_dict={X: test_X, Y: test_Y} 92 | ) #same function as cost above 93 | 94 | print(">>> Testing cost=", testing_cost) 95 | print(">>> Absolute mean square loss difference:", abs(training_cost - testing_cost)) 96 | 97 | fig = plt.figure(1) 98 | plt_train = fig.add_subplot(2,1,1) 99 | plt_test = fig.add_subplot(2,1,2, sharex=plt_train, sharey=plt_train) 100 | 101 | plt_train.plot(train_X, train_Y, 'ro', label='Original data') 102 | plt_train.plot(train_X, sess.run(W) * train_X + sess.run(b), label='Fitted line') 103 | plt_train.legend() 104 | 105 | plt_test.plot(test_X, test_Y, 'bo', label='Testing data') 106 | plt_test.plot(train_X, sess.run(W) * train_X + sess.run(b), label='Fitted line') 107 | plt_test.legend() 108 | 109 | plt.show() -------------------------------------------------------------------------------- /logistic_regression.py: -------------------------------------------------------------------------------- 1 | from __future__ import print_function 2 | import numpy as np 3 | import matplotlib.pyplot as plt 4 | import tensorflow as tf 5 | import os 6 | 7 | # Hide TensorFlows warning messages 8 | os.environ['TF_CPP_MIN_LOG_LEVEL'] = '3' 9 | 10 | # Import MNIST data 11 | from tensorflow.examples.tutorials.mnist import input_data 12 | mnist = input_data.read_data_sets("MNIST_data/", one_hot=True) 13 | 14 | # Parameters 15 | learning_rate = 0.01 16 | training_epochs = 25 17 | batch_size = 100 18 | display_step = 1 19 | 20 | # tf Graph Input 21 | x = tf.placeholder(tf.float32, [None, 784]) # mnist data image of shape 28*28=784 22 | y = tf.placeholder(tf.float32, [None, 10]) # 0-9 digits recognition => 10 classes 23 | 24 | # Set model weights 25 | W = tf.Variable(tf.zeros([784, 10])) 26 | b = tf.Variable(tf.zeros([10])) 27 | 28 | # Construct model 29 | pred = tf.nn.softmax(tf.matmul(x, W) + b) # Softmax for output probablity 30 | 31 | # Minimize error using cross entropy 32 | cost = tf.reduce_mean(-tf.reduce_sum(y*tf.log(pred), reduction_indices=1)) 33 | # Gradient Descent 34 | optimizer = tf.train.GradientDescentOptimizer(learning_rate).minimize(cost) 35 | 36 | # Initializing the variables 37 | init = tf.global_variables_initializer() 38 | 39 | # Launch the graph 40 | with tf.Session() as sess: 41 | sess.run(init) 42 | 43 | # Training cycle 44 | for epoch in range(training_epochs): 45 | avg_cost = 0. 46 | total_batch = int(mnist.train.num_examples/batch_size) 47 | # Loop over all batches 48 | for i in range(total_batch): 49 | batch_xs, batch_ys = mnist.train.next_batch(batch_size) 50 | # Run optimization op and cost op (to get loss value) 51 | _, c = sess.run([optimizer, cost], feed_dict={x: batch_xs, 52 | y: batch_ys}) 53 | # Compute average loss 54 | avg_cost += c / total_batch 55 | # Display logs per epoch step 56 | if (epoch+1) % display_step == 0: 57 | print(">>> Epoch:", '%04d' % (epoch+1), "cost=", "{:.9f}".format(avg_cost)) 58 | 59 | print(">>> Optimization Finished!") 60 | 61 | # Test model 62 | correct_prediction = tf.equal(tf.argmax(pred, 1), tf.argmax(y, 1)) 63 | # Calculate accuracy 64 | accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32)) 65 | print(">>> Accuracy:", accuracy.eval({x: mnist.test.images, y: mnist.test.labels})) 66 | 67 | # For fun show a few visual test cases 68 | f, ((ax1, ax2), (ax3, ax4)) = plt.subplots(2, 2) 69 | 70 | test1_index = 0 71 | test1_x = mnist.test.images[test1_index].reshape(1, 784) 72 | test1_img = mnist.test.images[test1_index].reshape((28,28)) 73 | test1_y = mnist.test.labels[test1_index].reshape(1, 10) 74 | test1_pred = sess.run(pred, feed_dict={x: test1_x, y: test1_y}) 75 | 76 | ax1.imshow(test1_img, cmap='gray') 77 | ax2.bar(list(range(0,10)), test1_pred[0]) 78 | 79 | test2_index = 1 80 | test2_x = mnist.test.images[test2_index].reshape(1, 784) 81 | test2_img = mnist.test.images[test2_index].reshape((28,28)) 82 | test2_y = mnist.test.labels[test2_index].reshape(1, 10) 83 | test2_pred = sess.run(pred, feed_dict={x: test2_x, y: test2_y}) 84 | 85 | ax3.imshow(test2_img, cmap='gray') 86 | ax4.bar(list(range(0,10)), test2_pred[0]) 87 | 88 | plt.show() -------------------------------------------------------------------------------- /murder_rates_data.csv: -------------------------------------------------------------------------------- 1 | Population,Percent with <$5000 income,Percent Unemployed,Murders per annum per 1mil population, 2 | 587000,16.5,6.2,11.2, 3 | 643000,20.5,6.4,13.4, 4 | 635000,26.3,9.3,40.7, 5 | 692000,16.5,5.3,5.3, 6 | 1248000,19.2,7.3,24.8, 7 | 643000,16.5,5.9,12.7, 8 | 1964000,20.2,6.4,20.9, 9 | 1531000,21.3,7.6,35.7, 10 | 713000,17.2,4.9,8.7, 11 | 749000,14.3,6.4,9.6, 12 | 7895000,18.1,6,14.5, 13 | 762000,23.1,7.4,26.9, 14 | 2793000,19.1,5.8,15.7, 15 | 741000,24.7,8.6,36.2, 16 | 625000,18.6,6.5,18.1, 17 | 854000,24.9,8.3,28.9, 18 | 716000,17.9,6.7,14.9, 19 | 921000,22.4,8.6,25.8, 20 | 595000,20.2,8.4,21.7, 21 | 3353000,16.9,6.7,25.7, 22 | -------------------------------------------------------------------------------- /output_linear_regression.PNG: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jaungiers/TensorFlow-Intro/184da76e884109ec26009d52a4476d5a3a82ef43/output_linear_regression.PNG -------------------------------------------------------------------------------- /output_logistic_regression.PNG: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/jaungiers/TensorFlow-Intro/184da76e884109ec26009d52a4476d5a3a82ef43/output_logistic_regression.PNG --------------------------------------------------------------------------------