├── README.md ├── download_data.py └── main.py /README.md: -------------------------------------------------------------------------------- 1 | 使用最基础的识别手写字体的案例,建立一个简单的神经网络，让大家了解如何使用Tensorboard。 2 | ## Requirements 3 | - Python 4 | - [Tensorflow](https://github.com/tensorflow/tensorflow) 5 | ## Usage 6 | **Step 1.** Download this repository with git or click the [download ZIP ](https://github.com/guoswang/TensorBoard/archive/master.zip)button 7 | 8 | ``` 9 | $ git clone https://github.com/guoswang/TensorBoard.git 10 | $ cd TensorBoard 11 | ``` 12 | **Step 2.** Download [MNIST](http://yann.lecun.com/exdb/mnist/) dataset. In this step, you have two choices: 13 | - a) Automatic downloading with ==download_data.py== script 14 | 15 | ``` 16 | $ python download_data.py 17 | ``` 18 | - b) Manual downloading with wget or other tools, move and extract dataset into ==data/mnist== directory, for example: 19 | 20 | ``` 21 | $ mkdir -p data/mnist 22 | $ wget -c -P data/mnist http://yann.lecun.com/exdb/mnist/train-images-idx3-ubyte.gz 23 | $ wget -c -P data/mnist http://yann.lecun.com/exdb/mnist/train-labels-idx1-ubyte.gz 24 | $ wget -c -P data/mnist http://yann.lecun.com/exdb/mnist/t10k-images-idx3-ubyte.gz 25 | $ wget -c -P data/mnist http://yann.lecun.com/exdb/mnist/t10k-labels-idx1-ubyte.gz 26 | $ gunzip data/mnist/*.gz 27 | ``` 28 | **Step 3.** Start the training and testing(Using the MNIST dataset by default): 29 | 30 | ``` 31 | $ python main.py 32 | $ tensorboard --logdir='log' 33 | ``` 34 | **Step 4.** You will get a website after Step 3. You can navigate to the website. 35 | ## Results 36 | 1. SCALARS 37 | 展示的是标量的信息，程序中用tf.summary.scalars()定义的信息都会在这个窗口。程序中定义的标量有：准确率accuracy,dropout的保留率，隐藏层中的参数信息，已经交叉熵损失。这些都在SCLARS窗口下显示出来了。 38 | 点开accuracy,红线表示test集的结果，蓝线表示train集的结果，可以看到随着循环次数的增加，两者的准确度也在通趋势增加 39 | ![这里写图片描述](http://img.blog.csdn.net/20180122100335577) 40 | 点开dropout，红线表示的测试集上的保留率始终是1，蓝线始终是0.9 41 | ![这里写图片描述](http://img.blog.csdn.net/20180122100613016) 42 | 点开layer1，查看第一个隐藏层的参数信息。 43 | ![这里写图片描述](http://img.blog.csdn.net/20180122100802424) 44 | 从中我们可以看到偏执项b的信息，随着迭代的加深，最大值越来越大，最小值越来越小，与此同时，也伴随着方差越来越大，这样的情况是我们愿意看到的，神经元之间的参数差异越来越大。因为理想的情况下每个神经元都应该去关注不同的特征，所以他们的参数也应有所不同。 45 | 权值w的信息，同理，最大值，最小值，标准差也都有与b相同的趋势，神经元之间的差异越来越明显。w的均值初始化的时候是0，随着迭代其绝对值也越来越大。 46 | 47 | 点开layer2 48 | ![这里写图片描述](http://img.blog.csdn.net/20180122100943794) 49 | 点开loss，可见损失的降低趋势。 50 | ![这里写图片描述](http://img.blog.csdn.net/20180122101408907) 51 | 2. Images记录图像数据。在程序中我们设置了一处保存了图像信息，就是在转变了输入特征的shape，然后记录到了image中，于是在tensorflow中就会还原出原始的图片了： 52 | ![这里写图片描述](http://img.blog.csdn.net/20180122103113002) 53 | 3. Graphs 这里展示的是整个训练过程的计算图graph,从中我们可以清洗地看到整个程序的逻辑与过程。 54 | ![这里写图片描述](http://img.blog.csdn.net/20180122103012692) 55 | 单击某个节点，可以查看属性，输入，输出等信息 56 | ![这里写图片描述](http://img.blog.csdn.net/20180122103516703) 57 | 单击节点上的“+”字样，可以看到该节点的内部信息。 58 | ![这里写图片描述](http://img.blog.csdn.net/20180122104955559) 59 | 另外还可以选择图像颜色的两者模型，基于结构的模式，相同的节点会有同样的颜色，基于预算硬件的，同一个硬件上的会有相同颜色。 60 | ![这里写图片描述](http://img.blog.csdn.net/20180122103810888) 61 | 4. DISTRIBUTIONS 62 | 这里查看的是神经元输出的分布，有激活函数之前的分布，激活函数之后的分布等。 63 | ![这里写图片描述](http://img.blog.csdn.net/20180122104002479) 64 | 5. HISTOGRAMS 65 | 也可以看以上数据的直方图 66 | ![这里写图片描述](http://img.blog.csdn.net/20180122104807419) 67 | 6. EMBEDDINGS 68 | 展示的是嵌入向量的可视化效果，本案例中没有使用这个功能。之后其他案例中再详述。 69 | 7. AUDIO 70 | 这里展示的是声音的信息，但本案例中没有涉及到声音的。 71 | ## More information in faiculty: 72 | ![这里写图片描述](http://img.blog.csdn.net/20180122102309282) 73 | -------------------------------------------------------------------------------- /download_data.py: -------------------------------------------------------------------------------- 1 | import os 2 | import sys 3 | import gzip 4 | import shutil 5 | from six.moves import urllib 6 | 7 | # mnist dataset 8 | HOMEPAGE = "http://yann.lecun.com/exdb/mnist/" 9 | MNIST_TRAIN_IMGS_URL = HOMEPAGE + "train-images-idx3-ubyte.gz" 10 | MNIST_TRAIN_LABELS_URL = HOMEPAGE + "train-labels-idx1-ubyte.gz" 11 | MNIST_TEST_IMGS_URL = HOMEPAGE + "t10k-images-idx3-ubyte.gz" 12 | MNIST_TEST_LABELS_URL = HOMEPAGE + "t10k-labels-idx1-ubyte.gz" 13 | 14 | 15 | def download_and_uncompress_zip(URL, dataset_dir, force=False): 16 | ''' 17 | Args: 18 | URL: the download links for data 19 | dataset_dir: the path to save data 20 | force: re-download data 21 | ''' 22 | filename = URL.split('/')[-1] 23 | filepath = os.path.join(dataset_dir, filename) 24 | if not os.path.exists(dataset_dir): 25 | os.mkdir(dataset_dir) 26 | extract_to = os.path.splitext(filepath)[0] 27 | 28 | def download_progress(count, block_size, total_size): 29 | sys.stdout.write("\r>> Downloading %s %.1f%%" % (filename, float(count * block_size) / float(total_size) * 100.)) 30 | sys.stdout.flush() 31 | 32 | if not force and os.path.exists(filepath): 33 | print("file %s already exist" % (filename)) 34 | else: 35 | filepath, _ = urllib.request.urlretrieve(URL, filepath, download_progress) 36 | print() 37 | print('Successfully Downloaded', filename) 38 | 39 | # with zipfile.ZipFile(filepath) as fd: 40 | with gzip.open(filepath, 'rb') as f_in, open(extract_to, 'wb') as f_out: 41 | print('Extracting ', filename) 42 | shutil.copyfileobj(f_in, f_out) 43 | print('Successfully extracted') 44 | print() 45 | 46 | 47 | def start_download(dataset, save_to, force): 48 | if not os.path.exists(save_to): 49 | os.makedirs(save_to) 50 | if dataset == 'mnist': 51 | download_and_uncompress_zip(MNIST_TRAIN_IMGS_URL, save_to, force) 52 | download_and_uncompress_zip(MNIST_TRAIN_LABELS_URL, save_to, force) 53 | download_and_uncompress_zip(MNIST_TEST_IMGS_URL, save_to, force) 54 | download_and_uncompress_zip(MNIST_TEST_LABELS_URL, save_to, force) 55 | else: 56 | raise Exception("Invalid dataset name! please check it: ", dataset) 57 | 58 | if __name__ == '__main__': 59 | import argparse 60 | parser = argparse.ArgumentParser('Script for automatically downloading datasets') 61 | parser.add_argument("--dataset", default='mnist', choices=['mnist']) 62 | save_to = os.path.join('data', 'mnist') 63 | parser.add_argument("--save_to", default=save_to) 64 | parser.add_argument("--force", default=False, type=bool) 65 | args = parser.parse_args() 66 | start_download(args.dataset, args.save_to, args.force) 67 | -------------------------------------------------------------------------------- /main.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | # -*- coding: utf-8 -*- 3 | from __future__ import absolute_import 4 | from __future__ import division 5 | from __future__ import print_function 6 | 7 | import argparse 8 | import sys 9 | import os 10 | import tensorflow as tf 11 | 12 | from tensorflow.examples.tutorials.mnist import input_data 13 | max_steps = 1000 # 最大迭代次数 14 | learning_rate = 0.001 # 学习率 15 | dropout = 0.9 # dropout时随机保留神经元的比例 16 | 17 | data_dir = os.path.join('data', 'mnist')# 样本数据存储的路径 18 | if not os.path.exists('log'): 19 | os.mkdir('log') 20 | log_dir = 'log' # 输出日志保存的路径 21 | mnist = input_data.read_data_sets(data_dir,one_hot=True) 22 | sess = tf.InteractiveSession() 23 | 24 | with tf.name_scope('input'): 25 | x = tf.placeholder(tf.float32, [None, 784], name='x-input') 26 | y_ = tf.placeholder(tf.float32, [None, 10], name='y-input') 27 | 28 | with tf.name_scope('input_reshape'): 29 | image_shaped_input = tf.reshape(x, [-1, 28, 28, 1]) 30 | tf.summary.image('input', image_shaped_input, 10) 31 | 32 | def weight_variable(shape): 33 | """Create a weight variable with appropriate initialization.""" 34 | initial = tf.truncated_normal(shape, stddev=0.1) 35 | return tf.Variable(initial) 36 | 37 | def bias_variable(shape): 38 | """Create a bias variable with appropriate initialization.""" 39 | initial = tf.constant(0.1, shape=shape) 40 | return tf.Variable(initial) 41 | 42 | def variable_summaries(var): 43 | """Attach a lot of summaries to a Tensor (for TensorBoard visualization).""" 44 | with tf.name_scope('summaries'): 45 | # 计算参数的均值，并使用tf.summary.scaler记录 46 | mean = tf.reduce_mean(var) 47 | tf.summary.scalar('mean', mean) 48 | 49 | # 计算参数的标准差 50 | with tf.name_scope('stddev'): 51 | stddev = tf.sqrt(tf.reduce_mean(tf.square(var - mean))) 52 | # 使用tf.summary.scaler记录记录下标准差，最大值，最小值 53 | tf.summary.scalar('stddev', stddev) 54 | tf.summary.scalar('max', tf.reduce_max(var)) 55 | tf.summary.scalar('min', tf.reduce_min(var)) 56 | # 用直方图记录参数的分布 57 | tf.summary.histogram('histogram', var) 58 | def nn_layer(input_tensor, input_dim, output_dim, layer_name, act=tf.nn.relu): 59 | """Reusable code for making a simple neural net layer. 60 | It does a matrix multiply, bias add, and then uses relu to nonlinearize. 61 | It also sets up name scoping so that the resultant graph is easy to read, 62 | and adds a number of summary ops. 63 | """ 64 | # 设置命名空间 65 | with tf.name_scope(layer_name): 66 | # 调用之前的方法初始化权重w，并且调用参数信息的记录方法，记录w的信息 67 | with tf.name_scope('weights'): 68 | weights = weight_variable([input_dim, output_dim]) 69 | variable_summaries(weights) 70 | # 调用之前的方法初始化权重b，并且调用参数信息的记录方法，记录b的信息 71 | with tf.name_scope('biases'): 72 | biases = bias_variable([output_dim]) 73 | variable_summaries(biases) 74 | # 执行wx+b的线性计算，并且用直方图记录下来 75 | with tf.name_scope('linear_compute'): 76 | preactivate = tf.matmul(input_tensor, weights) + biases 77 | tf.summary.histogram('linear', preactivate) 78 | # 将线性输出经过激励函数，并将输出也用直方图记录下来 79 | activations = act(preactivate, name='activation') 80 | tf.summary.histogram('activations', activations) 81 | 82 | # 返回激励层的最终输出 83 | return activations 84 | 85 | hidden1 = nn_layer(x, 784, 500, 'layer1') 86 | 87 | with tf.name_scope('dropout'): 88 | keep_prob = tf.placeholder(tf.float32) 89 | tf.summary.scalar('dropout_keep_probability', keep_prob) 90 | dropped = tf.nn.dropout(hidden1, keep_prob) 91 | 92 | y = nn_layer(dropped, 500, 10, 'layer2', act=tf.identity) 93 | with tf.name_scope('loss'): 94 | # 计算交叉熵损失（每个样本都会有一个损失） 95 | diff = tf.nn.softmax_cross_entropy_with_logits(labels=y_, logits=y) 96 | with tf.name_scope('total'): 97 | # 计算所有样本交叉熵损失的均值 98 | cross_entropy = tf.reduce_mean(diff) 99 | 100 | tf.summary.scalar('loss', cross_entropy) 101 | 102 | with tf.name_scope('train'): 103 | train_step = tf.train.AdamOptimizer(learning_rate).minimize( 104 | cross_entropy) 105 | with tf.name_scope('accuracy'): 106 | with tf.name_scope('correct_prediction'): 107 | # 分别将预测和真实的标签中取出最大值的索引，弱相同则返回1(true),不同则返回0(false) 108 | correct_prediction = tf.equal(tf.argmax(y, 1), tf.argmax(y_, 1)) 109 | with tf.name_scope('accuracy'): 110 | # 求均值即为准确率 111 | accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32)) 112 | 113 | tf.summary.scalar('accuracy', accuracy) 114 | # summaries合并 115 | merged = tf.summary.merge_all() 116 | # 写到指定的磁盘路径中 117 | #删除src路径下所有文件 118 | def delete_file_folder(src): 119 | '''delete files and folders''' 120 | if os.path.isfile(src): 121 | try: 122 | os.remove(src) 123 | except: 124 | pass 125 | elif os.path.isdir(src): 126 | for item in os.listdir(src): 127 | itemsrc=os.path.join(src,item) 128 | delete_file_folder(itemsrc) 129 | try: 130 | os.rmdir(src) 131 | except: 132 | pass 133 | #删除之前生成的log 134 | if os.path.exists(log_dir + '/train'): 135 | delete_file_folder(log_dir + '/train') 136 | if os.path.exists(log_dir + '/test'): 137 | delete_file_folder(log_dir + '/test') 138 | train_writer = tf.summary.FileWriter(log_dir + '/train', sess.graph) 139 | test_writer = tf.summary.FileWriter(log_dir + '/test') 140 | 141 | # 运行初始化所有变量 142 | tf.global_variables_initializer().run() 143 | 144 | def feed_dict(train): 145 | """Make a TensorFlow feed_dict: maps data onto Tensor placeholders.""" 146 | if train: 147 | xs, ys = mnist.train.next_batch(100) 148 | k = dropout 149 | else: 150 | xs, ys = mnist.test.images, mnist.test.labels 151 | k = 1.0 152 | return {x: xs, y_: ys, keep_prob: k} 153 | 154 | for i in range(max_steps): 155 | if i % 10 == 0: # 记录测试集的summary与accuracy 156 | summary, acc = sess.run([merged, accuracy], feed_dict=feed_dict(False)) 157 | test_writer.add_summary(summary, i) 158 | print('Accuracy at step %s: %s' % (i, acc)) 159 | else: # 记录训练集的summary 160 | if i % 100 == 99: # Record execution stats 161 | run_options = tf.RunOptions(trace_level=tf.RunOptions.FULL_TRACE) 162 | run_metadata = tf.RunMetadata() 163 | summary, _ = sess.run([merged, train_step], 164 | feed_dict=feed_dict(True), 165 | options=run_options, 166 | run_metadata=run_metadata) 167 | train_writer.add_run_metadata(run_metadata, 'step%03d' % i) 168 | train_writer.add_summary(summary, i) 169 | print('Adding run metadata for', i) 170 | else: # Record a summary 171 | summary, _ = sess.run([merged, train_step], feed_dict=feed_dict(True)) 172 | train_writer.add_summary(summary, i) 173 | 174 | train_writer.close() 175 | test_writer.close() 176 | --------------------------------------------------------------------------------