10 |
11 | Specifically, TDB is the combination of a Python library and a Jupyter notebook extension, built around Google's TensorFlow framework. Together, these extend TensorFlow with the following features:
12 |
13 | - **Breakpoints**: Set breakpoints on Ops and Tensors in the graph. Graph execution is paused on breakpoints and resumed by the user (via `tdb.c()`) Debugging features can be used with or without the visualization frontend.
14 | - **Arbitrary Summary Plots**: Real-time visualization of high-level information (e.g. histograms, gradient magnitudes, weight saturation) while the network is being trained. Supports arbitrary, user-defined plot functions.
15 | - **Flexible**: Mix user-defined Python and plotting functions with TensorFlow Nodes. These take in `tf.Tensors` and output placeholder nodes to be plugged into TensorFlow nodes. The below diagram illustrates how TDB nodes can be mixed with the TensorFlow graph.
16 |
17 | 
18 |
19 |
20 | ## Motivations
21 |
22 | Modern machine learning models are parametrically complex and require considerable intuition to fine-tune properly.
23 |
24 | In particular, Deep Learning methods are especially powerful, but hard to interpret in regards to their capabilities and learned representations.
25 |
26 | Can we enable better understanding of how neural nets learn, without having to change model code or sacrifice performance? Can I finish my thesis on time?
27 |
28 | TDB addresses these challenges by providing run-time visualization tools for neural nets. Real-time visual debugging allows training bugs to be detected sooner, thereby reducing the iteration time needed to build the right model.
29 |
30 | ## Setup
31 |
32 | To install the Python library,
33 |
34 | ```bash
35 | pip install tfdebugger
36 | ```
37 |
38 | To install the Jupyter Notebook extension, run the following in a Python terminal (you will need to have IPython or [Jupyter](https://jupyter.readthedocs.org/en/latest/install.html) installed)
39 |
40 | ```python
41 | import notebook.nbextensions
42 | import urllib
43 | import zipfile
44 | SOURCE_URL = 'https://github.com/ericjang/tdb/releases/download/tdb_ext_v0.1/tdb_ext.zip'
45 | urllib.urlretrieve(SOURCE_URL, 'tdb_ext.zip')
46 | with zipfile.ZipFile('tdb_ext.zip', "r") as z:
47 | z.extractall("")
48 | notebook.nbextensions.install_nbextension('tdb_ext',user=True)
49 | ```
50 |
51 | ## Tutorial
52 |
53 | To get started, check out the [MNIST Visualization Demo](notebooks/mnist_demo.ipynb). More examples and visualizations to come soon.
54 |
55 | ## User Guide
56 |
57 | ### Debugging
58 |
59 | #### Start
60 | ```python
61 | status,result=tdb.debug(evals,feed_dict=None,breakpoints=None,break_immediately=False,session=None)
62 | ```
63 |
64 | `debug()` behaves just like Tensorflow's Session.run(). If a breakpoint is hit, `status` is set to 'PAUSED' and `result` is set to `None`. Otherwise, `status` is set to 'FINISHED' and `result` is set to a list of evaluated values.
65 |
66 | #### Continue
67 | ```python
68 | status,result=tdb.c()
69 | ```
70 |
71 | Continues execution of a paused session, until the next breakpoint or end. Behaves like `debug`.
72 |
73 |
74 | #### Step
75 | ```python
76 | status,result=tdb.s()
77 | ```
78 |
79 | Evaluate the next node, then pause immediately to await user input. Unless we have reached the end of the execution queue, `status` will remain 'PAUSED'. `result` is set to the value of the node we just evaluated.
80 |
81 | #### Where
82 |
83 | ```python
84 | q=tdb.get_exe_queue()
85 | ```
86 |
87 | Return value: list of remaining nodes to be evaluated, in order.
88 |
89 | #### print
90 |
91 | ```python
92 | val=tdb.get_value(node)
93 | ```
94 |
95 | Returns value of an evaluated node (a string name or a tf.Tensor)
96 |
97 | ### Custom Nodes
98 |
99 | TDB supports 2 types of custom Ops:
100 |
101 | #### Python
102 |
103 | Here is an example of mixing tdb.PythonOps with TensorFlow.
104 |
105 | Define the following function:
106 | ```
107 | def myadd(ctx,a,b):
108 | return a+b
109 | ```
110 |
111 | ```python
112 | a=tf.constant(2)
113 | b=tf.constant(3)
114 | c=tdb.python_op(myadd,inputs=[a,b],outputs=[tf.placeholder(tf.int32)]) # a+b
115 | d=tf.neg(c)
116 | status,result=tdb.debug([d], feed_dict=None, breakpoints=None, break_immediately=False)
117 | ```
118 |
119 | When `myadd` gets evaluated, `ctx` is the instance of the PythonOp that it belongs to. You can use ctx to store state information (i.e. accumulate loss history).
120 |
121 | #### Plotting
122 |
123 | PlotOps are a special instance of PythonOp that send graphical output to the frontend.
124 |
125 | This only works with Matplotlib at the moment, but other plotting backends (Seaborn, Bokeh, Plotly) are coming soon.
126 |
127 | ```python
128 | def watch_loss(ctx,loss):
129 | if not hasattr(ctx, 'loss_history'):
130 | ctx.loss_history=[]
131 | ctx.loss_history.append(loss)
132 | plt.plot(ctx.loss_history)
133 | plt.ylabel('loss')
134 | ```
135 |
136 | ```python
137 | ploss=tdb.plot_op(viz.watch_loss,inputs=[loss])
138 | ```
139 |
140 | Refer to the [MNIST Visualization Demo](notebooks/mnist_demo.ipynb) for more examples. You can also find more examples in the [tests/](tdb/tests) directory.
141 |
142 | ## FAQ
143 |
144 | ### Is TDB affiliated with TensorFlow?
145 |
146 | No, but it is built on top of it.
147 |
148 | ### What is TDB good for?
149 |
150 | TDB is especially useful at the model prototyping stage and verifying correctness in an intuitive manner. It is also useful for high-level visualization of hidden layers during training.
151 |
152 | ### How is TDB different from TensorBoard?
153 |
154 | TensorBoard is a suite of visualization tools included with Tensorflow. Both TDB and TensorBoard attach auxiliary nodes to the TensorFlow graph in order to inspect data.
155 |
156 | TensorBoard cannot be used concurrently with running a TensorFlow graph; log files must be written first. TDB interfaces directly with the execution of a TensorFlow graph, and allows for stepping through execution one node at a time.
157 |
158 | Out of the box, TensorBoard currently only supports logging for a few predefined data formats.
159 |
160 | TDB is to TensorBoard as GDB is to printf. Both are useful in different contexts.
161 |
162 |
163 |
164 | ## License
165 |
166 | Apache 2.0
167 |
168 |
169 |
--------------------------------------------------------------------------------
/notebooks/mnist_demo.ipynb:
--------------------------------------------------------------------------------
1 | {
2 | "cells": [
3 | {
4 | "cell_type": "markdown",
5 | "metadata": {},
6 | "source": [
7 | "# MNIST Visualization Example\n",
8 | "\n",
9 | "Real-time visualization of MNIST training on a CNN, using TensorFlow and [TensorDebugger](https://github.com/ericjang/tdb)\n",
10 | "\n",
11 | "The visualizations in this notebook won't show up on http://nbviewer.ipython.org. To view the widgets and interact with them, you will need to download this notebook and run it with a Jupyter Notebook server."
12 | ]
13 | },
14 | {
15 | "cell_type": "markdown",
16 | "metadata": {},
17 | "source": [
18 | "## Step 1: Load TDB Notebook Extension"
19 | ]
20 | },
21 | {
22 | "cell_type": "code",
23 | "execution_count": 2,
24 | "metadata": {
25 | "collapsed": false
26 | },
27 | "outputs": [
28 | {
29 | "data": {
30 | "application/javascript": [
31 | "Jupyter.utils.load_extensions('tdb_ext/main')"
32 | ],
33 | "text/plain": [
34 | "
25 | {plots}
26 |
27 | )
28 | }
29 | })
30 |
31 | return PlotListView
32 | });
33 |
--------------------------------------------------------------------------------
/tdb_ext/components/plotview.jsx:
--------------------------------------------------------------------------------
1 | // plottable watch variable in HyperTree
2 |
3 | define([
4 | 'react',
5 | 'jquery',
6 | 'dispatcher',
7 | ], function(React,$,dispatcher){
8 |
9 |
10 | var PlotView = React.createClass({
11 | propTypes : {
12 | name: React.PropTypes.string,
13 | src: React.PropTypes.string
14 | },
15 | render: function() {
16 | // need an x close button
17 | var css ={
18 | border:"1px solid",
19 | borderRadius:"5px",
20 | margin:"10px",
21 | position: "relative"
22 | }
23 | var imgcss = {
24 | maxWidth: "100%",
25 | padding:"8px"
26 | }
27 | var h3css={
28 | textAlign: "center"
29 | }
30 | var closecss={
31 | position: "absolute",
32 | top: "20px",
33 | right: "20px"
34 | }
35 | return (
36 |
37 | 
42 |
43 | )
44 | },
45 | _onDestroyClick: function() {
46 | dispatcher.dispatch({
47 | actionType:'remove_plot',
48 | name:this.props.name
49 | })
50 | }
51 | })
52 |
53 | return PlotView
54 | });
55 |
--------------------------------------------------------------------------------
/tdb_ext/components/textinputview.jsx:
--------------------------------------------------------------------------------
1 | // plottable watch variable in HyperTree
2 |
3 | define([
4 | 'react',
5 | 'jquery',
6 | 'dispatcher',
7 | ], function(React,$,dispatcher){
8 |
9 |
10 | var TextInputView = React.createClass({
11 | propTypes : {
12 | name: React.PropTypes.string,
13 | src: React.PropTypes.string
14 | },
15 | render: function() {
16 | // need an x close button
17 | var css ={
18 | border:"1px solid",
19 | borderRadius:"5px",
20 | margin:"10px"
21 | }
22 | var imgcss = {
23 | maxWidth: "100%",
24 | margin:"5px"
25 | }
26 | return (
27 | {this.props.name}
38 | 41 |
28 |
30 |
31 | )
32 | }
33 | })
34 |
35 | return TextInputView
36 | });
37 |
--------------------------------------------------------------------------------
/tdb_ext/components/ui.jsx:
--------------------------------------------------------------------------------
1 | // class that manages the top-level React component
2 | // handles UI-related methods
3 |
4 | var COMPONENTS='/nbextensions/tdb_ext/components/'
5 |
6 | define([
7 | 'react',
8 | 'jquery',
9 | 'dispatcher',
10 | 'plotstore',
11 | 'jsx!' + COMPONENTS + 'plotlistview',
12 | 'jsx!' + COMPONENTS + 'user_msg_view'
13 | ], function(React,$,dispatcher,PlotStore, PlotListView, UserMsgView){
14 |
15 | var UI=function(){
16 | this.had_loaded=false
17 | // starting notebook width
18 | this._nbw = .65 // width of notebook (percentage) when HT pane is open
19 |
20 | this.UIView = React.createClass({
21 | getInitialState: function() {
22 | return this._getState()
23 | },
24 | render: function() {
25 | var css={
26 | 'backgroundColor':'#FFFFFF',
27 | }
28 | return (
29 | {this.props.name}
29 |
30 |
33 | )
34 | },
35 | componentDidMount: function() {
36 | // listen to PlotStore
37 | PlotStore.addChangeListener(this._onChange)
38 | },
39 | _onChange: function() {
40 | this.setState(this._getState())
41 | },
42 | _getState: function() {
43 | return {
44 | "plots":PlotStore.getPlots()
45 | }
46 | }
47 | })
48 |
49 | }
50 |
51 | UI.prototype.load_ui = function() {
52 | // initializes the UI
53 |
54 | $('#site').after('')
55 |
56 | var w=$('#site').width()
57 | var h=$('#site').height()
58 |
59 | $('#ht_separator').css({
60 | height: "100%",
61 | width: "1%",
62 | float: "left",
63 | backgroundColor: "grey"
64 | })
65 |
66 | $('#ht_separator').after('')
67 |
68 |
69 | $('#ht_main').width((1-this._nbw-.02)*w)
70 |
71 | $('#ht_main').css({
72 | float:'left',
73 | height:'100%',
74 | overflow:'scroll'
75 | })
76 |
77 | var self=this
78 | $('#ht_separator').draggable({
79 | axis: 'x',
80 | containment: 'parent',
81 | helper: 'clone',
82 | start: function(event, ui) {
83 | self.ow=$(window).width()
84 | },
85 | drag: function (event, ui) {
86 | //console.log(ui)
87 | var width=ui.offset.left
88 | $(this).prev().width(width)
89 | //console.log(self.ow)
90 | $(this).next().width(self.ow-width-0.01*self.ow)
91 | }
92 | });
93 |
94 | React.render(
18 |
20 | )
21 | },
22 | componentDidMount: function() {
23 | var self=this
24 | dispatcher.register(function(action){
25 | if (action.actionType == 'user_msg')
26 | self.setState({msg:action.data})
27 | })
28 | }
29 | })
30 |
31 | return UserMsgView
32 | });
33 |
34 |
--------------------------------------------------------------------------------
/tdb_ext/config.yaml:
--------------------------------------------------------------------------------
1 | # Jupyter Notebook Extension Description
2 | Name: HyperTree
3 | Description: Deep Learning IDE
4 | Link: evjang.com/hypertree
5 | #Icon: icon.png
6 | Main: main.js
7 | Compatibility: 4.x
8 |
--------------------------------------------------------------------------------
/tdb_ext/dispatcher.js:
--------------------------------------------------------------------------------
1 | // application-wide event dispatcher
2 |
3 | define([
4 | "flux"
5 | ], function(flux){
6 | return new flux.Dispatcher()
7 | })
8 |
--------------------------------------------------------------------------------
/tdb_ext/htapp.js:
--------------------------------------------------------------------------------
1 | // communicates with the corresponding htapp on the python side
2 |
3 |
4 | define([
5 | 'base/js/namespace',
6 | 'jsx!/nbextensions/tdb_ext/components/ui',
7 | 'base/js/events',
8 | 'dispatcher'
9 | ], function(Jupyter,UI,events,dispatcher){
10 |
11 | var HTApp=function(comm_manager){
12 | //
13 | // MEMBER VARIABLES
14 | //
15 |
16 | this.ui = new UI()
17 | this._comm = null
18 | this.is_connected = true
19 |
20 | //
21 | // CONSTRUCTOR INITIALIZATION
22 | //
23 | var self=this
24 | comm_manager.register_target('tdb', function(comm,msg){
25 | dispatcher.dispatch({
26 | actionType:'user_msg',
27 | data:'TDB connected: success'
28 | })
29 |
30 | self._comm=comm
31 | this.is_connected=true
32 | comm.on_msg(function(msg){
33 | var data=msg['content']['data']
34 |
35 | var msg_type=data['msg_type']
36 | if (msg_type == 'action') {
37 | // if we receive an action, send it to dispatcher
38 | var action={
39 | actionType: data['action'],
40 | data: data['params']
41 | }
42 | dispatcher.dispatch(action)
43 | } else {
44 | console.log('Unrecognized msg_type ' + msg_type)
45 | console.log(data)
46 | }
47 | })
48 | })
49 |
50 | events.on('kernel_restarting.Kernel', function() {
51 | dispatcher.dispatch({
52 | actionType: 'clear'
53 | })
54 | });
55 |
56 | }
57 |
58 |
59 | return HTApp
60 | })
61 |
62 |
63 |
--------------------------------------------------------------------------------
/tdb_ext/keymirror.js:
--------------------------------------------------------------------------------
1 |
2 | /**
3 | * AMD-version of Facebook's Keymirror
4 | * Constructs an enumeration with keys equal to their value.
5 | *
6 | * For example:
7 | *
8 | * var COLORS = keyMirror({blue: null, red: null});
9 | * var myColor = COLORS.blue;
10 | * var isColorValid = !!COLORS[myColor];
11 | *
12 | * The last line could not be performed if the values of the generated enum were
13 | * not equal to their keys.
14 | *
15 | * Input: {key1: val1, key2: val2}
16 | * Output: {key1: key1, key2: key2}
17 | *
18 | * @param {object} obj
19 | * @return {object}
20 | */
21 |
22 | define([],function(){
23 | var keyMirror = function(obj) {
24 | var ret = {};
25 | var key;
26 | if (!(obj instanceof Object && !Array.isArray(obj))) {
27 | throw new Error('keyMirror(...): Argument must be an object.');
28 | }
29 | for (key in obj) {
30 | if (obj.hasOwnProperty(key)) {
31 | ret[key] = key;
32 | }
33 | }
34 | return ret;
35 | };
36 | return keyMirror;
37 | })
38 |
--------------------------------------------------------------------------------
/tdb_ext/main.js:
--------------------------------------------------------------------------------
1 | /*
2 | This is the entry point into HyperTree nbextension
3 | after installing into ~/.jupyter/nbextensions/, do
4 | javascript:require(["nbextensions/tdb_ext"] function(ht){});
5 | */
6 |
7 | var ROOT = '/nbextensions/tdb_ext'
8 | var BOWER = ROOT+'/bower_components'
9 | var STORES = '/nbextensions/tdb_ext/stores'
10 |
11 | // easy access to vendor libs
12 |
13 | require.config({
14 | paths: {
15 | "dispatcher": ROOT + "/dispatcher",
16 | "flux":BOWER+"/flux/dist/Flux.min",
17 | "eventemitter2":BOWER + '/eventemitter2/lib/eventemitter2',
18 | "keyMirror":ROOT+'/keymirror',
19 | "raphael":BOWER + "/raphael/raphael-min",
20 | "dispatcher": ROOT + "/dispatcher",
21 | "react": BOWER + "/react/react-with-addons",
22 | "JSXTransformer": BOWER + "/react/JSXTransformer",
23 | "jsx": BOWER + "/requirejs-react-jsx/jsx",
24 | "text": BOWER + "/requirejs-text/text",
25 | "plotstore":STORES + "/plotstore"
26 | },
27 | shim : {
28 | "react": {
29 | "exports": "React"
30 | },
31 | "JSXTransformer": "JSXTransformer"
32 | },
33 | config: {
34 | jsx: {
35 | fileExtension: ".jsx",
36 | transformOptions: {
37 | harmony: true,
38 | stripTypes: false,
39 | inlineSourceMap: true
40 | },
41 | usePragma: false
42 | }
43 | }
44 | });
45 |
46 | define([
47 | 'base/js/namespace',
48 | ROOT + '/htapp.js'
49 | ], function(Jupyter,HTApp){
50 | var kernel = Jupyter.notebook.kernel
51 | var comm_manager=Jupyter.notebook.kernel.comm_manager
52 | // icky - binding as a global variable
53 | HT=new HTApp(comm_manager)
54 | HT.ui.load_ui()
55 | HT.ui.show_ui()
56 | console.log('HT nbextension loaded')
57 | })
58 |
--------------------------------------------------------------------------------
/tdb_ext/package.json:
--------------------------------------------------------------------------------
1 | {
2 | "name": "ht",
3 | "version": "0.0.0",
4 | "description": "Deep Learning IDE",
5 | "main": "",
6 | "scripts": {
7 | "test": "echo \"Error: no test specified\" && exit 1"
8 | },
9 | "author": "Eric Jang",
10 | "license": "Apache 2.0",
11 | "devDependencies": {
12 | "bower": "~1.6.7"
13 | }
14 | }
15 |
--------------------------------------------------------------------------------
/tdb_ext/stores/plotstore.js:
--------------------------------------------------------------------------------
1 |
2 | define([
3 | 'dispatcher',
4 | 'eventemitter2'
5 | ], function(dispatcher,EventEmitter){
6 |
7 | var CHANGE_EVENT = 'change'
8 |
9 | // key = PlotOp node name
10 | // value = img src
11 | var _plots = {}
12 |
13 |
14 | var PlotStore = Object.assign({}, EventEmitter.prototype, {
15 | getPlots: function() {
16 | return _plots
17 | },
18 | update: function(data) {
19 | _plots[data.name]=data.src
20 | },
21 | clear: function() {
22 | _plots={}
23 | },
24 | remove: function(name) {
25 | if (_plots.hasOwnProperty(name)) {
26 | delete _plots[name]
27 | }
28 | },
29 | emitChange: function() {
30 | this.emit(CHANGE_EVENT);
31 | },
32 | addChangeListener: function(callback) {
33 | this.on(CHANGE_EVENT, callback)
34 | },
35 | removeChangeListener: function(callback) {
36 | this.removeListener(CHANGE_EVENT, callback)
37 | }
38 | })
39 |
40 | // register store with the dispatcher
41 | dispatcher.register(function(action){
42 | switch(action.actionType){
43 | case 'update_plot':
44 | // create new plot or update an old one
45 | PlotStore.update(action.data)
46 | PlotStore.emitChange()
47 | break;
48 | case 'remove_plot':
49 | PlotStore.remove(action.name)
50 | PlotStore.emitChange()
51 | break;
52 | case 'clear':
53 | PlotStore.clear()
54 | PlotStore.emitChange()
55 | break;
56 | default:
57 | // no op
58 | }
59 | })
60 |
61 | return PlotStore
62 | })
63 |
--------------------------------------------------------------------------------
{this.state.msg}
19 |