├── README.md
├── mnist_example.ipynb
├── mnist_model.h5
├── mnist_tsne.ipynb
├── morad_scratch.ipynb
├── tsne_clean.ipynb
├── tsne_train_only.ipynb
└── tsne_utils.py


/README.md:
--------------------------------------------------------------------------------
  1 | # Signal Modulation Classification Using Machine Learning
  2 | ## Morad Shefa, Gerry Zhang, Steve Croft
  3 | If you want to skip all the readings and want to see what we provide and how you can use our code feel free to skip to the final section. Also, you can reach me at moradshefa@berkeley.edu
  4 | ## Background
  5 |      
  6 |   A deep convolutional neural network architecture is used for signal modulation classification. There are different reasons why signal modulation classification can be important. For example, radio-frequency interference (RFI) is a major problem in radio astronomy. This is especially prevalent in SETI where RFI plagues collected data and can exhibit characteristics we look for in SETI signals. <br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
  7 |   As instrumentation expands beyond frequencies allocated to radio astronomy and human generated technology fills more of the wireless spectrum classifying RFI as such becomes more important. <br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
  8 |   Modulation schemes are methods of encoding information onto a high frequency carrier wave, that are more practical for transmission. Human-generated RFI tends to utilize one of a limited number of modulation schemes. Most of these methods modulate the amplitude, frequency, or phase of the carrier wave. Thus one way of classifying RFI is to classify it as a certain modulation scheme.
  9 | 
 10 | 
 11 | <img align="left" width="581" height="327" src="https://user-images.githubusercontent.com/22650698/44496778-e247e580-a62a-11e8-9050-31ad4b8184ff.png">
 12 | 
 13 | <br><br>Examples of how information can be transmitted by changing the shape of a carrier wave. Picture credit: Tait Radio Academy<br><br><br><br><br><br><br><br>
 14 | 
 15 | 
 16 | ## Methods and Materials
 17 | * Datasets provided by the Army Rapid Capabilities Office’s Artificial Intelligence Signal Classification challenge
 18 | * Simulated signals of 24 different modulations: 16PSK, 2FSK_5KHz, 2FSK_75KHz, 8PSK, AM_DSB, AM_SSB, APSK16_c34,  APSK32_c34, BPSK, CPFSK_5KHz, CPFSK_75KHz, FM_NB, FM_WB,  GFSK_5KHz, GFSK_75KHz, GMSK, MSK, NOISE, OQPSK, PI4QPSK, QAM16,  QAM32, QAM64, QPSK
 19 | * 6 different signal to noise ratios (SNR): -10 dB, -6 dB, -2 dB, 2 dB, 6 dB, 10 dB
 20 | * Used deep convolutional neural networks for classification
 21 | * CNNs are widely used and have advanced performance in computer vision
 22 | * Convolutions with learned filters are used to extract features in the data
 23 | * Hierarchical classification: Classify into subgroups then use another classifier to identify modulation
 24 | * Data augmentation: Perturbing the data during training to avoid overfit
 25 | * Ensemble training: Train multiple models and average predictions
 26 | * Residual Connections: Allow for deeper networks by avoiding vanishing gradients
 27 | 
 28 | <img align="left" width="444" height="211" src="https://user-images.githubusercontent.com/22650698/44498003-348c0500-a631-11e8-87ee-67e76453b903.png">
 29 | 
 30 | <br><br>
 31 | Inception Layers: 
 32 | * Layers with filters of different dimensions
 33 | * 1x1 convolutions to reduce dimension
 34 | <br><br>Picture credit:  GoogLeNet
 35 | <br><br><br><br><br>
 36 | 
 37 | 
 38 | <img align="left" width="444" height="444" src="https://user-images.githubusercontent.com/22650698/44546315-669e7500-a6cc-11e8-997f-69378706b6f0.png">
 39 | 
 40 | <br><br>
 41 | Dimensionality reduction using t-distributed stochastic neighbor embedding (t-SNE) and principal component analysis (PCA) to visualize feature extraction and diagnose problems of the architecture. 
 42 | <br><br>These t-SNE plots helped us to evaluate our models on unlabelled test data that was distributed differently than training data.
 43 | <br><br>Dimensionality reduction after extracting features of 16PSK (red), 2FSK_5kHz (green),AM_DSB (blue)
 44 | <br><br><br><br><br><br><br><br>
 45 | 
 46 | 
 47 | <img align="left" width="444" height="211" src="https://user-images.githubusercontent.com/22650698/44546552-24296800-a6cd-11e8-8417-aaa4f244e42e.png">
 48 | 
 49 | <br>
 50 | Example of a vanilla convolutional neural network. <br>Picture credit: MDPI: "A Framework for Designing the Architectures of Deep Convolutional Neural Networks"
 51 | <br><br><br><br><br><br><br>
 52 | 
 53 | 
 54 | <img align="left" width="444" height="444" src="https://user-images.githubusercontent.com/22650698/44546670-85513b80-a6cd-11e8-93be-83c6feefab2d.png">
 55 | 
 56 | Embedding: 
 57 | * Extracting output of final inception layer; 100 per modulation (dimension: 5120)
 58 | * Reducing dimension using principal component analysis (dimension: 50)
 59 | * Reducing dimension using t-distributed neighbor embedding (dimension: 2)
 60 | 
 61 | <br>*Embedding of 24 modulations using one of our models. As we can see different modulations map to different clusters even in 2-dimensional space indicating that our model does well in extracting features that are specific to the different modulation schemes. The axis have no physical meaning. They merely represent the space found by t-SNE in which close points in high dimension stay close in lower dimension.*
 62 | <br><br>
 63 | 
 64 | ## Results
 65 | Results for one of our models without hierarchical inference.
 66 | 
 67 | <img align="left" width="259" height="294" src="https://user-images.githubusercontent.com/22650698/44547003-8d5dab00-a6ce-11e8-9ec8-5509adc178c8.png">
 68 | 
 69 | CNNs are able to achieve high accuracy in classification of signal modulations across different SNR values
 70 | <br><br><br><br><br><br><br><br><br><br><br><br><br>
 71 | 
 72 | <br><img src="https://user-images.githubusercontent.com/22650698/44547386-cba79a00-a6cf-11e8-9a28-e179aba3c12c.png"  width="330" height="330" /> <img src="https://user-images.githubusercontent.com/22650698/44547206-34424700-a6cf-11e8-8f20-6680c34529cc.png"  width="315" height="315" /> 
 73 | 
 74 | 
 75 | <img align="left" width="360" height="340" src="https://user-images.githubusercontent.com/22650698/44547394-d4986b80-a6cf-11e8-8f6d-d587482d9281.png">
 76 | <br><br>Confusion matrices for different SNR values (-10 dB, -6 dB, 10 dB) 
 77 | <br>As SNR increases accuracies increase and more predicted labels are true labels causing stronger diagonals
 78 | 
 79 | <br><br><br><br><br><br><br><br><br>
 80 | 
 81 | ## Conclusion
 82 | * The ability of CNNs to classify signal modulations at high accuracy shows great promise in the future of using CNNs and other machine learning methods to classify RFI
 83 | * Future work can focus on extending these methods to classify modulations in real data
 84 | * One can use machine learning methods  to extend these models to real data
 85 |   * Use domain adaptation to find performing model for a target distribution that is different from the source distribution/ training data
 86 |   * Label real data and train a classifier
 87 | 
 88 | <img align="left" width="390" height="300" src="https://user-images.githubusercontent.com/22650698/44547872-4624e980-a6d1-11e8-849f-0ae5f8f9fe6b.png">
 89 | Adapting models to domains that are related but different than what was trained on is a common challenge for machine learning systems. Picture credit: Oxford Robotics Institute
 90 | <br><br><br><br><br><br><br><br><br><br><br><br>
 91 | 
 92 | <img align="left" width="390" height="170" src="https://user-images.githubusercontent.com/22650698/44547982-a2880900-a6d1-11e8-915d-75943d41cdb6.png">
 93 | When the target distribution is different classification performance can suffer. Domain adaptation methods aim at finding a space in which the discrepancy is low. Picture credit: Science Direct: “Unsupervised domain adaptation techniques based on auto-encoder for non-stationary EEG-based emotion recognition”
 94 | <br><br><br>
 95 | 
 96 | ## Provided
 97 | Unfortunately, as part of the army challenge rules we are not allowed to distribute any of the provided datasets. 
 98 | However, we will provide:
 99 | * a notebook that we used to experiment with different models and that is able to achieve 
100 | our results with our data (morad_scatch.ipynb)
101 | * a notebook that builds a similar model but simplified to classify handwritten digits on the mnist dataset that achieves 99.43% accuracy (mnist_example.ipynb)
102 | * the notebook we used to get the t-SNE embeddings on training and unlabelled test data to evaluate models (tsne_clean.ipynb)
103 | * simplified code that can be used to get your own t-SNE embeddings on your own Keras models and plot them interactively using Bokeh if you desire (tsne_utils.py) 
104 | * a notebook that uses tsne_utils.py and one of our models to get embeddings for signal modulation data on training data only (tsne_train_only.ipynb)
105 | * the mnist model (mnist_model.h5)
106 | * a notebook to do t-SNE on the mnist data and model (mnist_tsne.ipynb)
107 | 
108 | <br><img align="left" width="470" height="420" src="https://user-images.githubusercontent.com/22650698/44552995-f77f4b80-a6e0-11e8-9267-2e95bd400fbb.png">
109 | 
110 | Simple embedding of our small mnist model (no legend, no prediction probability). As we can see the data maps decently into 10 different clusters.
111 | <br><br><br><br><br><br><br><br><br><br><br><br><br><br><br><br>
112 | 
113 | <br><img align="left" width="470" height="420" src="https://user-images.githubusercontent.com/22650698/44553335-f69ae980-a6e1-11e8-82d9-29231cc6791a.png">
114 | 
115 | Embedding showing the legend and the predicted probability for each point. The point over which we hover is labelled 1 with predicted probability 0.822.
116 | 


--------------------------------------------------------------------------------
/mnist_model.h5:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/moradshefa/ml_signal_modulation_classification/d476f867b7e593eb68fd84ab80f966db13bf8a44/mnist_model.h5


--------------------------------------------------------------------------------
/tsne_train_only.ipynb:
--------------------------------------------------------------------------------
  1 | {
  2 |  "cells": [
  3 |   {
  4 |    "cell_type": "markdown",
  5 |    "metadata": {},
  6 |    "source": [
  7 |     "This notebook does t-SNE on the army signal classification data set using our model. \n",
  8 |     "It takes 100 points from each modulation from a training set which was used as a validation set during training."
  9 |    ]
 10 |   },
 11 |   {
 12 |    "cell_type": "code",
 13 |    "execution_count": 1,
 14 |    "metadata": {},
 15 |    "outputs": [
 16 |     {
 17 |      "name": "stderr",
 18 |      "output_type": "stream",
 19 |      "text": [
 20 |       "/home/morads/anaconda3/envs/py36/lib/python3.6/site-packages/h5py/__init__.py:34: FutureWarning: Conversion of the second argument of issubdtype from `float` to `np.floating` is deprecated. In future, it will be treated as `np.float64 == np.dtype(float).type`.\n",
 21 |       "  from ._conv import register_converters as _register_converters\n",
 22 |       "Using TensorFlow backend.\n"
 23 |      ]
 24 |     }
 25 |    ],
 26 |    "source": [
 27 |     "from data_loader import *\n",
 28 |     "from keras.models import Model, load_model\n",
 29 |     "from keras import backend as K\n",
 30 |     "from sklearn.manifold import TSNE\n",
 31 |     "from sklearn.decomposition import PCA\n",
 32 |     "\n",
 33 |     "from bokeh.plotting import figure, output_notebook, show, ColumnDataSource\n",
 34 |     "from bokeh.models import HoverTool\n",
 35 |     "\n",
 36 |     "from tsne_utils import *"
 37 |    ]
 38 |   },
 39 |   {
 40 |    "cell_type": "code",
 41 |    "execution_count": 2,
 42 |    "metadata": {
 43 |     "collapsed": true
 44 |    },
 45 |    "outputs": [],
 46 |    "source": [
 47 |     "CLASSES_24 = ['16PSK', '2FSK_5KHz', '2FSK_75KHz', '8PSK', 'AM_DSB', 'AM_SSB', 'APSK16_c34',\n",
 48 |     " 'APSK32_c34', 'BPSK', 'CPFSK_5KHz', 'CPFSK_75KHz', 'FM_NB', 'FM_WB',\n",
 49 |     " 'GFSK_5KHz', 'GFSK_75KHz', 'GMSK', 'MSK', 'NOISE', 'OQPSK', 'PI4QPSK', 'QAM16',\n",
 50 |     " 'QAM32', 'QAM64', 'QPSK']\n",
 51 |     "\n",
 52 |     "BOOKEH_COLORS = {\n",
 53 |     "    '16PSK': 'aqua', \n",
 54 |     "    '2FSK_5KHz': 'aquamarine',  \n",
 55 |     "    '2FSK_75KHz': 'bisque',  \n",
 56 |     "    '8PSK': 'black', \n",
 57 |     "    'AM_DSB': 'blue', \n",
 58 |     "    'AM_SSB':'blueviolet', \n",
 59 |     "    'APSK16_c34': 'brown',\n",
 60 |     "    'APSK32_c34': 'burlywood', \n",
 61 |     "    'BPSK': 'cadetblue', \n",
 62 |     "    'CPFSK_5KHz': 'chartreuse', \n",
 63 |     "    'CPFSK_75KHz': 'chocolate', \n",
 64 |     "    'FM_NB': 'cornflowerblue', \n",
 65 |     "    'FM_WB': 'crimson',\n",
 66 |     "    'GFSK_5KHz': 'darkcyan', \n",
 67 |     "    'GFSK_75KHz': 'darkgoldenrod', \n",
 68 |     "    'GMSK': 'darkgray', \n",
 69 |     "    'MSK': 'darkgreen', \n",
 70 |     "    'NOISE': 'darkorange', \n",
 71 |     "    'OQPSK': 'deeppink', \n",
 72 |     "    'PI4QPSK': 'fuchsia', \n",
 73 |     "    'QAM16': 'gold',\n",
 74 |     "    'QAM32': 'lightblue', \n",
 75 |     "    'QAM64': 'magenta', \n",
 76 |     "    'QPSK': 'plum'\n",
 77 |     "}\n",
 78 |     "\n",
 79 |     "\n",
 80 |     "BOOKEH_SHAPES = {\n",
 81 |     "    '16PSK':1,\n",
 82 |     "    '2FSK_5KHz':1,\n",
 83 |     "    '2FSK_75KHz':1,\n",
 84 |     "    '8PSK':1,\n",
 85 |     "    'AM_DSB':1,\n",
 86 |     "    'AM_SSB':1,\n",
 87 |     "    'APSK16_c34':1,\n",
 88 |     "    'APSK32_c34':1,\n",
 89 |     "    'BPSK':1,\n",
 90 |     "    'CPFSK_5KHz':1,\n",
 91 |     "    'CPFSK_75KHz':1,\n",
 92 |     "    'FM_NB':1,\n",
 93 |     "    'FM_WB':1,\n",
 94 |     "    'GFSK_5KHz':1,\n",
 95 |     "    'GFSK_75KHz':1,\n",
 96 |     "    'GMSK':1,\n",
 97 |     "    'MSK':1,\n",
 98 |     "    'NOISE':1,\n",
 99 |     "    'OQPSK':1,\n",
100 |     "    'PI4QPSK':1,\n",
101 |     "    'QAM16':2,\n",
102 |     "    'QAM32':2,\n",
103 |     "    'QAM64':2,\n",
104 |     "    'QPSK':2,\n",
105 |     "}"
106 |    ]
107 |   },
108 |   {
109 |    "cell_type": "code",
110 |    "execution_count": 3,
111 |    "metadata": {
112 |     "collapsed": true
113 |    },
114 |    "outputs": [],
115 |    "source": [
116 |     "def load_training_data(data_file,num_samples=100, mods = None, spectrum=False):\n",
117 |     "    testdata = LoadModRecData(data_file, 1., 0., 0., load_snrs=[10], num_samples_per_key=num_samples, load_mods = mods,spectrum=spectrum)\n",
118 |     "    train_data = testdata.signalData\n",
119 |     "    train_labels = testdata.signalLabels[:,0]\n",
120 |     "    return train_data, train_labels\n"
121 |    ]
122 |   },
123 |   {
124 |    "cell_type": "code",
125 |    "execution_count": 4,
126 |    "metadata": {},
127 |    "outputs": [
128 |     {
129 |      "name": "stderr",
130 |      "output_type": "stream",
131 |      "text": [
132 |       "/home/morads/.local/lib/python3.6/site-packages/keras/engine/saving.py:270: UserWarning: No training configuration found in save file: the model was *not* compiled. Compile it manually.\n",
133 |       "  warnings.warn('No training configuration found in save file: '\n"
134 |      ]
135 |     }
136 |    ],
137 |    "source": [
138 |     "model = load_model('modulation_classification_example_model.h5')\n",
139 |     "train_file_path = \"/datax/yzhang/training_data/training_data_chunk_14.pkl\"\n",
140 |     "train_file_path = \"/datax/yzhang/army_challenge/training_data/training_data_chunk_14.pkl\"\n",
141 |     "\n",
142 |     "num_samples_from_train = 20"
143 |    ]
144 |   },
145 |   {
146 |    "cell_type": "code",
147 |    "execution_count": 5,
148 |    "metadata": {},
149 |    "outputs": [
150 |     {
151 |      "name": "stdout",
152 |      "output_type": "stream",
153 |      "text": [
154 |       "[Data Loader] - Loading Datafile,  /datax/yzhang/army_challenge/training_data/training_data_chunk_14.pkl (time series)\n",
155 |       "[Data Loader] - Counting Number of Examples in Dataset...\n",
156 |       "[Data Loader] - Number of Examples in Dataset: 480\n",
157 |       "[Data Loader] - [Modulation Dataset] Adding Collects for: 16PSK\n",
158 |       "[Data Loader] - [Modulation Dataset] Adding Collects for: 2FSK_5KHz\n",
159 |       "[Data Loader] - [Modulation Dataset] Adding Collects for: 2FSK_75KHz\n",
160 |       "[Data Loader] - [Modulation Dataset] Adding Collects for: 8PSK\n",
161 |       "[Data Loader] - [Modulation Dataset] Adding Collects for: AM_DSB\n",
162 |       "[Data Loader] - [Modulation Dataset] Adding Collects for: AM_SSB\n",
163 |       "[Data Loader] - [Modulation Dataset] Adding Collects for: APSK16_c34\n",
164 |       "[Data Loader] - [Modulation Dataset] Adding Collects for: APSK32_c34\n",
165 |       "[Data Loader] - [Modulation Dataset] Adding Collects for: BPSK\n",
166 |       "[Data Loader] - [Modulation Dataset] Adding Collects for: CPFSK_5KHz\n",
167 |       "[Data Loader] - [Modulation Dataset] Adding Collects for: CPFSK_75KHz\n",
168 |       "[Data Loader] - [Modulation Dataset] Adding Collects for: FM_NB\n",
169 |       "[Data Loader] - [Modulation Dataset] Adding Collects for: FM_WB\n",
170 |       "[Data Loader] - [Modulation Dataset] Adding Collects for: GFSK_5KHz\n",
171 |       "[Data Loader] - [Modulation Dataset] Adding Collects for: GFSK_75KHz\n",
172 |       "[Data Loader] - [Modulation Dataset] Adding Collects for: GMSK\n",
173 |       "[Data Loader] - [Modulation Dataset] Adding Collects for: MSK\n",
174 |       "[Data Loader] - [Modulation Dataset] Adding Collects for: NOISE\n",
175 |       "[Data Loader] - [Modulation Dataset] Adding Collects for: OQPSK\n",
176 |       "[Data Loader] - [Modulation Dataset] Adding Collects for: PI4QPSK\n",
177 |       "[Data Loader] - [Modulation Dataset] Adding Collects for: QAM16\n",
178 |       "[Data Loader] - [Modulation Dataset] Adding Collects for: QAM32\n",
179 |       "[Data Loader] - [Modulation Dataset] Adding Collects for: QAM64\n",
180 |       "[Data Loader] - [Modulation Dataset] Adding Collects for: QPSK\n",
181 |       "[Data Loader] - Converting to numpy arrays...\n",
182 |       "[Data Loader] - Shuffling Data...\n",
183 |       "[Data Loader] - Splitting Data...\n",
184 |       "[Data Loader] - Train Size: 480 Validation Size: 0 Test Size: 0\n",
185 |       "[Data Loader] - Done.\n",
186 |       "\n"
187 |      ]
188 |     }
189 |    ],
190 |    "source": [
191 |     "data, labels = load_training_data(train_file_path,num_samples=num_samples_from_train,mods=None)"
192 |    ]
193 |   },
194 |   {
195 |    "cell_type": "code",
196 |    "execution_count": 7,
197 |    "metadata": {
198 |     "scrolled": false
199 |    },
200 |    "outputs": [
201 |     {
202 |      "data": {
203 |       "text/html": [
204 |        "\n",
205 |        "    <div class=\"bk-root\">\n",
206 |        "        <a href=\"http://bokeh.pydata.org\" target=\"_blank\" class=\"bk-logo bk-logo-small bk-logo-notebook\"></a>\n",
207 |        "        <span id=\"167b5046-3bf8-4af6-912d-c3c17cb8be74\">Loading BokehJS ...</span>\n",
208 |        "    </div>"
209 |       ]
210 |      },
211 |      "metadata": {},
212 |      "output_type": "display_data"
213 |     },
214 |     {
215 |      "data": {
216 |       "application/javascript": [
217 |        "\n",
218 |        "(function(global) {\n",
219 |        "  function now() {\n",
220 |        "    return new Date();\n",
221 |        "  }\n",
222 |        "\n",
223 |        "  var force = true;\n",
224 |        "\n",
225 |        "  if (typeof (window._bokeh_onload_callbacks) === \"undefined\" || force === true) {\n",
226 |        "    window._bokeh_onload_callbacks = [];\n",
227 |        "    window._bokeh_is_loading = undefined;\n",
228 |        "  }\n",
229 |        "\n",
230 |        "\n",
231 |        "  \n",
232 |        "  if (typeof (window._bokeh_timeout) === \"undefined\" || force === true) {\n",
233 |        "    window._bokeh_timeout = Date.now() + 5000;\n",
234 |        "    window._bokeh_failed_load = false;\n",
235 |        "  }\n",
236 |        "\n",
237 |        "  var NB_LOAD_WARNING = {'data': {'text/html':\n",
238 |        "     \"<div style='background-color: #fdd'>\\n\"+\n",
239 |        "     \"<p>\\n\"+\n",
240 |        "     \"BokehJS does not appear to have successfully loaded. If loading BokehJS from CDN, this \\n\"+\n",
241 |        "     \"may be due to a slow or bad network connection. Possible fixes:\\n\"+\n",
242 |        "     \"</p>\\n\"+\n",
243 |        "     \"<ul>\\n\"+\n",
244 |        "     \"<li>re-rerun `output_notebook()` to attempt to load from CDN again, or</li>\\n\"+\n",
245 |        "     \"<li>use INLINE resources instead, as so:</li>\\n\"+\n",
246 |        "     \"</ul>\\n\"+\n",
247 |        "     \"<code>\\n\"+\n",
248 |        "     \"from bokeh.resources import INLINE\\n\"+\n",
249 |        "     \"output_notebook(resources=INLINE)\\n\"+\n",
250 |        "     \"</code>\\n\"+\n",
251 |        "     \"</div>\"}};\n",
252 |        "\n",
253 |        "  function display_loaded() {\n",
254 |        "    if (window.Bokeh !== undefined) {\n",
255 |        "      var el = document.getElementById(\"167b5046-3bf8-4af6-912d-c3c17cb8be74\");\n",
256 |        "      el.textContent = \"BokehJS \" + Bokeh.version + \" successfully loaded.\";\n",
257 |        "    } else if (Date.now() < window._bokeh_timeout) {\n",
258 |        "      setTimeout(display_loaded, 100)\n",
259 |        "    }\n",
260 |        "  }\n",
261 |        "\n",
262 |        "  function run_callbacks() {\n",
263 |        "    window._bokeh_onload_callbacks.forEach(function(callback) { callback() });\n",
264 |        "    delete window._bokeh_onload_callbacks\n",
265 |        "    console.info(\"Bokeh: all callbacks have finished\");\n",
266 |        "  }\n",
267 |        "\n",
268 |        "  function load_libs(js_urls, callback) {\n",
269 |        "    window._bokeh_onload_callbacks.push(callback);\n",
270 |        "    if (window._bokeh_is_loading > 0) {\n",
271 |        "      console.log(\"Bokeh: BokehJS is being loaded, scheduling callback at\", now());\n",
272 |        "      return null;\n",
273 |        "    }\n",
274 |        "    if (js_urls == null || js_urls.length === 0) {\n",
275 |        "      run_callbacks();\n",
276 |        "      return null;\n",
277 |        "    }\n",
278 |        "    console.log(\"Bokeh: BokehJS not loaded, scheduling load and callback at\", now());\n",
279 |        "    window._bokeh_is_loading = js_urls.length;\n",
280 |        "    for (var i = 0; i < js_urls.length; i++) {\n",
281 |        "      var url = js_urls[i];\n",
282 |        "      var s = document.createElement('script');\n",
283 |        "      s.src = url;\n",
284 |        "      s.async = false;\n",
285 |        "      s.onreadystatechange = s.onload = function() {\n",
286 |        "        window._bokeh_is_loading--;\n",
287 |        "        if (window._bokeh_is_loading === 0) {\n",
288 |        "          console.log(\"Bokeh: all BokehJS libraries loaded\");\n",
289 |        "          run_callbacks()\n",
290 |        "        }\n",
291 |        "      };\n",
292 |        "      s.onerror = function() {\n",
293 |        "        console.warn(\"failed to load library \" + url);\n",
294 |        "      };\n",
295 |        "      console.log(\"Bokeh: injecting script tag for BokehJS library: \", url);\n",
296 |        "      document.getElementsByTagName(\"head\")[0].appendChild(s);\n",
297 |        "    }\n",
298 |        "  };var element = document.getElementById(\"167b5046-3bf8-4af6-912d-c3c17cb8be74\");\n",
299 |        "  if (element == null) {\n",
300 |        "    console.log(\"Bokeh: ERROR: autoload.js configured with elementid '167b5046-3bf8-4af6-912d-c3c17cb8be74' but no matching script tag was found. \")\n",
301 |        "    return false;\n",
302 |        "  }\n",
303 |        "\n",
304 |        "  var js_urls = [\"https://cdn.pydata.org/bokeh/release/bokeh-0.12.5.min.js\", \"https://cdn.pydata.org/bokeh/release/bokeh-widgets-0.12.5.min.js\"];\n",
305 |        "\n",
306 |        "  var inline_js = [\n",
307 |        "    function(Bokeh) {\n",
308 |        "      Bokeh.set_log_level(\"info\");\n",
309 |        "    },\n",
310 |        "    \n",
311 |        "    function(Bokeh) {\n",
312 |        "      \n",
313 |        "    },\n",
314 |        "    \n",
315 |        "    function(Bokeh) {\n",
316 |        "      \n",
317 |        "      document.getElementById(\"167b5046-3bf8-4af6-912d-c3c17cb8be74\").textContent = \"BokehJS is loading...\";\n",
318 |        "    },\n",
319 |        "    function(Bokeh) {\n",
320 |        "      console.log(\"Bokeh: injecting CSS: https://cdn.pydata.org/bokeh/release/bokeh-0.12.5.min.css\");\n",
321 |        "      Bokeh.embed.inject_css(\"https://cdn.pydata.org/bokeh/release/bokeh-0.12.5.min.css\");\n",
322 |        "      console.log(\"Bokeh: injecting CSS: https://cdn.pydata.org/bokeh/release/bokeh-widgets-0.12.5.min.css\");\n",
323 |        "      Bokeh.embed.inject_css(\"https://cdn.pydata.org/bokeh/release/bokeh-widgets-0.12.5.min.css\");\n",
324 |        "    }\n",
325 |        "  ];\n",
326 |        "\n",
327 |        "  function run_inline_js() {\n",
328 |        "    \n",
329 |        "    if ((window.Bokeh !== undefined) || (force === true)) {\n",
330 |        "      for (var i = 0; i < inline_js.length; i++) {\n",
331 |        "        inline_js[i](window.Bokeh);\n",
332 |        "      }if (force === true) {\n",
333 |        "        display_loaded();\n",
334 |        "      }} else if (Date.now() < window._bokeh_timeout) {\n",
335 |        "      setTimeout(run_inline_js, 100);\n",
336 |        "    } else if (!window._bokeh_failed_load) {\n",
337 |        "      console.log(\"Bokeh: BokehJS failed to load within specified timeout.\");\n",
338 |        "      window._bokeh_failed_load = true;\n",
339 |        "    } else if (force !== true) {\n",
340 |        "      var cell = $(document.getElementById(\"167b5046-3bf8-4af6-912d-c3c17cb8be74\")).parents('.cell').data().cell;\n",
341 |        "      cell.output_area.append_execute_result(NB_LOAD_WARNING)\n",
342 |        "    }\n",
343 |        "\n",
344 |        "  }\n",
345 |        "\n",
346 |        "  if (window._bokeh_is_loading === 0) {\n",
347 |        "    console.log(\"Bokeh: BokehJS loaded, going straight to plotting\");\n",
348 |        "    run_inline_js();\n",
349 |        "  } else {\n",
350 |        "    load_libs(js_urls, function() {\n",
351 |        "      console.log(\"Bokeh: BokehJS plotting callback run at\", now());\n",
352 |        "      run_inline_js();\n",
353 |        "    });\n",
354 |        "  }\n",
355 |        "}(this));"
356 |       ]
357 |      },
358 |      "metadata": {},
359 |      "output_type": "display_data"
360 |     },
361 |     {
362 |      "data": {
363 |       "text/html": [
364 |        "\n",
365 |        "\n",
366 |        "    <div class=\"bk-root\">\n",
367 |        "        <div class=\"bk-plotdiv\" id=\"96de6e86-f9fd-4d6b-b45d-c87894b084f5\"></div>\n",
368 |        "    </div>\n",
369 |        "<script type=\"text/javascript\">\n",
370 |        "  \n",
371 |        "  (function(global) {\n",
372 |        "    function now() {\n",
373 |        "      return new Date();\n",
374 |        "    }\n",
375 |        "  \n",
376 |        "    var force = false;\n",
377 |        "  \n",
378 |        "    if (typeof (window._bokeh_onload_callbacks) === \"undefined\" || force === true) {\n",
379 |        "      window._bokeh_onload_callbacks = [];\n",
380 |        "      window._bokeh_is_loading = undefined;\n",
381 |        "    }\n",
382 |        "  \n",
383 |        "  \n",
384 |        "    \n",
385 |        "    if (typeof (window._bokeh_timeout) === \"undefined\" || force === true) {\n",
386 |        "      window._bokeh_timeout = Date.now() + 0;\n",
387 |        "      window._bokeh_failed_load = false;\n",
388 |        "    }\n",
389 |        "  \n",
390 |        "    var NB_LOAD_WARNING = {'data': {'text/html':\n",
391 |        "       \"<div style='background-color: #fdd'>\\n\"+\n",
392 |        "       \"<p>\\n\"+\n",
393 |        "       \"BokehJS does not appear to have successfully loaded. If loading BokehJS from CDN, this \\n\"+\n",
394 |        "       \"may be due to a slow or bad network connection. Possible fixes:\\n\"+\n",
395 |        "       \"</p>\\n\"+\n",
396 |        "       \"<ul>\\n\"+\n",
397 |        "       \"<li>re-rerun `output_notebook()` to attempt to load from CDN again, or</li>\\n\"+\n",
398 |        "       \"<li>use INLINE resources instead, as so:</li>\\n\"+\n",
399 |        "       \"</ul>\\n\"+\n",
400 |        "       \"<code>\\n\"+\n",
401 |        "       \"from bokeh.resources import INLINE\\n\"+\n",
402 |        "       \"output_notebook(resources=INLINE)\\n\"+\n",
403 |        "       \"</code>\\n\"+\n",
404 |        "       \"</div>\"}};\n",
405 |        "  \n",
406 |        "    function display_loaded() {\n",
407 |        "      if (window.Bokeh !== undefined) {\n",
408 |        "        var el = document.getElementById(\"96de6e86-f9fd-4d6b-b45d-c87894b084f5\");\n",
409 |        "        el.textContent = \"BokehJS \" + Bokeh.version + \" successfully loaded.\";\n",
410 |        "      } else if (Date.now() < window._bokeh_timeout) {\n",
411 |        "        setTimeout(display_loaded, 100)\n",
412 |        "      }\n",
413 |        "    }\n",
414 |        "  \n",
415 |        "    function run_callbacks() {\n",
416 |        "      window._bokeh_onload_callbacks.forEach(function(callback) { callback() });\n",
417 |        "      delete window._bokeh_onload_callbacks\n",
418 |        "      console.info(\"Bokeh: all callbacks have finished\");\n",
419 |        "    }\n",
420 |        "  \n",
421 |        "    function load_libs(js_urls, callback) {\n",
422 |        "      window._bokeh_onload_callbacks.push(callback);\n",
423 |        "      if (window._bokeh_is_loading > 0) {\n",
424 |        "        console.log(\"Bokeh: BokehJS is being loaded, scheduling callback at\", now());\n",
425 |        "        return null;\n",
426 |        "      }\n",
427 |        "      if (js_urls == null || js_urls.length === 0) {\n",
428 |        "        run_callbacks();\n",
429 |        "        return null;\n",
430 |        "      }\n",
431 |        "      console.log(\"Bokeh: BokehJS not loaded, scheduling load and callback at\", now());\n",
432 |        "      window._bokeh_is_loading = js_urls.length;\n",
433 |        "      for (var i = 0; i < js_urls.length; i++) {\n",
434 |        "        var url = js_urls[i];\n",
435 |        "        var s = document.createElement('script');\n",
436 |        "        s.src = url;\n",
437 |        "        s.async = false;\n",
438 |        "        s.onreadystatechange = s.onload = function() {\n",
439 |        "          window._bokeh_is_loading--;\n",
440 |        "          if (window._bokeh_is_loading === 0) {\n",
441 |        "            console.log(\"Bokeh: all BokehJS libraries loaded\");\n",
442 |        "            run_callbacks()\n",
443 |        "          }\n",
444 |        "        };\n",
445 |        "        s.onerror = function() {\n",
446 |        "          console.warn(\"failed to load library \" + url);\n",
447 |        "        };\n",
448 |        "        console.log(\"Bokeh: injecting script tag for BokehJS library: \", url);\n",
449 |        "        document.getElementsByTagName(\"head\")[0].appendChild(s);\n",
450 |        "      }\n",
451 |        "    };var element = document.getElementById(\"96de6e86-f9fd-4d6b-b45d-c87894b084f5\");\n",
452 |        "    if (element == null) {\n",
453 |        "      console.log(\"Bokeh: ERROR: autoload.js configured with elementid '96de6e86-f9fd-4d6b-b45d-c87894b084f5' but no matching script tag was found. \")\n",
454 |        "      return false;\n",
455 |        "    }\n",
456 |        "  \n",
457 |        "    var js_urls = [];\n",
458 |        "  \n",
459 |        "    var inline_js = [\n",
460 |        "      function(Bokeh) {\n",
461 |        "        (function() {\n",
462 |        "          var fn = function() {\n",
463 |        "            var docs_json = {\"6eec101d-8ffe-43e3-933a-7208100efa27\":{\"roots\":{\"references\":[{\"attributes\":{\"fill_alpha\":{\"value\":0.9},\"fill_color\":{\"value\":\"bisque\"},\"line_alpha\":{\"value\":0.9},\"line_color\":{\"value\":\"#1f77b4\"},\"line_width\":{\"value\":0},\"size\":{\"units\":\"screen\",\"value\":7},\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"247da18a-e13b-4a28-92f9-da345b1fbbe0\",\"type\":\"Circle\"},{\"attributes\":{\"callback\":null,\"column_names\":[\"x\",\"y\",\"index\",\"label\",\"preds\"],\"data\":{\"index\":[40,53,58,64,67,70,112,146,195,222,223,254,316,317,328,329,341,407,463,467],\"label\":[\"CPFSK_75KHz\",\"CPFSK_75KHz\",\"CPFSK_75KHz\",\"CPFSK_75KHz\",\"CPFSK_75KHz\",\"CPFSK_75KHz\",\"CPFSK_75KHz\",\"CPFSK_75KHz\",\"CPFSK_75KHz\",\"CPFSK_75KHz\",\"CPFSK_75KHz\",\"CPFSK_75KHz\",\"CPFSK_75KHz\",\"CPFSK_75KHz\",\"CPFSK_75KHz\",\"CPFSK_75KHz\",\"CPFSK_75KHz\",\"CPFSK_75KHz\",\"CPFSK_75KHz\",\"CPFSK_75KHz\"],\"preds\":{\"__ndarray__\":\"AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA==\",\"dtype\":\"float64\",\"shape\":[20]},\"x\":{\"__ndarray__\":\"i97EJSXCNkD4yg5pd+Q4QJ+/5NcVXBDAuo8REIO1OEBW0NMKZOk3QH3dalHIhTdAAvIE4kYPKcAE89fTKSURwO41tslljUBAhnTE0eGGOkAxgIxONlM/QGVQ8ha1djdA7+Duo0qfOkAZtKXmwFc2QNhRwpRepjpAANopQ692E8DQXYa0eiQtwBERMk6XJkBABGrC5G+TPkC9ED8gPrEswA==\",\"dtype\":\"float64\",\"shape\":[20]},\"y\":{\"__ndarray__\":\"VxFV6ciXGMBVYUxwWhANwKa4O8Gst+U/jnLUDG6/B8ADDVTS4hMowLyJduFYsCjA1YKWGgHRH8BPhzb7AibxP7ZvRtFCQhPA8wy1WGQlKsBtvqruBpsUwGQpIIkvJxDAdjcX25DZJcBUyWjYcPsTwBrmHEXFJSDAYtSt1msZ8b/yJbqLGK4kwFykxIGQ0PK/XnRuI9XN/r/hW66jxw4lwA==\",\"dtype\":\"float64\",\"shape\":[20]}}},\"id\":\"f8c9d04b-a086-4d36-8120-a4be61414696\",\"type\":\"ColumnDataSource\"},{\"attributes\":{\"fill_alpha\":{\"value\":0.9},\"fill_color\":{\"value\":\"crimson\"},\"line_alpha\":{\"value\":0.9},\"line_color\":{\"value\":\"#1f77b4\"},\"line_width\":{\"value\":0},\"size\":{\"units\":\"screen\",\"value\":7},\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"591e9b06-b40c-4bfd-ac36-70f4b0221d20\",\"type\":\"Circle\"},{\"attributes\":{\"bottom_units\":\"screen\",\"fill_alpha\":{\"value\":0.5},\"fill_color\":{\"value\":\"lightgrey\"},\"left_units\":\"screen\",\"level\":\"overlay\",\"line_alpha\":{\"value\":1.0},\"line_color\":{\"value\":\"black\"},\"line_dash\":[4,4],\"line_width\":{\"value\":2},\"plot\":null,\"render_mode\":\"css\",\"right_units\":\"screen\",\"top_units\":\"screen\"},\"id\":\"2d6a7223-8886-41d7-8d48-0eb498c0261b\",\"type\":\"BoxAnnotation\"},{\"attributes\":{\"overlay\":{\"id\":\"2d6a7223-8886-41d7-8d48-0eb498c0261b\",\"type\":\"BoxAnnotation\"},\"plot\":{\"id\":\"d34a9099-ec1e-4108-9d5c-fd70c22ab802\",\"subtype\":\"Figure\",\"type\":\"Plot\"}},\"id\":\"dc795aa2-156e-458d-ae4e-005147aae99b\",\"type\":\"BoxZoomTool\"},{\"attributes\":{\"callback\":null,\"column_names\":[\"x\",\"y\",\"index\",\"label\",\"preds\"],\"data\":{\"index\":[7,36,61,93,106,137,152,172,202,208,218,245,252,253,273,306,338,364,413,453],\"label\":[\"FM_NB\",\"FM_NB\",\"FM_NB\",\"FM_NB\",\"FM_NB\",\"FM_NB\",\"FM_NB\",\"FM_NB\",\"FM_NB\",\"FM_NB\",\"FM_NB\",\"FM_NB\",\"FM_NB\",\"FM_NB\",\"FM_NB\",\"FM_NB\",\"FM_NB\",\"FM_NB\",\"FM_NB\",\"FM_NB\"],\"preds\":{\"__ndarray__\":\"AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA==\",\"dtype\":\"float64\",\"shape\":[20]},\"x\":{\"__ndarray__\":\"lMot2hg1D8ByDh195XnCP8N6kVPHUAvAKqN6fiSP879VMG+TVjTEP4uMJLi0Mw3A5KXCq6eAGkDKN5DAceQZQKKtMZQuawhARkoYZOrqB0DFOFJDtQv4v9f3fWxKmAtAiDtyLGu/sT9j8QdqKU4HwJUDxQxUvPe/UBN0ru2R0D84E/C4HWP6P8BNI51dWBdA6QCXEfrmFkB6i/AX2OsQwA==\",\"dtype\":\"float64\",\"shape\":[20]},\"y\":{\"__ndarray__\":\"ughQLax3OkC72IlrFq05QNL+Jw+a5DpArfI1BT0ZN0CqIikZeto+QDkI6pcOsTlAmJc69H57O0DevTtSwws2QDzY6MU5XzpAMMf/aqVZN0ARSap+R7Y6QMXUiVWHgzpAdrluggHmPUC+g7FU7ok5QIUfPrHCXjdAH9NBvBC0OEAOoEq/BtI4QAamxs+b0DhAxyY+7uY2NkCtWkZzlNI6QA==\",\"dtype\":\"float64\",\"shape\":[20]}}},\"id\":\"69449fdf-1cb6-40cc-90df-9d091287d99c\",\"type\":\"ColumnDataSource\"},{\"attributes\":{\"callback\":null,\"column_names\":[\"x\",\"y\",\"index\",\"label\",\"preds\"],\"data\":{\"index\":[9,37,72,91,116,122,124,127,139,150,190,204,229,285,325,380,456,465,470,479],\"label\":[\"2FSK_75KHz\",\"2FSK_75KHz\",\"2FSK_75KHz\",\"2FSK_75KHz\",\"2FSK_75KHz\",\"2FSK_75KHz\",\"2FSK_75KHz\",\"2FSK_75KHz\",\"2FSK_75KHz\",\"2FSK_75KHz\",\"2FSK_75KHz\",\"2FSK_75KHz\",\"2FSK_75KHz\",\"2FSK_75KHz\",\"2FSK_75KHz\",\"2FSK_75KHz\",\"2FSK_75KHz\",\"2FSK_75KHz\",\"2FSK_75KHz\",\"2FSK_75KHz\"],\"preds\":{\"__ndarray__\":\"AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA==\",\"dtype\":\"float64\",\"shape\":[20]},\"x\":{\"__ndarray__\":\"oxpy/ohVPMAhSOQO1/c9wHwMaHOF9z3A3BXuBZtwOsC4R+2lA/w7wCz2Onc6OjvA/QhtvyogPMAvtp9CUMA+wF/UgkHJrjrAJ3VyLajoOMBrhOpnq8g6wGEQDpHDtDjAwAiO6/5qP8D/fyN1DD08wPKNvpdf6DjAhY39Nqf7PcDsYw83O6U9wBhygXyKFzzAWY6ZirhKPsDFk/MsMR06wA==\",\"dtype\":\"float64\",\"shape\":[20]},\"y\":{\"__ndarray__\":\"YOs3DAqKLUBarCjvFtAmQC2zV9Ln/C5AkQaSXSR4KEC3S0OZ6swwQKS2e0wcPzFA8BG1nkQqJkB7W52AV2MuQLBggZUQ6yZAFRanDjRqK0C2DaPTv3EtQMu41mV+oy5Ao92e8TvFKkD+bHExqrMrQBX6oS7deidAZNc/bq9SMUAeuufBpUQrQCGzVoLnGydAK9NKtl8vKUDnvNDqce4uQA==\",\"dtype\":\"float64\",\"shape\":[20]}}},\"id\":\"076cc2e5-c2c9-4aee-9a78-ec848c857b8a\",\"type\":\"ColumnDataSource\"},{\"attributes\":{\"callback\":null,\"column_names\":[\"x\",\"y\",\"index\",\"label\",\"preds\"],\"data\":{\"index\":[38,39,41,62,87,95,104,107,251,278,298,301,310,349,361,421,423,430,436,451],\"label\":[\"8PSK\",\"8PSK\",\"8PSK\",\"8PSK\",\"8PSK\",\"8PSK\",\"8PSK\",\"8PSK\",\"8PSK\",\"8PSK\",\"8PSK\",\"8PSK\",\"8PSK\",\"8PSK\",\"8PSK\",\"8PSK\",\"8PSK\",\"8PSK\",\"8PSK\",\"8PSK\"],\"preds\":{\"__ndarray__\":\"AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA==\",\"dtype\":\"float64\",\"shape\":[20]},\"x\":{\"__ndarray__\":\"BszgRERSE0Dq+qJ2jMEYwMVJfOCMASDAqNg+Ht8XAMD5r9aVhhQYwIoowHgfeA3AwMJSX5NIEsA0ahq9hc0nwORTi4lPV/6/pjjgMLJH/r9OsOKlrWkewFKbSZguCfU/xxnmj4ncGsBJ9b9T9Lr5vyglTbtkRrS/iApmCx0a0b/fjUAA+poUwPkT8k5tVwjAoevh8xL/EcCqr2PLvJAUwA==\",\"dtype\":\"float64\",\"shape\":[20]},\"y\":{\"__ndarray__\":\"trCUCPl4JcALWPOSLZQ0wAhK5tJMzCzAstWvY6bPLMDARQKltJQxwOoTrbszizHAhKnx2bLFMcDzY9g/0WAvwGNiopPymjbAVMuc4gH+KsC2edz/gl0wwFhNeca50SjAEojLlpxZNcCes/W5lFUwwEZAt3NCKTHAkYDSIYF0MsC00YbqHBUywG4VMWWraTTAkAITAvduL8B5ERqsXdIrwA==\",\"dtype\":\"float64\",\"shape\":[20]}}},\"id\":\"090e68c1-dc9f-4ed1-adc8-3dcc7611bc8f\",\"type\":\"ColumnDataSource\"},{\"attributes\":{\"fill_alpha\":{\"value\":0.9},\"fill_color\":{\"value\":\"aqua\"},\"line_alpha\":{\"value\":0.9},\"line_color\":{\"value\":\"#1f77b4\"},\"line_width\":{\"value\":0},\"size\":{\"units\":\"screen\",\"value\":7},\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"7abb3dff-c8c4-4fdc-af82-28b70185dcce\",\"type\":\"Circle\"},{\"attributes\":{\"fill_alpha\":{\"value\":0.9},\"fill_color\":{\"value\":\"black\"},\"line_alpha\":{\"value\":0.9},\"line_color\":{\"value\":\"#1f77b4\"},\"line_width\":{\"value\":0},\"size\":{\"units\":\"screen\",\"value\":7},\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"6ace1017-5ea5-484e-85f0-838d01570050\",\"type\":\"Circle\"},{\"attributes\":{\"fill_alpha\":{\"value\":0.9},\"fill_color\":{\"value\":\"blueviolet\"},\"line_alpha\":{\"value\":0.9},\"line_color\":{\"value\":\"#1f77b4\"},\"line_width\":{\"value\":0},\"size\":{\"units\":\"screen\",\"value\":7},\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"68cfec4a-9851-4564-91db-79b14d4527c1\",\"type\":\"Circle\"},{\"attributes\":{\"plot\":{\"id\":\"d34a9099-ec1e-4108-9d5c-fd70c22ab802\",\"subtype\":\"Figure\",\"type\":\"Plot\"}},\"id\":\"20bad771-8379-4cbb-bb24-be078834fc9a\",\"type\":\"WheelZoomTool\"},{\"attributes\":{\"plot\":{\"id\":\"d34a9099-ec1e-4108-9d5c-fd70c22ab802\",\"subtype\":\"Figure\",\"type\":\"Plot\"}},\"id\":\"5ec759fd-10f7-45ad-b57c-c5d732c2a587\",\"type\":\"ResetTool\"},{\"attributes\":{\"plot\":{\"id\":\"d34a9099-ec1e-4108-9d5c-fd70c22ab802\",\"subtype\":\"Figure\",\"type\":\"Plot\"}},\"id\":\"552f37b1-9c49-4003-8564-57f8ff8ef6cf\",\"type\":\"SaveTool\"},{\"attributes\":{},\"id\":\"6e58fe80-c549-45e4-bb5d-9d9fa6c70dae\",\"type\":\"BasicTickFormatter\"},{\"attributes\":{\"callback\":null,\"column_names\":[\"x\",\"y\",\"index\",\"label\",\"preds\"],\"data\":{\"index\":[6,30,114,121,151,167,168,178,198,303,324,344,366,408,411,431,434,447,450,460],\"label\":[\"CPFSK_5KHz\",\"CPFSK_5KHz\",\"CPFSK_5KHz\",\"CPFSK_5KHz\",\"CPFSK_5KHz\",\"CPFSK_5KHz\",\"CPFSK_5KHz\",\"CPFSK_5KHz\",\"CPFSK_5KHz\",\"CPFSK_5KHz\",\"CPFSK_5KHz\",\"CPFSK_5KHz\",\"CPFSK_5KHz\",\"CPFSK_5KHz\",\"CPFSK_5KHz\",\"CPFSK_5KHz\",\"CPFSK_5KHz\",\"CPFSK_5KHz\",\"CPFSK_5KHz\",\"CPFSK_5KHz\"],\"preds\":{\"__ndarray__\":\"AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA==\",\"dtype\":\"float64\",\"shape\":[20]},\"x\":{\"__ndarray__\":\"gh0NNKLtRUDDvH6MjPxBQJhwwCjXR0ZA6Cbjm3ZURECnHF3rxPhEQH5VcS4TOEFA6pe4WFynRECLbXcTfU9EQC52tDtCvURAvXq4zPvEQ0AlnNXhtjc6QLHLSHSOeUNA3LW/8wZkQkCx1t8TAEhCQFNFurMTMENAd1ZXAg85QkCFNPUo4zdDQMAJeY93KUNA8HFTC+IxQ0CVYw4MzV1DQA==\",\"dtype\":\"float64\",\"shape\":[20]},\"y\":{\"__ndarray__\":\"u288jljwQMCUiEobCqlAwBlDEDQ/HD/AdavdB0FdQMARqdOPAb84wGaBvlFsbD3AkQzo6TQtPcAQPA8V9lU8wAcBId124D7A4xjjFlXqQcCYkVLi0bE7wK3PcXdx4j7And1tt5sjQsDBHKOg4oBCwIVam+Pj4kLAKXkCEJOWQcBOBoSthaxAwAbl2KbC6TfAw7V//dEzP8BBt1z7+To6wA==\",\"dtype\":\"float64\",\"shape\":[20]}}},\"id\":\"6344b69a-a74c-42ec-8e26-253c965af26b\",\"type\":\"ColumnDataSource\"},{\"attributes\":{\"fill_alpha\":{\"value\":0.9},\"fill_color\":{\"value\":\"chartreuse\"},\"line_alpha\":{\"value\":0.9},\"line_color\":{\"value\":\"#1f77b4\"},\"line_width\":{\"value\":0},\"size\":{\"units\":\"screen\",\"value\":7},\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"3094ff71-a01c-41c2-9cca-befa2c20937d\",\"type\":\"Circle\"},{\"attributes\":{\"data_source\":{\"id\":\"076cc2e5-c2c9-4aee-9a78-ec848c857b8a\",\"type\":\"ColumnDataSource\"},\"glyph\":{\"id\":\"247da18a-e13b-4a28-92f9-da345b1fbbe0\",\"type\":\"Circle\"},\"hover_glyph\":null,\"muted_glyph\":null,\"name\":\"test\",\"nonselection_glyph\":{\"id\":\"7adc6ba0-98bd-4fa7-825c-afe09678f69e\",\"type\":\"Circle\"},\"selection_glyph\":null},\"id\":\"40723d19-5f23-4abc-83a0-e7383347d7b8\",\"type\":\"GlyphRenderer\"},{\"attributes\":{},\"id\":\"990a1bb6-7655-4d3e-b483-b57fabc6e7dd\",\"type\":\"BasicTickFormatter\"},{\"attributes\":{\"plot\":null,\"text\":\"\"},\"id\":\"a2f6ab30-dc22-465e-bd84-167d94c3f202\",\"type\":\"Title\"},{\"attributes\":{\"fill_alpha\":{\"value\":0.1},\"fill_color\":{\"value\":\"#1f77b4\"},\"line_alpha\":{\"value\":0.1},\"line_color\":{\"value\":\"#1f77b4\"},\"line_width\":{\"value\":0},\"size\":{\"units\":\"screen\",\"value\":7},\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"06be495b-ee26-418b-870d-7b9ea2bf354a\",\"type\":\"Circle\"},{\"attributes\":{\"fill_alpha\":{\"value\":0.1},\"fill_color\":{\"value\":\"#1f77b4\"},\"line_alpha\":{\"value\":0.1},\"line_color\":{\"value\":\"#1f77b4\"},\"line_width\":{\"value\":0},\"size\":{\"units\":\"screen\",\"value\":7},\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"9cabe5d1-ae3a-4f4a-a428-412a7554833f\",\"type\":\"Circle\"},{\"attributes\":{\"callback\":null,\"column_names\":[\"x\",\"y\",\"index\",\"label\",\"preds\"],\"data\":{\"index\":[0,52,76,77,79,101,103,173,205,242,246,247,269,326,334,348,356,398,433,471],\"label\":[\"FM_WB\",\"FM_WB\",\"FM_WB\",\"FM_WB\",\"FM_WB\",\"FM_WB\",\"FM_WB\",\"FM_WB\",\"FM_WB\",\"FM_WB\",\"FM_WB\",\"FM_WB\",\"FM_WB\",\"FM_WB\",\"FM_WB\",\"FM_WB\",\"FM_WB\",\"FM_WB\",\"FM_WB\",\"FM_WB\"],\"preds\":{\"__ndarray__\":\"AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA==\",\"dtype\":\"float64\",\"shape\":[20]},\"x\":{\"__ndarray__\":\"9Wr87vxsOkBPmt5G7HndP7jr9igS5gBAC6N54G4IOEAK2xtOrMs9QMVncdj90fu/ym6B+YqQNkDJ5jrY/cw8QLwdEgOi3ztACiolvHesOUBKyZT46JPgv9YFL8LKxwrAPq+CcraGN0DaXWQF4ynkP7TnpzpANTlAQ0VtH0CyEUCV7nUJTTw2QK5pJXtGTh1AvhYiWOkRIEB24q5rC4QEQA==\",\"dtype\":\"float64\",\"shape\":[20]},\"y\":{\"__ndarray__\":\"rMMZBss/QcAS1QLZEdYyQMyE7nGcpzRAg4QfzshzPsD1TvLfRJI+wFHlKbr6yz1A7+M6rQ2dOcAnP29JSAU6wL0DtePgWD3AJGKajWEBP8AU2PvZ1NQ0QP3zR1pc0jZAKiCyM/DpPcAha0NVS7A1QMC3wW/TkEDADBiKhelYPUAwcLE53WY/wGsEHy3DYjJApjoSpMUhMUDnhH56eFc+QA==\",\"dtype\":\"float64\",\"shape\":[20]}}},\"id\":\"0b945696-6129-4692-9567-b791f336be42\",\"type\":\"ColumnDataSource\"},{\"attributes\":{\"data_source\":{\"id\":\"ebfa2e7b-2cd1-4fdd-943e-2811bda9551d\",\"type\":\"ColumnDataSource\"},\"glyph\":{\"id\":\"7abb3dff-c8c4-4fdc-af82-28b70185dcce\",\"type\":\"Circle\"},\"hover_glyph\":null,\"muted_glyph\":null,\"name\":\"test\",\"nonselection_glyph\":{\"id\":\"8fb3eac3-a3f6-4d45-a21f-d2c5497cd303\",\"type\":\"Circle\"},\"selection_glyph\":null},\"id\":\"3a746e3b-b78f-4c3f-beb7-d66451c9c1cb\",\"type\":\"GlyphRenderer\"},{\"attributes\":{\"fill_alpha\":{\"value\":0.1},\"fill_color\":{\"value\":\"#1f77b4\"},\"line_alpha\":{\"value\":0.1},\"line_color\":{\"value\":\"#1f77b4\"},\"line_width\":{\"value\":0},\"size\":{\"units\":\"screen\",\"value\":7},\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"8fb3eac3-a3f6-4d45-a21f-d2c5497cd303\",\"type\":\"Circle\"},{\"attributes\":{\"fill_alpha\":{\"value\":0.1},\"fill_color\":{\"value\":\"#1f77b4\"},\"line_alpha\":{\"value\":0.1},\"line_color\":{\"value\":\"#1f77b4\"},\"line_width\":{\"value\":0},\"size\":{\"units\":\"screen\",\"value\":7},\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"7adc6ba0-98bd-4fa7-825c-afe09678f69e\",\"type\":\"Circle\"},{\"attributes\":{\"data_source\":{\"id\":\"f8c9d04b-a086-4d36-8120-a4be61414696\",\"type\":\"ColumnDataSource\"},\"glyph\":{\"id\":\"1b29b66a-fd5b-4c29-96f5-c52741d70f74\",\"type\":\"Circle\"},\"hover_glyph\":null,\"muted_glyph\":null,\"name\":\"test\",\"nonselection_glyph\":{\"id\":\"5406f111-027a-4923-a9be-da0fa0cf9e0d\",\"type\":\"Circle\"},\"selection_glyph\":null},\"id\":\"77f1d668-b3da-480f-a5b6-a4d3d0ea95a9\",\"type\":\"GlyphRenderer\"},{\"attributes\":{\"data_source\":{\"id\":\"d66dbe86-625a-4a81-bc73-6d7f9704330e\",\"type\":\"ColumnDataSource\"},\"glyph\":{\"id\":\"178de19f-7c34-493b-9dbd-d2f6687a15c4\",\"type\":\"Diamond\"},\"hover_glyph\":null,\"muted_glyph\":null,\"name\":\"test\",\"nonselection_glyph\":{\"id\":\"706f45a9-d930-4651-bbe1-73a1f4659a34\",\"type\":\"Diamond\"},\"selection_glyph\":null},\"id\":\"0a9516c1-21f9-4236-af63-9b4378ea3b33\",\"type\":\"GlyphRenderer\"},{\"attributes\":{\"fill_alpha\":{\"value\":0.9},\"fill_color\":{\"value\":\"burlywood\"},\"line_alpha\":{\"value\":0.9},\"line_color\":{\"value\":\"#1f77b4\"},\"line_width\":{\"value\":0},\"size\":{\"units\":\"screen\",\"value\":7},\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"69db5e34-5771-4d00-99dc-a795a4413d97\",\"type\":\"Circle\"},{\"attributes\":{\"data_source\":{\"id\":\"090e68c1-dc9f-4ed1-adc8-3dcc7611bc8f\",\"type\":\"ColumnDataSource\"},\"glyph\":{\"id\":\"6ace1017-5ea5-484e-85f0-838d01570050\",\"type\":\"Circle\"},\"hover_glyph\":null,\"muted_glyph\":null,\"name\":\"test\",\"nonselection_glyph\":{\"id\":\"06be495b-ee26-418b-870d-7b9ea2bf354a\",\"type\":\"Circle\"},\"selection_glyph\":null},\"id\":\"37391d63-06fe-4d4a-9886-8f7c29cb2452\",\"type\":\"GlyphRenderer\"},{\"attributes\":{\"fill_alpha\":{\"value\":0.1},\"fill_color\":{\"value\":\"#1f77b4\"},\"line_alpha\":{\"value\":0.1},\"line_color\":{\"value\":\"#1f77b4\"},\"line_width\":{\"value\":0},\"size\":{\"units\":\"screen\",\"value\":7},\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"517a1618-eca9-4267-9a82-3006261cbebc\",\"type\":\"Circle\"},{\"attributes\":{\"plot\":{\"id\":\"d34a9099-ec1e-4108-9d5c-fd70c22ab802\",\"subtype\":\"Figure\",\"type\":\"Plot\"}},\"id\":\"c6fdf9f2-eda1-455f-b2c2-e306beaa4468\",\"type\":\"PanTool\"},{\"attributes\":{\"callback\":null,\"column_names\":[\"x\",\"y\",\"index\",\"label\",\"preds\"],\"data\":{\"index\":[5,28,55,102,189,233,239,268,308,337,339,346,365,378,406,444,461,466,468,478],\"label\":[\"16PSK\",\"16PSK\",\"16PSK\",\"16PSK\",\"16PSK\",\"16PSK\",\"16PSK\",\"16PSK\",\"16PSK\",\"16PSK\",\"16PSK\",\"16PSK\",\"16PSK\",\"16PSK\",\"16PSK\",\"16PSK\",\"16PSK\",\"16PSK\",\"16PSK\",\"16PSK\"],\"preds\":{\"__ndarray__\":\"AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA==\",\"dtype\":\"float64\",\"shape\":[20]},\"x\":{\"__ndarray__\":\"su8CSOOkNUBoGYb9LTA3QEYKUkOe8jZATew9bVoNQECcITrdCBY4QBCJTj1mNUBAc+y0W+dOOUCi7jDpdZg8QCuTN9H1ZD9ALZtVrykNOkCh9TmDP3o+QAfsB/3xkTxAHYQ5kOdHMkAxV74iUQU7QFRXZFYQjzlAPNsciWoEN0Bjqn/+Ug44QPvcwg6TlTpAXLFDKSyNNUB8q/I4w3g+QA==\",\"dtype\":\"float64\",\"shape\":[20]},\"y\":{\"__ndarray__\":\"OntJpXRYREBS3l19XgpEQJMfPzz2JEFAyApoS6gSQUBnCvQCLP1BQAEUESdAcEFAeiDEsB9KQUC8oDVXEH48QKvS7GaFQ0BAmGJut3EVQkBnmPs3tqtDQAT+eUiVpUNAyWlPTdauQUCg3+bZm99CQMUGMHXuKENAUmWNCRR6QUAzBZavVodDQJ8mC+wUhzxAwAF1yTlBRUAzaic8T58+QA==\",\"dtype\":\"float64\",\"shape\":[20]}}},\"id\":\"ebfa2e7b-2cd1-4fdd-943e-2811bda9551d\",\"type\":\"ColumnDataSource\"},{\"attributes\":{\"fill_alpha\":{\"value\":0.1},\"fill_color\":{\"value\":\"#1f77b4\"},\"line_alpha\":{\"value\":0.1},\"line_color\":{\"value\":\"#1f77b4\"},\"line_width\":{\"value\":0},\"size\":{\"units\":\"screen\",\"value\":7},\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"a70cbcec-52e0-4948-a6dd-f39609e95178\",\"type\":\"Circle\"},{\"attributes\":{\"data_source\":{\"id\":\"419fa8d6-3576-4683-b352-728511a824e3\",\"type\":\"ColumnDataSource\"},\"glyph\":{\"id\":\"cc5b5f23-5c59-45ed-8d5a-388f0028ed82\",\"type\":\"Circle\"},\"hover_glyph\":null,\"muted_glyph\":null,\"name\":\"test\",\"nonselection_glyph\":{\"id\":\"43db22e2-25ec-419e-8584-676c262a69e6\",\"type\":\"Circle\"},\"selection_glyph\":null},\"id\":\"2852085b-a612-4f26-bfb0-d012d07596b3\",\"type\":\"GlyphRenderer\"},{\"attributes\":{\"data_source\":{\"id\":\"1e1123a7-4ce5-4676-99f6-0e240f7d814f\",\"type\":\"ColumnDataSource\"},\"glyph\":{\"id\":\"2448155b-db10-4458-9314-f7df7a85d63d\",\"type\":\"Circle\"},\"hover_glyph\":null,\"muted_glyph\":null,\"name\":\"test\",\"nonselection_glyph\":{\"id\":\"5bec655c-8bc4-4234-b740-61efae2b2a64\",\"type\":\"Circle\"},\"selection_glyph\":null},\"id\":\"d36a7eb1-8f11-42bb-b9e8-cd8bdb97c99a\",\"type\":\"GlyphRenderer\"},{\"attributes\":{\"callback\":null,\"column_names\":[\"x\",\"y\",\"index\",\"label\",\"preds\"],\"data\":{\"index\":[3,20,50,71,81,96,97,125,134,141,166,174,203,250,293,363,391,400,440,476],\"label\":[\"APSK32_c34\",\"APSK32_c34\",\"APSK32_c34\",\"APSK32_c34\",\"APSK32_c34\",\"APSK32_c34\",\"APSK32_c34\",\"APSK32_c34\",\"APSK32_c34\",\"APSK32_c34\",\"APSK32_c34\",\"APSK32_c34\",\"APSK32_c34\",\"APSK32_c34\",\"APSK32_c34\",\"APSK32_c34\",\"APSK32_c34\",\"APSK32_c34\",\"APSK32_c34\",\"APSK32_c34\"],\"preds\":{\"__ndarray__\":\"AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA==\",\"dtype\":\"float64\",\"shape\":[20]},\"x\":{\"__ndarray__\":\"aBLx7KQuQsD8CNazBOFBwC+B5wVMET/A4nuGtmadQsCd5k9Ja7lDwDeGvM9xgUPAU+SIwz0dQcDa4U6OMY5AwG3eiXOlqULAtpd84fs6QsCGytrVR8tCwLiw2yOjN0PA5LMeY1TOQsCwsm1wBhRBwGdQwMrI0UDAYjUDCcc4RMBtkLWz9LJCwJMkG0J4kUPA0ShLEEFBRcDeM5RY/HpDwA==\",\"dtype\":\"float64\",\"shape\":[20]},\"y\":{\"__ndarray__\":\"+ZfKW8LnMcCcEX8o0w4wwNwQ6gVu2zPAJR3dFViTMcBhRSH4Ny80wDrBgO6HPTTAQ4pJv4wDMcC+iZbNj5AowGzKrT+d/TLA5HDsh382OcDg1R/9vEUvwA04WxS6HzbAdZL2/oL5LcA9ENb62dI0wCZP55h0dDHALdzCwtOvMsCXnQIQ8UQ0wIemVCLD2zPAH7R4rkPVL8BQGV5B0IwvwA==\",\"dtype\":\"float64\",\"shape\":[20]}}},\"id\":\"6c574c1c-d55a-424f-aaa9-3da58bb023c5\",\"type\":\"ColumnDataSource\"},{\"attributes\":{\"data_source\":{\"id\":\"0b945696-6129-4692-9567-b791f336be42\",\"type\":\"ColumnDataSource\"},\"glyph\":{\"id\":\"591e9b06-b40c-4bfd-ac36-70f4b0221d20\",\"type\":\"Circle\"},\"hover_glyph\":null,\"muted_glyph\":null,\"name\":\"test\",\"nonselection_glyph\":{\"id\":\"517a1618-eca9-4267-9a82-3006261cbebc\",\"type\":\"Circle\"},\"selection_glyph\":null},\"id\":\"60ee0b92-1a69-4704-bcac-cef1ae7849b2\",\"type\":\"GlyphRenderer\"},{\"attributes\":{\"data_source\":{\"id\":\"d990bae9-1554-4949-ab19-4300da115f9b\",\"type\":\"ColumnDataSource\"},\"glyph\":{\"id\":\"cc0b8570-3aad-40f9-af73-553ff2db4680\",\"type\":\"Circle\"},\"hover_glyph\":null,\"muted_glyph\":null,\"name\":\"test\",\"nonselection_glyph\":{\"id\":\"f2968167-d25a-4775-aa55-8b3bc27ca172\",\"type\":\"Circle\"},\"selection_glyph\":null},\"id\":\"fd790ea3-a64d-4b74-91ed-ff0dd52bca07\",\"type\":\"GlyphRenderer\"},{\"attributes\":{\"fill_alpha\":{\"value\":0.1},\"fill_color\":{\"value\":\"#1f77b4\"},\"line_alpha\":{\"value\":0.1},\"line_color\":{\"value\":\"#1f77b4\"},\"line_width\":{\"value\":0},\"size\":{\"units\":\"screen\",\"value\":7},\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"3491f700-4b6f-4804-83a0-b95aad188a61\",\"type\":\"Circle\"},{\"attributes\":{\"data_source\":{\"id\":\"03009945-ca02-4536-88fd-4f98f3350e39\",\"type\":\"ColumnDataSource\"},\"glyph\":{\"id\":\"9e5fd5e7-35a8-435e-aa77-0289f10d677c\",\"type\":\"Circle\"},\"hover_glyph\":null,\"muted_glyph\":null,\"name\":\"test\",\"nonselection_glyph\":{\"id\":\"5cf4b258-99cb-422e-b45a-9f83ed30cd96\",\"type\":\"Circle\"},\"selection_glyph\":null},\"id\":\"d6d127e4-eb3c-4687-b6f7-5ba37094cbea\",\"type\":\"GlyphRenderer\"},{\"attributes\":{\"callback\":null,\"column_names\":[\"x\",\"y\",\"index\",\"label\",\"preds\"],\"data\":{\"index\":[15,22,68,89,123,128,144,148,154,184,193,224,270,275,283,359,382,388,404,438],\"label\":[\"QPSK\",\"QPSK\",\"QPSK\",\"QPSK\",\"QPSK\",\"QPSK\",\"QPSK\",\"QPSK\",\"QPSK\",\"QPSK\",\"QPSK\",\"QPSK\",\"QPSK\",\"QPSK\",\"QPSK\",\"QPSK\",\"QPSK\",\"QPSK\",\"QPSK\",\"QPSK\"],\"preds\":{\"__ndarray__\":\"AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA==\",\"dtype\":\"float64\",\"shape\":[20]},\"x\":{\"__ndarray__\":\"1EFNxnXtJEBirH/wf1sxQKavgQ9vyjBAFSjarceiK0ANAFLZ1YEtQAcw8sc2sx1AKCCfREw1NUC49ubUyXofQLLY9kX6kilAFQlGNyOQMEBhP8f2DzUzQHS7Qh/oPS9A03y/1Rb1J0DKoiOGvzUwQJquL2IxRidA8n8vF3r0MkCoVf00+VYzQCq/qL+2LDNA88uTnt9yJkBQfFzD1+MvQA==\",\"dtype\":\"float64\",\"shape\":[20]},\"y\":{\"__ndarray__\":\"qETWTl9fF0DLQ6G0IQshQOcdg7iQDxtAlNkYAjEdJEB6fPVxgsMnQGZRF3SRlAdASYvpcvYfGkCtJ2k5X2QgQC0ZPgEnRRZAEo6rWXKTKUCrkaAmHRAoQDeA9K0xViFAmNq8sF1wEUBkDZXGt6gNQMwg+MnvSyVAKLKayDJAEkCn1cqlaTIZQKKaaYih2RlAmPR5WQUAI0CaAN+zR30VQA==\",\"dtype\":\"float64\",\"shape\":[20]}}},\"id\":\"ea9a81dd-c504-47ab-a823-bc29597c6f54\",\"type\":\"ColumnDataSource\"},{\"attributes\":{\"fill_alpha\":{\"value\":0.9},\"fill_color\":{\"value\":\"brown\"},\"line_alpha\":{\"value\":0.9},\"line_color\":{\"value\":\"#1f77b4\"},\"line_width\":{\"value\":0},\"size\":{\"units\":\"screen\",\"value\":7},\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"cc5b5f23-5c59-45ed-8d5a-388f0028ed82\",\"type\":\"Circle\"},{\"attributes\":{\"callback\":null,\"column_names\":[\"x\",\"y\",\"index\",\"label\",\"preds\"],\"data\":{\"index\":[86,90,99,118,119,138,183,186,294,295,302,309,340,351,419,427,445,458,469,474],\"label\":[\"PI4QPSK\",\"PI4QPSK\",\"PI4QPSK\",\"PI4QPSK\",\"PI4QPSK\",\"PI4QPSK\",\"PI4QPSK\",\"PI4QPSK\",\"PI4QPSK\",\"PI4QPSK\",\"PI4QPSK\",\"PI4QPSK\",\"PI4QPSK\",\"PI4QPSK\",\"PI4QPSK\",\"PI4QPSK\",\"PI4QPSK\",\"PI4QPSK\",\"PI4QPSK\",\"PI4QPSK\"],\"preds\":{\"__ndarray__\":\"AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA==\",\"dtype\":\"float64\",\"shape\":[20]},\"x\":{\"__ndarray__\":\"0NY9iHWtJsDUvS/dRdQtwBURw+ePrC7A39pxZ6auMMCb03cwUps4wJj4LAXAvSbAcmZMmMFoKsCn/CItNPYgwDbevPu27CnAWDvkn2t3LMD8z5rc8/MnwDywYsE/EyfAgD0PFcgBLcAbQCEMaW0swBkyhCFhBDDALmS13f89McBWdvUqXyopwJPldzaXkSPAJGTgSxMIL8CBEcVk34gxwA==\",\"dtype\":\"float64\",\"shape\":[20]},\"y\":{\"__ndarray__\":\"9oIpgW09JkAy2BxM7vcHQFClHLqGTde/jMs2oguAEEDH3SsxHBoRQL0FF+5ymihAJG03cqzqHUBXhapOBa0OQNQ4u45Hui5AQ3EPXKs8I0DiYKQza4cLQDwAHx+okuA/NW0D++YiGUBor67Dktz2PzqrIA9lZhZAzNjOLGKI9D+0e+qSpL4pQPwVa9JgHB9AuQgFEJXxIkDWBc9qdFkaQA==\",\"dtype\":\"float64\",\"shape\":[20]}}},\"id\":\"d990bae9-1554-4949-ab19-4300da115f9b\",\"type\":\"ColumnDataSource\"},{\"attributes\":{\"data_source\":{\"id\":\"69449fdf-1cb6-40cc-90df-9d091287d99c\",\"type\":\"ColumnDataSource\"},\"glyph\":{\"id\":\"48af2826-8c9a-4f04-a655-d664411f7f1b\",\"type\":\"Circle\"},\"hover_glyph\":null,\"muted_glyph\":null,\"name\":\"test\",\"nonselection_glyph\":{\"id\":\"d7bb8037-09f4-4987-94c8-6b4636af9a20\",\"type\":\"Circle\"},\"selection_glyph\":null},\"id\":\"27517319-3b54-4154-b120-fd831911d7be\",\"type\":\"GlyphRenderer\"},{\"attributes\":{\"fill_alpha\":{\"value\":0.1},\"fill_color\":{\"value\":\"#1f77b4\"},\"line_alpha\":{\"value\":0.1},\"line_color\":{\"value\":\"#1f77b4\"},\"line_width\":{\"value\":0},\"size\":{\"units\":\"screen\",\"value\":7},\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"43db22e2-25ec-419e-8584-676c262a69e6\",\"type\":\"Circle\"},{\"attributes\":{\"data_source\":{\"id\":\"91f9e4b5-0061-480e-bb69-ab32dd8363de\",\"type\":\"ColumnDataSource\"},\"glyph\":{\"id\":\"68cfec4a-9851-4564-91db-79b14d4527c1\",\"type\":\"Circle\"},\"hover_glyph\":null,\"muted_glyph\":null,\"name\":\"test\",\"nonselection_glyph\":{\"id\":\"9cabe5d1-ae3a-4f4a-a428-412a7554833f\",\"type\":\"Circle\"},\"selection_glyph\":null},\"id\":\"79479eff-55de-456f-84ca-5d4c23fca017\",\"type\":\"GlyphRenderer\"},{\"attributes\":{\"fill_alpha\":{\"value\":0.9},\"fill_color\":{\"value\":\"cadetblue\"},\"line_alpha\":{\"value\":0.9},\"line_color\":{\"value\":\"#1f77b4\"},\"line_width\":{\"value\":0},\"size\":{\"units\":\"screen\",\"value\":7},\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"2448155b-db10-4458-9314-f7df7a85d63d\",\"type\":\"Circle\"},{\"attributes\":{\"callback\":null,\"column_names\":[\"x\",\"y\",\"index\",\"label\",\"preds\"],\"data\":{\"index\":[23,34,63,75,100,110,200,210,221,225,234,235,265,321,332,358,360,396,425,441],\"label\":[\"APSK16_c34\",\"APSK16_c34\",\"APSK16_c34\",\"APSK16_c34\",\"APSK16_c34\",\"APSK16_c34\",\"APSK16_c34\",\"APSK16_c34\",\"APSK16_c34\",\"APSK16_c34\",\"APSK16_c34\",\"APSK16_c34\",\"APSK16_c34\",\"APSK16_c34\",\"APSK16_c34\",\"APSK16_c34\",\"APSK16_c34\",\"APSK16_c34\",\"APSK16_c34\",\"APSK16_c34\"],\"preds\":{\"__ndarray__\":\"AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA==\",\"dtype\":\"float64\",\"shape\":[20]},\"x\":{\"__ndarray__\":\"iveiO1MMO8B4m0e4fxw0wFXf43XHaDvA/e63QQMOPsClCI3iQfM7wM2LoRcR+znA9+FWVnMEPsCShSrXyrE7wHUZ1OIlvzXAVRyEOeAKNsBx1mQQMQk1wA/Qz0f/0zzAjAFiDNywOMA0eLRKQPU6wFre6GAoeznAXiXAOCrKPMCv/9A9u9M4wOUMkB6GMj3A+WhnETnqOsASSoKEbT82wA==\",\"dtype\":\"float64\",\"shape\":[20]},\"y\":{\"__ndarray__\":\"T/cOle1RQECgBMMGQH1BQCIEjFVccUBAkfhhimHuQUB8kXQgprg9QGQ9jNwtrkJAxUmDpxEEQUCkXCqje89DQMn/3nptB0JAYmKdK9bsQEDrUtvGLHI/QHlzK1J5WkJAtTpNAuJjQEDCmzyzRU1CQN4svsySe0FA0k6xXV2YQkDF/3Hy8/88QBxY5PEKzz5Ac7Sl5ghyQEAR2XbefaFCQA==\",\"dtype\":\"float64\",\"shape\":[20]}}},\"id\":\"419fa8d6-3576-4683-b352-728511a824e3\",\"type\":\"ColumnDataSource\"},{\"attributes\":{\"callback\":null,\"column_names\":[\"x\",\"y\",\"index\",\"label\",\"preds\"],\"data\":{\"index\":[14,24,66,83,133,136,163,171,185,209,261,282,284,288,315,336,369,403,429,473],\"label\":[\"BPSK\",\"BPSK\",\"BPSK\",\"BPSK\",\"BPSK\",\"BPSK\",\"BPSK\",\"BPSK\",\"BPSK\",\"BPSK\",\"BPSK\",\"BPSK\",\"BPSK\",\"BPSK\",\"BPSK\",\"BPSK\",\"BPSK\",\"BPSK\",\"BPSK\",\"BPSK\"],\"preds\":{\"__ndarray__\":\"AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA==\",\"dtype\":\"float64\",\"shape\":[20]},\"x\":{\"__ndarray__\":\"zkleM7wIMMCsMNSJ0eoswHHsYffezi7A5GgGeRU7MMC0cCeLDlAvwGflhqSuSSrAh97HljX8LsDlg9kSIGQrwBuJEkPmvi/AAPeTqOOqKcD9gNhDVZkrwKNhdMRoQjDAPytLl1sKKMCKm6hM0aEwwHAuqjWiqjHAJd6NfyyILcCqssukkQsqwBgJzJ600TTAMiC39cnXLsAeJGIoixwywA==\",\"dtype\":\"float64\",\"shape\":[20]},\"y\":{\"__ndarray__\":\"7Zt0+7clQMBtep0Fp3pAwPKrOfjcPD7AgyrTzTgWQMAisAQYc99AwLU/i/JvCjrATCl5E0pqO8Btv9/yuFU9wIYSa2SG2jzATydXu3hOQMB7RKZZwlhBwIhZm9n6kjzAVch4Go0VQcCqHuqZCXZAwAJBsrT3HkDAfq3ZVzdsP8Cldlp2+Ps6wALHMsyb60HAUwTOT9iTQMCoXkCvlRBCwA==\",\"dtype\":\"float64\",\"shape\":[20]}}},\"id\":\"1e1123a7-4ce5-4676-99f6-0e240f7d814f\",\"type\":\"ColumnDataSource\"},{\"attributes\":{\"fill_alpha\":{\"value\":0.1},\"fill_color\":{\"value\":\"#1f77b4\"},\"line_alpha\":{\"value\":0.1},\"line_color\":{\"value\":\"#1f77b4\"},\"line_width\":{\"value\":0},\"size\":{\"units\":\"screen\",\"value\":7},\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"5406f111-027a-4923-a9be-da0fa0cf9e0d\",\"type\":\"Circle\"},{\"attributes\":{\"callback\":null,\"column_names\":[\"x\",\"y\",\"index\",\"label\",\"preds\"],\"data\":{\"index\":[2,25,65,73,80,84,126,156,177,188,220,256,276,368,379,384,409,415,422,428],\"label\":[\"AM_DSB\",\"AM_DSB\",\"AM_DSB\",\"AM_DSB\",\"AM_DSB\",\"AM_DSB\",\"AM_DSB\",\"AM_DSB\",\"AM_DSB\",\"AM_DSB\",\"AM_DSB\",\"AM_DSB\",\"AM_DSB\",\"AM_DSB\",\"AM_DSB\",\"AM_DSB\",\"AM_DSB\",\"AM_DSB\",\"AM_DSB\",\"AM_DSB\"],\"preds\":{\"__ndarray__\":\"AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA==\",\"dtype\":\"float64\",\"shape\":[20]},\"x\":{\"__ndarray__\":\"jw8PlLbeJ8Bx4zFCrc8rwCDUG67jbiHAj/3+ITMXLcAurUUA1TEwwPExsPtKYiHA9qalBhgp+r/KkhCIX+wqwFbj5gC1QijAEryxYBlhJsDejzPehYUswITuCHJUojDAW4+plyYsJcATRdtwmoE2wK7WT+vpQCrAmLyETjNDHMAt59g+/JsswPx9Ju/FOSnAU56cyC3cMMC7XzOeHcslwA==\",\"dtype\":\"float64\",\"shape\":[20]},\"y\":{\"__ndarray__\":\"pMn35RzjN0AqJ2EjscwzQKLXPVvQYzNAyxhbNdChOkDeMkmZ40k4QNkNAl/CkDlAutjcUysbQkAtovMMXxU/QA+88E3/NTJAtmCsVgAWO0Bo6x63rBM+QGcIepIgvzFAEJxNpwIjNECIY3yhEjE5QIRfeUC3aTxAQJ1LYyrjPUBl+1MwcWs7QBrf8DDj+TlAsYCivOAaOEBs2jQxpxA9QA==\",\"dtype\":\"float64\",\"shape\":[20]}}},\"id\":\"d15c89b5-0fdd-4471-a9d2-9ff55e458eeb\",\"type\":\"ColumnDataSource\"},{\"attributes\":{\"data_source\":{\"id\":\"6c574c1c-d55a-424f-aaa9-3da58bb023c5\",\"type\":\"ColumnDataSource\"},\"glyph\":{\"id\":\"69db5e34-5771-4d00-99dc-a795a4413d97\",\"type\":\"Circle\"},\"hover_glyph\":null,\"muted_glyph\":null,\"name\":\"test\",\"nonselection_glyph\":{\"id\":\"a70cbcec-52e0-4948-a6dd-f39609e95178\",\"type\":\"Circle\"},\"selection_glyph\":null},\"id\":\"9c4bb6b8-1898-4265-9576-42114b42591f\",\"type\":\"GlyphRenderer\"},{\"attributes\":{\"fill_alpha\":{\"value\":0.9},\"fill_color\":{\"value\":\"chocolate\"},\"line_alpha\":{\"value\":0.9},\"line_color\":{\"value\":\"#1f77b4\"},\"line_width\":{\"value\":0},\"size\":{\"units\":\"screen\",\"value\":7},\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"1b29b66a-fd5b-4c29-96f5-c52741d70f74\",\"type\":\"Circle\"},{\"attributes\":{\"fill_alpha\":{\"value\":0.9},\"fill_color\":{\"value\":\"plum\"},\"line_alpha\":{\"value\":0.9},\"line_color\":{\"value\":\"#1f77b4\"},\"line_width\":{\"value\":0},\"size\":{\"units\":\"screen\",\"value\":7},\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"a8f6c77b-6754-469a-ae0d-2960d7ad8c6f\",\"type\":\"Diamond\"},{\"attributes\":{\"fill_alpha\":{\"value\":0.1},\"fill_color\":{\"value\":\"#1f77b4\"},\"line_alpha\":{\"value\":0.1},\"line_color\":{\"value\":\"#1f77b4\"},\"line_width\":{\"value\":0},\"size\":{\"units\":\"screen\",\"value\":7},\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"5bec655c-8bc4-4234-b740-61efae2b2a64\",\"type\":\"Circle\"},{\"attributes\":{\"data_source\":{\"id\":\"6344b69a-a74c-42ec-8e26-253c965af26b\",\"type\":\"ColumnDataSource\"},\"glyph\":{\"id\":\"3094ff71-a01c-41c2-9cca-befa2c20937d\",\"type\":\"Circle\"},\"hover_glyph\":null,\"muted_glyph\":null,\"name\":\"test\",\"nonselection_glyph\":{\"id\":\"3491f700-4b6f-4804-83a0-b95aad188a61\",\"type\":\"Circle\"},\"selection_glyph\":null},\"id\":\"36f3f5dc-cb7d-4dc1-8153-59dd702d3966\",\"type\":\"GlyphRenderer\"},{\"attributes\":{\"callback\":null,\"column_names\":[\"x\",\"y\",\"index\",\"label\",\"preds\"],\"data\":{\"index\":[29,82,115,164,176,187,215,217,249,281,289,319,350,352,367,374,376,377,432,472],\"label\":[\"GMSK\",\"GMSK\",\"GMSK\",\"GMSK\",\"GMSK\",\"GMSK\",\"GMSK\",\"GMSK\",\"GMSK\",\"GMSK\",\"GMSK\",\"GMSK\",\"GMSK\",\"GMSK\",\"GMSK\",\"GMSK\",\"GMSK\",\"GMSK\",\"GMSK\",\"GMSK\"],\"preds\":{\"__ndarray__\":\"AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA==\",\"dtype\":\"float64\",\"shape\":[20]},\"x\":{\"__ndarray__\":\"wDVKXqfpA0Acf1jtufHev2onONImxfi/YOO+ZpQD3L/KbkyYD7oHwBuhzh+hgBPAPJJVnA+L1L+Fk3a44UoMwEhVw44wYwnA5rLizjo5/j+Lkx0kdHsQwDyZbEb25B3A2HS75R0J+j8Ojn9Ca0LmP4Fs8T7HnP6/N1j1Mo8hC8BztQaPQs8PwAB2inU7RBfAWIFs/j01/r8UVS98PVzgvw==\",\"dtype\":\"float64\",\"shape\":[20]},\"y\":{\"__ndarray__\":\"bKK6VC8S5T8p+kHuAgYewBaaJDgtiB3AMSZTcAoU+j/LzUr7AZ0ZwEMYqCXS1hHAlu5Z3ROaIsBX5Hu6m/QFwMKEGkiwdR3AvFyJ1nzoB8BCz4VXeQcewBP/SI9p5iLA9jmupbi8+78HpSBYAYQWwDslAdUHSRrAgh6F2Vh6I8AUXqRW7xAawJ+XcbGj1iPAJKY93TexI8AZpGwobKAQwA==\",\"dtype\":\"float64\",\"shape\":[20]}}},\"id\":\"eb028c6f-8e08-4956-afc7-a5e6121df2c2\",\"type\":\"ColumnDataSource\"},{\"attributes\":{\"callback\":null,\"column_names\":[\"x\",\"y\",\"index\",\"label\",\"preds\"],\"data\":{\"index\":[11,13,42,45,48,88,129,142,159,160,230,291,297,304,345,357,390,420,439,448],\"label\":[\"QAM32\",\"QAM32\",\"QAM32\",\"QAM32\",\"QAM32\",\"QAM32\",\"QAM32\",\"QAM32\",\"QAM32\",\"QAM32\",\"QAM32\",\"QAM32\",\"QAM32\",\"QAM32\",\"QAM32\",\"QAM32\",\"QAM32\",\"QAM32\",\"QAM32\",\"QAM32\"],\"preds\":{\"__ndarray__\":\"AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA==\",\"dtype\":\"float64\",\"shape\":[20]},\"x\":{\"__ndarray__\":\"xSpYHagdOcAFjEHH4vI4wISmVjjkizTAcgWpVQ7eN8COX0rYiF0+wD0rc7GXTTrAj+RpA39RPcAd0hjya5E3wLtIT3pXpTnAgLCQQ3+sPsC7Z4ZdvFo8wJ2OtfQ35TnAOqjXI+ZBOsBeawshbjo3wPyAeWjUKzzAgBZ3dcZtP8BYQ0vakKs4wJXlJy/8zjjAIcALrMCdNcAgSELxSmg3wA==\",\"dtype\":\"float64\",\"shape\":[20]},\"y\":{\"__ndarray__\":\"ulm0Wg4AGcCX5dbJH3L+v2+c7LsKTRXA9C2JfAEGEsDCJny6iUUOwL6MT69pvRDAdNFlkZ+cF8BYWO1odPj4v2I/mJWWPBLAcioEH5ZsD8CC3oS7RSgUwKBUzElXmum/2nQP13wKCcDqKNmemXMbwNtKAJiN8xLA9MZSafpc9r8Kzc+sBb4fwMrmLqUDDBPAp4qbbsksIcCP6GxbbZQjwA==\",\"dtype\":\"float64\",\"shape\":[20]}}},\"id\":\"7ece1dab-b634-4068-8f51-9c9e7e4a6133\",\"type\":\"ColumnDataSource\"},{\"attributes\":{\"callback\":null,\"plot\":{\"id\":\"d34a9099-ec1e-4108-9d5c-fd70c22ab802\",\"subtype\":\"Figure\",\"type\":\"Plot\"},\"tooltips\":[[\"p\",\"(@x, @y)\"],[\"label\",\"@label\"],[\"index\",\"@index\"]]},\"id\":\"de5cc6f8-50a9-4841-bc87-b6e884003ed9\",\"type\":\"HoverTool\"},{\"attributes\":{\"callback\":null,\"column_names\":[\"x\",\"y\",\"index\",\"label\",\"preds\"],\"data\":{\"index\":[85,92,120,149,170,191,211,236,267,287,322,323,327,354,370,371,381,387,399,402],\"label\":[\"AM_SSB\",\"AM_SSB\",\"AM_SSB\",\"AM_SSB\",\"AM_SSB\",\"AM_SSB\",\"AM_SSB\",\"AM_SSB\",\"AM_SSB\",\"AM_SSB\",\"AM_SSB\",\"AM_SSB\",\"AM_SSB\",\"AM_SSB\",\"AM_SSB\",\"AM_SSB\",\"AM_SSB\",\"AM_SSB\",\"AM_SSB\",\"AM_SSB\"],\"preds\":{\"__ndarray__\":\"AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA==\",\"dtype\":\"float64\",\"shape\":[20]},\"x\":{\"__ndarray__\":\"LBOy1beiLMBpOTr5olUnwMt8j56kIyvA3IOlpzquJMCZtNUhiZ8zwJUzjMZvLDTAZfsxz+LnMsCMFCySsocZwHgHBBNdejLATWUZd9R0MMCod5dHDTcjwE+DN+ud1TDApoFAJID0L8DTJb+9bg40wDoc+FTwoi3A6cwLTclILMBW5KgSeLsywDfHaZkfoBfAeSXprzwrI8BW8VD4V6gzwA==\",\"dtype\":\"float64\",\"shape\":[20]},\"y\":{\"__ndarray__\":\"Tgy+LRzhOEDWTnFP9uM1QHEcMlyNYzdAQtUBLh1DOUDr87YN+XQ8QLZCVW8FdCxAdgJt2RBOPEBQxoDIT+ZAQPKGcBmvJzNAkmaBoWCROkCiJwhpPTc6QOYSkvS0JTlA4DCbeKrSMUBfeRaLaJgsQKFeWn7piDdAHgic+Z94OUCYioz29xM2QKe6UopbvjVAM+ns3QWGPEAtJT7bZUY7QA==\",\"dtype\":\"float64\",\"shape\":[20]}}},\"id\":\"91f9e4b5-0061-480e-bb69-ab32dd8363de\",\"type\":\"ColumnDataSource\"},{\"attributes\":{\"fill_alpha\":{\"value\":0.9},\"fill_color\":{\"value\":\"blue\"},\"line_alpha\":{\"value\":0.9},\"line_color\":{\"value\":\"#1f77b4\"},\"line_width\":{\"value\":0},\"size\":{\"units\":\"screen\",\"value\":7},\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"34e51ecf-80e2-44d2-976a-eef230c38b06\",\"type\":\"Circle\"},{\"attributes\":{\"fill_alpha\":{\"value\":0.1},\"fill_color\":{\"value\":\"#1f77b4\"},\"line_alpha\":{\"value\":0.1},\"line_color\":{\"value\":\"#1f77b4\"},\"line_width\":{\"value\":0},\"size\":{\"units\":\"screen\",\"value\":7},\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"f2968167-d25a-4775-aa55-8b3bc27ca172\",\"type\":\"Circle\"},{\"attributes\":{\"data_source\":{\"id\":\"d15c89b5-0fdd-4471-a9d2-9ff55e458eeb\",\"type\":\"ColumnDataSource\"},\"glyph\":{\"id\":\"34e51ecf-80e2-44d2-976a-eef230c38b06\",\"type\":\"Circle\"},\"hover_glyph\":null,\"muted_glyph\":null,\"name\":\"test\",\"nonselection_glyph\":{\"id\":\"5c1450b0-d967-46c7-b389-c09f64cb5caa\",\"type\":\"Circle\"},\"selection_glyph\":null},\"id\":\"9e5cc56c-847a-4cee-9aef-f536a5b32b46\",\"type\":\"GlyphRenderer\"},{\"attributes\":{\"data_source\":{\"id\":\"eb028c6f-8e08-4956-afc7-a5e6121df2c2\",\"type\":\"ColumnDataSource\"},\"glyph\":{\"id\":\"3b345c7e-9886-41e7-bf27-5dcef5837dfa\",\"type\":\"Circle\"},\"hover_glyph\":null,\"muted_glyph\":null,\"name\":\"test\",\"nonselection_glyph\":{\"id\":\"da125f6b-7dc9-48ea-8209-a49230d118d0\",\"type\":\"Circle\"},\"selection_glyph\":null},\"id\":\"dc9e756c-ae1b-4842-830f-644eada008eb\",\"type\":\"GlyphRenderer\"},{\"attributes\":{\"data_source\":{\"id\":\"ea9a81dd-c504-47ab-a823-bc29597c6f54\",\"type\":\"ColumnDataSource\"},\"glyph\":{\"id\":\"a8f6c77b-6754-469a-ae0d-2960d7ad8c6f\",\"type\":\"Diamond\"},\"hover_glyph\":null,\"muted_glyph\":null,\"name\":\"test\",\"nonselection_glyph\":{\"id\":\"42ea9112-50b1-46ee-a26c-095eaae44ab5\",\"type\":\"Diamond\"},\"selection_glyph\":null},\"id\":\"dca1e20f-797d-425d-961e-94c779bfc46e\",\"type\":\"GlyphRenderer\"},{\"attributes\":{\"callback\":null,\"column_names\":[\"x\",\"y\",\"index\",\"label\",\"preds\"],\"data\":{\"index\":[12,46,78,135,161,180,182,226,227,237,238,263,274,292,305,320,412,418,449,454],\"label\":[\"MSK\",\"MSK\",\"MSK\",\"MSK\",\"MSK\",\"MSK\",\"MSK\",\"MSK\",\"MSK\",\"MSK\",\"MSK\",\"MSK\",\"MSK\",\"MSK\",\"MSK\",\"MSK\",\"MSK\",\"MSK\",\"MSK\",\"MSK\"],\"preds\":{\"__ndarray__\":\"AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA==\",\"dtype\":\"float64\",\"shape\":[20]},\"x\":{\"__ndarray__\":\"rMvaH8hWHUCo+VfOpbkUQN5baa37CiNAgOotyfgKIUDmdVRFp7kiQHZMZhC49ARAwEkiX0oaBEAJiG1XI4roP5PbN7/f6hdAltkfOTYvDkDc6x9Wf0oVQNiEXgj4iBRA8MqfGtU4IUDVC1UHj6ggQM2Y0v5AShtAKBwR3ZYMF0BWe8ZI63QnQL5MDCMGwhxAeCk0FHgEEkAx2zlPT5EdQA==\",\"dtype\":\"float64\",\"shape\":[20]},\"y\":{\"__ndarray__\":\"VZTP3KN/O8BwEkeBuRE+wAO0LWCF7zzAnTtIW04KO8CgJjEDuc05wC8dxq16fTvAZ16zhKbRPMAVqGvNrPk6wF2p0MPCnzrAwRUerHOMPsAEb8SRw7w6wDIG1n89VzvAhRhCcXCHPsDmXO17KWBAwOQ5Tg+0sTfAY6zB01f0PsDK42q8C0M6wEii0/auWzvAJnhQmKIyPcDyr4+mHac/wA==\",\"dtype\":\"float64\",\"shape\":[20]}}},\"id\":\"30e46b64-677b-49ea-a69b-6ff2d49add1e\",\"type\":\"ColumnDataSource\"},{\"attributes\":{\"fill_alpha\":{\"value\":0.1},\"fill_color\":{\"value\":\"#1f77b4\"},\"line_alpha\":{\"value\":0.1},\"line_color\":{\"value\":\"#1f77b4\"},\"line_width\":{\"value\":0},\"size\":{\"units\":\"screen\",\"value\":7},\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"5c1450b0-d967-46c7-b389-c09f64cb5caa\",\"type\":\"Circle\"},{\"attributes\":{\"data_source\":{\"id\":\"711eeb62-025b-4040-a084-a55688a828cf\",\"type\":\"ColumnDataSource\"},\"glyph\":{\"id\":\"dabaa085-2fc3-44e6-9c45-aa41106f39cc\",\"type\":\"Circle\"},\"hover_glyph\":null,\"muted_glyph\":null,\"name\":\"test\",\"nonselection_glyph\":{\"id\":\"6c9d0aaf-3ca8-490a-9999-15a41f59fe96\",\"type\":\"Circle\"},\"selection_glyph\":null},\"id\":\"02c4c72a-5b48-4c30-8493-5165e39ad145\",\"type\":\"GlyphRenderer\"},{\"attributes\":{\"fill_alpha\":{\"value\":0.1},\"fill_color\":{\"value\":\"#1f77b4\"},\"line_alpha\":{\"value\":0.1},\"line_color\":{\"value\":\"#1f77b4\"},\"line_width\":{\"value\":0},\"size\":{\"units\":\"screen\",\"value\":7},\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"b0bfd2df-5e36-4ae8-8859-dca06dae860c\",\"type\":\"Circle\"},{\"attributes\":{\"callback\":null,\"column_names\":[\"x\",\"y\",\"index\",\"label\",\"preds\"],\"data\":{\"index\":[56,109,117,192,197,206,216,277,290,318,333,343,347,353,362,389,395,417,455,477],\"label\":[\"QAM64\",\"QAM64\",\"QAM64\",\"QAM64\",\"QAM64\",\"QAM64\",\"QAM64\",\"QAM64\",\"QAM64\",\"QAM64\",\"QAM64\",\"QAM64\",\"QAM64\",\"QAM64\",\"QAM64\",\"QAM64\",\"QAM64\",\"QAM64\",\"QAM64\",\"QAM64\"],\"preds\":{\"__ndarray__\":\"AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA==\",\"dtype\":\"float64\",\"shape\":[20]},\"x\":{\"__ndarray__\":\"1SAgJGvYNsCWfpNj0t8ywEnsHJN0RTTAkBpIHiy6NMA6AUX0DXc4wGojX5kEMDPAi1/3inkoNMBK2WwfxrI1wFgRN/mWPDfAlPQXQP4zOsDR/mi0jWM2wJIwuZBotTXA3+I6qPvLN8DPbrFsbnQ0wAqVzne7BzLAsx0jhKsHN8A/y44O24YywK6vT6Gi7DjAdz/zJwJ8MsDGyioLMj82wA==\",\"dtype\":\"float64\",\"shape\":[20]},\"y\":{\"__ndarray__\":\"vfIDoRESNMD29ozRRroxwEfa1j8lNy3AOnduWdPGL8CqyD+BEJIzwAFPNDYxEjPA3WHMLGTdMMDor52mIdcuwJCjCOwg2SnAsRp+cJCBNMBZz64CBeYvwMIyLYY2IjPAxAZio1U5MsBxC2+durkywHJ9gpa/BzTAzrG12mdAN8DYn+JYBmcwwCmK9uXCTzDA5mxU70JvM8ChQpTrS/E0wA==\",\"dtype\":\"float64\",\"shape\":[20]}}},\"id\":\"d66dbe86-625a-4a81-bc73-6d7f9704330e\",\"type\":\"ColumnDataSource\"},{\"attributes\":{\"fill_alpha\":{\"value\":0.9},\"fill_color\":{\"value\":\"darkgray\"},\"line_alpha\":{\"value\":0.9},\"line_color\":{\"value\":\"#1f77b4\"},\"line_width\":{\"value\":0},\"size\":{\"units\":\"screen\",\"value\":7},\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"3b345c7e-9886-41e7-bf27-5dcef5837dfa\",\"type\":\"Circle\"},{\"attributes\":{\"fill_alpha\":{\"value\":0.1},\"fill_color\":{\"value\":\"#1f77b4\"},\"line_alpha\":{\"value\":0.1},\"line_color\":{\"value\":\"#1f77b4\"},\"line_width\":{\"value\":0},\"size\":{\"units\":\"screen\",\"value\":7},\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"da125f6b-7dc9-48ea-8209-a49230d118d0\",\"type\":\"Circle\"},{\"attributes\":{\"data_source\":{\"id\":\"1ef1db29-0eb0-46eb-abcc-44d49575e858\",\"type\":\"ColumnDataSource\"},\"glyph\":{\"id\":\"4b637bac-79ab-4d74-84d1-dba43bcf1176\",\"type\":\"Diamond\"},\"hover_glyph\":null,\"muted_glyph\":null,\"name\":\"test\",\"nonselection_glyph\":{\"id\":\"e9d9e749-e428-4705-ac14-71f4781d6201\",\"type\":\"Diamond\"},\"selection_glyph\":null},\"id\":\"73f31d66-7fb8-473d-a00e-196c826f6e0d\",\"type\":\"GlyphRenderer\"},{\"attributes\":{\"data_source\":{\"id\":\"30e46b64-677b-49ea-a69b-6ff2d49add1e\",\"type\":\"ColumnDataSource\"},\"glyph\":{\"id\":\"3eb31487-8945-4f13-b0ac-40811f858849\",\"type\":\"Circle\"},\"hover_glyph\":null,\"muted_glyph\":null,\"name\":\"test\",\"nonselection_glyph\":{\"id\":\"51e527d6-6f15-4f4d-a421-2b6d423dbc82\",\"type\":\"Circle\"},\"selection_glyph\":null},\"id\":\"56b0eb63-365b-4b5b-8e11-7b0c0c0ac546\",\"type\":\"GlyphRenderer\"},{\"attributes\":{\"callback\":null,\"column_names\":[\"x\",\"y\",\"index\",\"label\",\"preds\"],\"data\":{\"index\":[32,35,44,49,143,201,207,212,228,241,259,279,300,311,312,313,383,410,414,424],\"label\":[\"OQPSK\",\"OQPSK\",\"OQPSK\",\"OQPSK\",\"OQPSK\",\"OQPSK\",\"OQPSK\",\"OQPSK\",\"OQPSK\",\"OQPSK\",\"OQPSK\",\"OQPSK\",\"OQPSK\",\"OQPSK\",\"OQPSK\",\"OQPSK\",\"OQPSK\",\"OQPSK\",\"OQPSK\",\"OQPSK\"],\"preds\":{\"__ndarray__\":\"AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA==\",\"dtype\":\"float64\",\"shape\":[20]},\"x\":{\"__ndarray__\":\"Kmsr2PIMAsAZw5WHAgsTQECCV0cUCw7AhsDdBbDSBMDqZcJhXG/2v4lz5N57A+0/R9YUXs3C579ButX5BPgHQPWOnb0IlRDAqiSkx+hl1z8FZwGVbWEVwGA8eTtGrAfAPnL8XWq5BMCFwzJa7PL8PzXHwjZ7afi/Oc4vd74TDsCdRyVaylf+P/mSR2esDQjAZLogAlXc4r8uEi54vUoCQA==\",\"dtype\":\"float64\",\"shape\":[20]},\"y\":{\"__ndarray__\":\"/CwPIJRsJUCusipFZV4qQOOa6dudYS5AMtsayIiaJUBL9+arifMvQGG/cxDCbyxAwG6XOZABKEBUwIl6xqcvQJLXZR425iVAo3QVuZm2J0AfQSLpnc4pQDqCKj4BLylAAwZRgoK0GkAejsamagIlQDz2BxgxNiFAvln88/5rMUDVRZNAeosqQAVjVuOs6CZA6v2NNBbEKkCU5xQimEctQA==\",\"dtype\":\"float64\",\"shape\":[20]}}},\"id\":\"03009945-ca02-4536-88fd-4f98f3350e39\",\"type\":\"ColumnDataSource\"},{\"attributes\":{\"fill_alpha\":{\"value\":0.9},\"fill_color\":{\"value\":\"cornflowerblue\"},\"line_alpha\":{\"value\":0.9},\"line_color\":{\"value\":\"#1f77b4\"},\"line_width\":{\"value\":0},\"size\":{\"units\":\"screen\",\"value\":7},\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"48af2826-8c9a-4f04-a655-d664411f7f1b\",\"type\":\"Circle\"},{\"attributes\":{\"callback\":null,\"column_names\":[\"x\",\"y\",\"index\",\"label\",\"preds\"],\"data\":{\"index\":[1,17,18,47,51,54,98,111,153,157,232,257,262,266,271,272,314,375,397,459],\"label\":[\"2FSK_5KHz\",\"2FSK_5KHz\",\"2FSK_5KHz\",\"2FSK_5KHz\",\"2FSK_5KHz\",\"2FSK_5KHz\",\"2FSK_5KHz\",\"2FSK_5KHz\",\"2FSK_5KHz\",\"2FSK_5KHz\",\"2FSK_5KHz\",\"2FSK_5KHz\",\"2FSK_5KHz\",\"2FSK_5KHz\",\"2FSK_5KHz\",\"2FSK_5KHz\",\"2FSK_5KHz\",\"2FSK_5KHz\",\"2FSK_5KHz\",\"2FSK_5KHz\"],\"preds\":{\"__ndarray__\":\"AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA==\",\"dtype\":\"float64\",\"shape\":[20]},\"x\":{\"__ndarray__\":\"8QxhCiF9RECb91hfnNlCQH5bTVDRr0FAQGyo13j4QUAe2uQk2DhAQIHm42S8d0NAIiLhvm+RQUBxpmiKvDc+QJZrW360rEFA39UArtZhPkBBOMc/BO0/QB9VWAUDE0RA8ONsbDtZQ0D3Fp+a7X1CQFYxy85kMURASSEZYbeVQUArwB1DGEBCQFEk8HyF+T9AgLib+q+dQEBoEzRV9GlAQA==\",\"dtype\":\"float64\",\"shape\":[20]},\"y\":{\"__ndarray__\":\"0h+42P4XMcDPBDujuFszwKgqi0MEUjTArKGA9hTBLcBPQdQ3eGM0wKARYxIYjC3Al0BWO92QMsAJ39E7X2QswAbKBP6IdC/Apwvm/s7wMsBVxvZPFxExwDrDTTdIKTDAx6cJ4rxGMsA03GU7ZYArwOXLcYE49DHAAEDFcuD3LMBAYLJ2p7AvwFprWabicSrAzkifcXoNMMBkWnC5Pr0ywA==\",\"dtype\":\"float64\",\"shape\":[20]}}},\"id\":\"64ba833d-d627-4b48-b703-cc640882d04f\",\"type\":\"ColumnDataSource\"},{\"attributes\":{\"data_source\":{\"id\":\"7ece1dab-b634-4068-8f51-9c9e7e4a6133\",\"type\":\"ColumnDataSource\"},\"glyph\":{\"id\":\"1382fc82-d38d-4ef5-b327-aea9fde9a523\",\"type\":\"Diamond\"},\"hover_glyph\":null,\"muted_glyph\":null,\"name\":\"test\",\"nonselection_glyph\":{\"id\":\"cd689c23-32b7-4551-9e2f-e25eb1748265\",\"type\":\"Diamond\"},\"selection_glyph\":null},\"id\":\"e804f784-46ae-4832-a6fb-80ddd226e634\",\"type\":\"GlyphRenderer\"},{\"attributes\":{\"data_source\":{\"id\":\"e8730493-7839-4bed-bb54-25455b9194ea\",\"type\":\"ColumnDataSource\"},\"glyph\":{\"id\":\"f95ced6b-7ac0-42c4-8667-e5a788dcbb92\",\"type\":\"Circle\"},\"hover_glyph\":null,\"muted_glyph\":null,\"name\":\"test\",\"nonselection_glyph\":{\"id\":\"b0bfd2df-5e36-4ae8-8859-dca06dae860c\",\"type\":\"Circle\"},\"selection_glyph\":null},\"id\":\"63fcdcc6-d417-408d-a269-fde70a7c8e2f\",\"type\":\"GlyphRenderer\"},{\"attributes\":{\"fill_alpha\":{\"value\":0.1},\"fill_color\":{\"value\":\"#1f77b4\"},\"line_alpha\":{\"value\":0.1},\"line_color\":{\"value\":\"#1f77b4\"},\"line_width\":{\"value\":0},\"size\":{\"units\":\"screen\",\"value\":7},\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"5cf4b258-99cb-422e-b45a-9f83ed30cd96\",\"type\":\"Circle\"},{\"attributes\":{\"fill_alpha\":{\"value\":0.1},\"fill_color\":{\"value\":\"#1f77b4\"},\"line_alpha\":{\"value\":0.1},\"line_color\":{\"value\":\"#1f77b4\"},\"line_width\":{\"value\":0},\"size\":{\"units\":\"screen\",\"value\":7},\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"6c9d0aaf-3ca8-490a-9999-15a41f59fe96\",\"type\":\"Circle\"},{\"attributes\":{\"fill_alpha\":{\"value\":0.1},\"fill_color\":{\"value\":\"#1f77b4\"},\"line_alpha\":{\"value\":0.1},\"line_color\":{\"value\":\"#1f77b4\"},\"line_width\":{\"value\":0},\"size\":{\"units\":\"screen\",\"value\":7},\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"51e527d6-6f15-4f4d-a421-2b6d423dbc82\",\"type\":\"Circle\"},{\"attributes\":{\"fill_alpha\":{\"value\":0.1},\"fill_color\":{\"value\":\"#1f77b4\"},\"line_alpha\":{\"value\":0.1},\"line_color\":{\"value\":\"#1f77b4\"},\"line_width\":{\"value\":0},\"size\":{\"units\":\"screen\",\"value\":7},\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"cd689c23-32b7-4551-9e2f-e25eb1748265\",\"type\":\"Diamond\"},{\"attributes\":{\"data_source\":{\"id\":\"64ba833d-d627-4b48-b703-cc640882d04f\",\"type\":\"ColumnDataSource\"},\"glyph\":{\"id\":\"5dac7ba5-5cca-41b3-9301-c74d1327f8a3\",\"type\":\"Circle\"},\"hover_glyph\":null,\"muted_glyph\":null,\"name\":\"test\",\"nonselection_glyph\":{\"id\":\"463f050e-2cf9-4720-a5b7-904893ce4e06\",\"type\":\"Circle\"},\"selection_glyph\":null},\"id\":\"1ca30285-7fb6-4e50-89ff-d95ae2737110\",\"type\":\"GlyphRenderer\"},{\"attributes\":{\"data_source\":{\"id\":\"7cfa598f-5ddd-4626-8f61-8ba0ba059f5c\",\"type\":\"ColumnDataSource\"},\"glyph\":{\"id\":\"4f52835d-2739-4348-b4a1-9305372fc994\",\"type\":\"Circle\"},\"hover_glyph\":null,\"muted_glyph\":null,\"name\":\"test\",\"nonselection_glyph\":{\"id\":\"47d10dc1-2076-4311-a971-2362df502f11\",\"type\":\"Circle\"},\"selection_glyph\":null},\"id\":\"011810b0-2852-4db3-a0b6-c9c97de30a49\",\"type\":\"GlyphRenderer\"},{\"attributes\":{\"fill_alpha\":{\"value\":0.9},\"fill_color\":{\"value\":\"lightblue\"},\"line_alpha\":{\"value\":0.9},\"line_color\":{\"value\":\"#1f77b4\"},\"line_width\":{\"value\":0},\"size\":{\"units\":\"screen\",\"value\":7},\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"1382fc82-d38d-4ef5-b327-aea9fde9a523\",\"type\":\"Diamond\"},{\"attributes\":{\"formatter\":{\"id\":\"6e58fe80-c549-45e4-bb5d-9d9fa6c70dae\",\"type\":\"BasicTickFormatter\"},\"plot\":{\"id\":\"d34a9099-ec1e-4108-9d5c-fd70c22ab802\",\"subtype\":\"Figure\",\"type\":\"Plot\"},\"ticker\":{\"id\":\"5dd4fd1f-d4ff-49d9-a91b-593b6f35964e\",\"type\":\"BasicTicker\"}},\"id\":\"9e9016df-4cb8-4eef-88bb-8b6ed7d8bc14\",\"type\":\"LinearAxis\"},{\"attributes\":{\"callback\":null,\"column_names\":[\"x\",\"y\",\"index\",\"label\",\"preds\"],\"data\":{\"index\":[10,131,155,158,175,181,196,213,240,255,330,355,372,393,416,426,443,462,464,475],\"label\":[\"GFSK_75KHz\",\"GFSK_75KHz\",\"GFSK_75KHz\",\"GFSK_75KHz\",\"GFSK_75KHz\",\"GFSK_75KHz\",\"GFSK_75KHz\",\"GFSK_75KHz\",\"GFSK_75KHz\",\"GFSK_75KHz\",\"GFSK_75KHz\",\"GFSK_75KHz\",\"GFSK_75KHz\",\"GFSK_75KHz\",\"GFSK_75KHz\",\"GFSK_75KHz\",\"GFSK_75KHz\",\"GFSK_75KHz\",\"GFSK_75KHz\",\"GFSK_75KHz\"],\"preds\":{\"__ndarray__\":\"AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA==\",\"dtype\":\"float64\",\"shape\":[20]},\"x\":{\"__ndarray__\":\"M9MuYk1jKEBL4fddEK4nQFFDIPf0FTJAv2/F22OHNEDxEYrWCAUwQAz2sC0HBSxASKnBUKr+L0CgYXGpWygwQPQ6wTTeYCxAmam6311AIEDGlF9Piz0wQHLxisdleStAR0hmzimFMUBVcTGw7oI0QPhnY/7jPy1Au8JbTsr8MkDS7/GCgH4qQDehIOUY+CpAVbnuaD+3LUCc+fTQE100QA==\",\"dtype\":\"float64\",\"shape\":[20]},\"y\":{\"__ndarray__\":\"PgCAsWS/PkBbZrSJ3zs7QK3m7N9+HkBAvAfeaW30OkC4qscAXXE3QFwuQLHHDz1APzgWhi7YOkAmOCzG+j85QNBe6nfjED1Az2qGgKeaPUAbgqwwe2k5QAmwLGQQbjtA7acFAs3VPEBv9B7M4vc5QH+Ok91fWz1AWmbdWnwVO0BLHySTiqQ5QFqEF/ugnz5Ap+oYuofpP0B8fDgqtvo5QA==\",\"dtype\":\"float64\",\"shape\":[20]}}},\"id\":\"711eeb62-025b-4040-a084-a55688a828cf\",\"type\":\"ColumnDataSource\"},{\"attributes\":{\"callback\":null,\"column_names\":[\"x\",\"y\",\"index\",\"label\",\"preds\"],\"data\":{\"index\":[8,21,26,43,69,94,105,108,132,145,147,194,199,219,243,248,280,373,405,442],\"label\":[\"NOISE\",\"NOISE\",\"NOISE\",\"NOISE\",\"NOISE\",\"NOISE\",\"NOISE\",\"NOISE\",\"NOISE\",\"NOISE\",\"NOISE\",\"NOISE\",\"NOISE\",\"NOISE\",\"NOISE\",\"NOISE\",\"NOISE\",\"NOISE\",\"NOISE\",\"NOISE\"],\"preds\":{\"__ndarray__\":\"AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA==\",\"dtype\":\"float64\",\"shape\":[20]},\"x\":{\"__ndarray__\":\"C5BSID9tG8AiWhDjon0SwAk8933g/hjA+JC0YSld/78n4SvXgC0AQMErHxUcayPAL0cDdj+MG8AVTiFeCEUEwLUrE+Xc0SnAgNYx2vtMxb9ESkQv8EYXwI90swNHrxvA6uz337/gHMC8wTaY5nEXwM6Arl37Cfi/KOxntL5K3L8pe+07JAAgwCwcTwMuCBPAFwCSDMgsBcAbPKuV1mwhwA==\",\"dtype\":\"float64\",\"shape\":[20]},\"y\":{\"__ndarray__\":\"JpsLggMoQ0A1Yg+BPgNDQN7o3Jhqf0JA4W4tF54AQ0AlpNBRhLRDQBDY0mW6f0FA6cwlXbb2QUDQp0nPUBdBQKJj4QJHW0JA7toAGOp4QkAkkSo5xu5DQGYhupLIIEFA25vZb250QkDupHQMhxlDQISikGjqUUBAxAVwbzVvQkC96Wd8dKJBQNMmvBMC6UBA8CjmfQ92QUCOdUxz7RlDQA==\",\"dtype\":\"float64\",\"shape\":[20]}}},\"id\":\"e8730493-7839-4bed-bb54-25455b9194ea\",\"type\":\"ColumnDataSource\"},{\"attributes\":{\"fill_alpha\":{\"value\":0.1},\"fill_color\":{\"value\":\"#1f77b4\"},\"line_alpha\":{\"value\":0.1},\"line_color\":{\"value\":\"#1f77b4\"},\"line_width\":{\"value\":0},\"size\":{\"units\":\"screen\",\"value\":7},\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"42ea9112-50b1-46ee-a26c-095eaae44ab5\",\"type\":\"Diamond\"},{\"attributes\":{\"fill_alpha\":{\"value\":0.1},\"fill_color\":{\"value\":\"#1f77b4\"},\"line_alpha\":{\"value\":0.1},\"line_color\":{\"value\":\"#1f77b4\"},\"line_width\":{\"value\":0},\"size\":{\"units\":\"screen\",\"value\":7},\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"463f050e-2cf9-4720-a5b7-904893ce4e06\",\"type\":\"Circle\"},{\"attributes\":{\"fill_alpha\":{\"value\":0.9},\"fill_color\":{\"value\":\"aquamarine\"},\"line_alpha\":{\"value\":0.9},\"line_color\":{\"value\":\"#1f77b4\"},\"line_width\":{\"value\":0},\"size\":{\"units\":\"screen\",\"value\":7},\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"5dac7ba5-5cca-41b3-9301-c74d1327f8a3\",\"type\":\"Circle\"},{\"attributes\":{\"fill_alpha\":{\"value\":0.9},\"fill_color\":{\"value\":\"magenta\"},\"line_alpha\":{\"value\":0.9},\"line_color\":{\"value\":\"#1f77b4\"},\"line_width\":{\"value\":0},\"size\":{\"units\":\"screen\",\"value\":7},\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"178de19f-7c34-493b-9dbd-d2f6687a15c4\",\"type\":\"Diamond\"},{\"attributes\":{\"fill_alpha\":{\"value\":0.9},\"fill_color\":{\"value\":\"darkgoldenrod\"},\"line_alpha\":{\"value\":0.9},\"line_color\":{\"value\":\"#1f77b4\"},\"line_width\":{\"value\":0},\"size\":{\"units\":\"screen\",\"value\":7},\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"dabaa085-2fc3-44e6-9c45-aa41106f39cc\",\"type\":\"Circle\"},{\"attributes\":{\"callback\":null,\"column_names\":[\"x\",\"y\",\"index\",\"label\",\"preds\"],\"data\":{\"index\":[4,19,27,33,60,169,179,214,231,244,258,296,299,335,385,386,401,435,446,452],\"label\":[\"QAM16\",\"QAM16\",\"QAM16\",\"QAM16\",\"QAM16\",\"QAM16\",\"QAM16\",\"QAM16\",\"QAM16\",\"QAM16\",\"QAM16\",\"QAM16\",\"QAM16\",\"QAM16\",\"QAM16\",\"QAM16\",\"QAM16\",\"QAM16\",\"QAM16\",\"QAM16\"],\"preds\":{\"__ndarray__\":\"AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA==\",\"dtype\":\"float64\",\"shape\":[20]},\"x\":{\"__ndarray__\":\"AU7/0MEeLEBBIPw1VvUgQDxeGfj/cyNALNAwuM0NIUCr3Ea5KB8lQCztdKMQKSlA9o8qij95JUC5nKD8g7crQBlHBRUe9ytAs7GeCDeNKEDU/34vBSsoQOd2Be6U0yVA0HoHzylrFUAm7PxSVgwwQFHs/YcFFytAmkFjOXYWGkAhLzsmNtEjQBfWBNeK5C1AcinmRCIaNEDjmzipbDguQA==\",\"dtype\":\"float64\",\"shape\":[20]},\"y\":{\"__ndarray__\":\"2YYQ1ZOGF8ABDE2Aj5MgwK7uoOKVUiTAK869iIeyEsAhTLgH8GMhwERmaHV/Xh/AYQMmo4ikJ8BtdI07ybMGwEt1otmYohHAqg5OIEiGGcBbbQiBHrYawIy8kgOMpxXAsOLb/X8fIMDej/oVeVUqwFHFKnXhsR7AyO3WIhj+IcD4nNu0ZJwgwPjI2Obs6ibAyAUDS0QhHMB0tAZwTtgKwA==\",\"dtype\":\"float64\",\"shape\":[20]}}},\"id\":\"1ef1db29-0eb0-46eb-abcc-44d49575e858\",\"type\":\"ColumnDataSource\"},{\"attributes\":{\"formatter\":{\"id\":\"990a1bb6-7655-4d3e-b483-b57fabc6e7dd\",\"type\":\"BasicTickFormatter\"},\"plot\":{\"id\":\"d34a9099-ec1e-4108-9d5c-fd70c22ab802\",\"subtype\":\"Figure\",\"type\":\"Plot\"},\"ticker\":{\"id\":\"cdcb5ea3-b859-4c00-9d43-5eb8ef63d516\",\"type\":\"BasicTicker\"}},\"id\":\"5ed46ed3-5594-4e3c-ba99-6b081e4792a0\",\"type\":\"LinearAxis\"},{\"attributes\":{\"callback\":null,\"column_names\":[\"x\",\"y\",\"index\",\"label\",\"preds\"],\"data\":{\"index\":[16,31,57,59,74,113,130,140,162,165,260,264,286,307,331,342,392,394,437,457],\"label\":[\"GFSK_5KHz\",\"GFSK_5KHz\",\"GFSK_5KHz\",\"GFSK_5KHz\",\"GFSK_5KHz\",\"GFSK_5KHz\",\"GFSK_5KHz\",\"GFSK_5KHz\",\"GFSK_5KHz\",\"GFSK_5KHz\",\"GFSK_5KHz\",\"GFSK_5KHz\",\"GFSK_5KHz\",\"GFSK_5KHz\",\"GFSK_5KHz\",\"GFSK_5KHz\",\"GFSK_5KHz\",\"GFSK_5KHz\",\"GFSK_5KHz\",\"GFSK_5KHz\"],\"preds\":{\"__ndarray__\":\"AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA==\",\"dtype\":\"float64\",\"shape\":[20]},\"x\":{\"__ndarray__\":\"dixycNHKPkBRmO5buphBQHnYMJ9x10JAruxOp1f5P0AwaRxgB9xAQC0tuOo/GUNAKFWsm/O0RECF6hCjs0JFQCaPAamtGD5Av1qNqo+uRUApLUkp+mZBQFLl9tsMkkFAejq7DQbEQkAEmaI/bQJAQO9XkT1yEEJATipRItSeQkAcR8SyW2Q/QE2Qz46JpUFATzKtL3ZhREDkRFhqMJ9FQA==\",\"dtype\":\"float64\",\"shape\":[20]},\"y\":{\"__ndarray__\":\"ZqInBrIjQMByXxF7SIo5wFZaCXKgIDvAo2Ae+EksPMDPzO1+oaM7wOj+2hCo0z/AMO98TZAoQcDN/3PPkng7wC1593+6lEHAZTTy56EQPcBYKQcmeqpCwCmydw0kwjjAv09qB3GOPcD+1NIzOa85wFoxiOnJjT3A7LqUfH1jQsBYIxUT+Xw+wL58RUJKNjvAPeUh01QTQsAzdTgoqvY8wA==\",\"dtype\":\"float64\",\"shape\":[20]}}},\"id\":\"7cfa598f-5ddd-4626-8f61-8ba0ba059f5c\",\"type\":\"ColumnDataSource\"},{\"attributes\":{\"fill_alpha\":{\"value\":0.9},\"fill_color\":{\"value\":\"fuchsia\"},\"line_alpha\":{\"value\":0.9},\"line_color\":{\"value\":\"#1f77b4\"},\"line_width\":{\"value\":0},\"size\":{\"units\":\"screen\",\"value\":7},\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"cc0b8570-3aad-40f9-af73-553ff2db4680\",\"type\":\"Circle\"},{\"attributes\":{\"fill_alpha\":{\"value\":0.9},\"fill_color\":{\"value\":\"gold\"},\"line_alpha\":{\"value\":0.9},\"line_color\":{\"value\":\"#1f77b4\"},\"line_width\":{\"value\":0},\"size\":{\"units\":\"screen\",\"value\":7},\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"4b637bac-79ab-4d74-84d1-dba43bcf1176\",\"type\":\"Diamond\"},{\"attributes\":{\"fill_alpha\":{\"value\":0.9},\"fill_color\":{\"value\":\"darkorange\"},\"line_alpha\":{\"value\":0.9},\"line_color\":{\"value\":\"#1f77b4\"},\"line_width\":{\"value\":0},\"size\":{\"units\":\"screen\",\"value\":7},\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"f95ced6b-7ac0-42c4-8667-e5a788dcbb92\",\"type\":\"Circle\"},{\"attributes\":{\"fill_alpha\":{\"value\":0.1},\"fill_color\":{\"value\":\"#1f77b4\"},\"line_alpha\":{\"value\":0.1},\"line_color\":{\"value\":\"#1f77b4\"},\"line_width\":{\"value\":0},\"size\":{\"units\":\"screen\",\"value\":7},\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"706f45a9-d930-4651-bbe1-73a1f4659a34\",\"type\":\"Diamond\"},{\"attributes\":{\"below\":[{\"id\":\"9e9016df-4cb8-4eef-88bb-8b6ed7d8bc14\",\"type\":\"LinearAxis\"}],\"left\":[{\"id\":\"5ed46ed3-5594-4e3c-ba99-6b081e4792a0\",\"type\":\"LinearAxis\"}],\"plot_height\":700,\"plot_width\":700,\"renderers\":[{\"id\":\"9e9016df-4cb8-4eef-88bb-8b6ed7d8bc14\",\"type\":\"LinearAxis\"},{\"id\":\"b5a0aa66-00de-499c-8a0b-c06802df14d5\",\"type\":\"Grid\"},{\"id\":\"5ed46ed3-5594-4e3c-ba99-6b081e4792a0\",\"type\":\"LinearAxis\"},{\"id\":\"d9f71961-54e9-41a8-a579-4cec534a20e4\",\"type\":\"Grid\"},{\"id\":\"2d6a7223-8886-41d7-8d48-0eb498c0261b\",\"type\":\"BoxAnnotation\"},{\"id\":\"3a746e3b-b78f-4c3f-beb7-d66451c9c1cb\",\"type\":\"GlyphRenderer\"},{\"id\":\"1ca30285-7fb6-4e50-89ff-d95ae2737110\",\"type\":\"GlyphRenderer\"},{\"id\":\"40723d19-5f23-4abc-83a0-e7383347d7b8\",\"type\":\"GlyphRenderer\"},{\"id\":\"37391d63-06fe-4d4a-9886-8f7c29cb2452\",\"type\":\"GlyphRenderer\"},{\"id\":\"9e5cc56c-847a-4cee-9aef-f536a5b32b46\",\"type\":\"GlyphRenderer\"},{\"id\":\"79479eff-55de-456f-84ca-5d4c23fca017\",\"type\":\"GlyphRenderer\"},{\"id\":\"2852085b-a612-4f26-bfb0-d012d07596b3\",\"type\":\"GlyphRenderer\"},{\"id\":\"9c4bb6b8-1898-4265-9576-42114b42591f\",\"type\":\"GlyphRenderer\"},{\"id\":\"d36a7eb1-8f11-42bb-b9e8-cd8bdb97c99a\",\"type\":\"GlyphRenderer\"},{\"id\":\"36f3f5dc-cb7d-4dc1-8153-59dd702d3966\",\"type\":\"GlyphRenderer\"},{\"id\":\"77f1d668-b3da-480f-a5b6-a4d3d0ea95a9\",\"type\":\"GlyphRenderer\"},{\"id\":\"27517319-3b54-4154-b120-fd831911d7be\",\"type\":\"GlyphRenderer\"},{\"id\":\"60ee0b92-1a69-4704-bcac-cef1ae7849b2\",\"type\":\"GlyphRenderer\"},{\"id\":\"011810b0-2852-4db3-a0b6-c9c97de30a49\",\"type\":\"GlyphRenderer\"},{\"id\":\"02c4c72a-5b48-4c30-8493-5165e39ad145\",\"type\":\"GlyphRenderer\"},{\"id\":\"dc9e756c-ae1b-4842-830f-644eada008eb\",\"type\":\"GlyphRenderer\"},{\"id\":\"56b0eb63-365b-4b5b-8e11-7b0c0c0ac546\",\"type\":\"GlyphRenderer\"},{\"id\":\"63fcdcc6-d417-408d-a269-fde70a7c8e2f\",\"type\":\"GlyphRenderer\"},{\"id\":\"d6d127e4-eb3c-4687-b6f7-5ba37094cbea\",\"type\":\"GlyphRenderer\"},{\"id\":\"fd790ea3-a64d-4b74-91ed-ff0dd52bca07\",\"type\":\"GlyphRenderer\"},{\"id\":\"73f31d66-7fb8-473d-a00e-196c826f6e0d\",\"type\":\"GlyphRenderer\"},{\"id\":\"e804f784-46ae-4832-a6fb-80ddd226e634\",\"type\":\"GlyphRenderer\"},{\"id\":\"0a9516c1-21f9-4236-af63-9b4378ea3b33\",\"type\":\"GlyphRenderer\"},{\"id\":\"dca1e20f-797d-425d-961e-94c779bfc46e\",\"type\":\"GlyphRenderer\"}],\"title\":{\"id\":\"a2f6ab30-dc22-465e-bd84-167d94c3f202\",\"type\":\"Title\"},\"tool_events\":{\"id\":\"f25d7757-6f46-4c77-bb0c-39ef9760fc95\",\"type\":\"ToolEvents\"},\"toolbar\":{\"id\":\"85d3664f-3eef-4fbc-a523-d4324b3173e0\",\"type\":\"Toolbar\"},\"x_range\":{\"id\":\"976c0f27-a188-4c86-9d6b-92e712a2f5d2\",\"type\":\"DataRange1d\"},\"y_range\":{\"id\":\"cb588267-994d-4c25-bbe5-c67523494ffa\",\"type\":\"DataRange1d\"}},\"id\":\"d34a9099-ec1e-4108-9d5c-fd70c22ab802\",\"subtype\":\"Figure\",\"type\":\"Plot\"},{\"attributes\":{\"callback\":null},\"id\":\"cb588267-994d-4c25-bbe5-c67523494ffa\",\"type\":\"DataRange1d\"},{\"attributes\":{\"plot\":{\"id\":\"d34a9099-ec1e-4108-9d5c-fd70c22ab802\",\"subtype\":\"Figure\",\"type\":\"Plot\"},\"ticker\":{\"id\":\"5dd4fd1f-d4ff-49d9-a91b-593b6f35964e\",\"type\":\"BasicTicker\"}},\"id\":\"b5a0aa66-00de-499c-8a0b-c06802df14d5\",\"type\":\"Grid\"},{\"attributes\":{\"fill_alpha\":{\"value\":0.9},\"fill_color\":{\"value\":\"deeppink\"},\"line_alpha\":{\"value\":0.9},\"line_color\":{\"value\":\"#1f77b4\"},\"line_width\":{\"value\":0},\"size\":{\"units\":\"screen\",\"value\":7},\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"9e5fd5e7-35a8-435e-aa77-0289f10d677c\",\"type\":\"Circle\"},{\"attributes\":{\"fill_alpha\":{\"value\":0.1},\"fill_color\":{\"value\":\"#1f77b4\"},\"line_alpha\":{\"value\":0.1},\"line_color\":{\"value\":\"#1f77b4\"},\"line_width\":{\"value\":0},\"size\":{\"units\":\"screen\",\"value\":7},\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"e9d9e749-e428-4705-ac14-71f4781d6201\",\"type\":\"Diamond\"},{\"attributes\":{},\"id\":\"f25d7757-6f46-4c77-bb0c-39ef9760fc95\",\"type\":\"ToolEvents\"},{\"attributes\":{\"active_drag\":\"auto\",\"active_scroll\":\"auto\",\"active_tap\":\"auto\",\"tools\":[{\"id\":\"de5cc6f8-50a9-4841-bc87-b6e884003ed9\",\"type\":\"HoverTool\"},{\"id\":\"c6fdf9f2-eda1-455f-b2c2-e306beaa4468\",\"type\":\"PanTool\"},{\"id\":\"20bad771-8379-4cbb-bb24-be078834fc9a\",\"type\":\"WheelZoomTool\"},{\"id\":\"5ec759fd-10f7-45ad-b57c-c5d732c2a587\",\"type\":\"ResetTool\"},{\"id\":\"dc795aa2-156e-458d-ae4e-005147aae99b\",\"type\":\"BoxZoomTool\"},{\"id\":\"552f37b1-9c49-4003-8564-57f8ff8ef6cf\",\"type\":\"SaveTool\"}]},\"id\":\"85d3664f-3eef-4fbc-a523-d4324b3173e0\",\"type\":\"Toolbar\"},{\"attributes\":{},\"id\":\"5dd4fd1f-d4ff-49d9-a91b-593b6f35964e\",\"type\":\"BasicTicker\"},{\"attributes\":{\"fill_alpha\":{\"value\":0.1},\"fill_color\":{\"value\":\"#1f77b4\"},\"line_alpha\":{\"value\":0.1},\"line_color\":{\"value\":\"#1f77b4\"},\"line_width\":{\"value\":0},\"size\":{\"units\":\"screen\",\"value\":7},\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"47d10dc1-2076-4311-a971-2362df502f11\",\"type\":\"Circle\"},{\"attributes\":{\"fill_alpha\":{\"value\":0.9},\"fill_color\":{\"value\":\"darkgreen\"},\"line_alpha\":{\"value\":0.9},\"line_color\":{\"value\":\"#1f77b4\"},\"line_width\":{\"value\":0},\"size\":{\"units\":\"screen\",\"value\":7},\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"3eb31487-8945-4f13-b0ac-40811f858849\",\"type\":\"Circle\"},{\"attributes\":{\"fill_alpha\":{\"value\":0.1},\"fill_color\":{\"value\":\"#1f77b4\"},\"line_alpha\":{\"value\":0.1},\"line_color\":{\"value\":\"#1f77b4\"},\"line_width\":{\"value\":0},\"size\":{\"units\":\"screen\",\"value\":7},\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"d7bb8037-09f4-4987-94c8-6b4636af9a20\",\"type\":\"Circle\"},{\"attributes\":{},\"id\":\"cdcb5ea3-b859-4c00-9d43-5eb8ef63d516\",\"type\":\"BasicTicker\"},{\"attributes\":{\"dimension\":1,\"plot\":{\"id\":\"d34a9099-ec1e-4108-9d5c-fd70c22ab802\",\"subtype\":\"Figure\",\"type\":\"Plot\"},\"ticker\":{\"id\":\"cdcb5ea3-b859-4c00-9d43-5eb8ef63d516\",\"type\":\"BasicTicker\"}},\"id\":\"d9f71961-54e9-41a8-a579-4cec534a20e4\",\"type\":\"Grid\"},{\"attributes\":{\"callback\":null},\"id\":\"976c0f27-a188-4c86-9d6b-92e712a2f5d2\",\"type\":\"DataRange1d\"},{\"attributes\":{\"fill_alpha\":{\"value\":0.9},\"fill_color\":{\"value\":\"darkcyan\"},\"line_alpha\":{\"value\":0.9},\"line_color\":{\"value\":\"#1f77b4\"},\"line_width\":{\"value\":0},\"size\":{\"units\":\"screen\",\"value\":7},\"x\":{\"field\":\"x\"},\"y\":{\"field\":\"y\"}},\"id\":\"4f52835d-2739-4348-b4a1-9305372fc994\",\"type\":\"Circle\"}],\"root_ids\":[\"d34a9099-ec1e-4108-9d5c-fd70c22ab802\"]},\"title\":\"Bokeh Application\",\"version\":\"0.12.5\"}};\n",
464 |        "            var render_items = [{\"docid\":\"6eec101d-8ffe-43e3-933a-7208100efa27\",\"elementid\":\"96de6e86-f9fd-4d6b-b45d-c87894b084f5\",\"modelid\":\"d34a9099-ec1e-4108-9d5c-fd70c22ab802\"}];\n",
465 |        "            \n",
466 |        "            Bokeh.embed.embed_items(docs_json, render_items);\n",
467 |        "          };\n",
468 |        "          if (document.readyState != \"loading\") fn();\n",
469 |        "          else document.addEventListener(\"DOMContentLoaded\", fn);\n",
470 |        "        })();\n",
471 |        "      },\n",
472 |        "      function(Bokeh) {\n",
473 |        "      }\n",
474 |        "    ];\n",
475 |        "  \n",
476 |        "    function run_inline_js() {\n",
477 |        "      \n",
478 |        "      if ((window.Bokeh !== undefined) || (force === true)) {\n",
479 |        "        for (var i = 0; i < inline_js.length; i++) {\n",
480 |        "          inline_js[i](window.Bokeh);\n",
481 |        "        }if (force === true) {\n",
482 |        "          display_loaded();\n",
483 |        "        }} else if (Date.now() < window._bokeh_timeout) {\n",
484 |        "        setTimeout(run_inline_js, 100);\n",
485 |        "      } else if (!window._bokeh_failed_load) {\n",
486 |        "        console.log(\"Bokeh: BokehJS failed to load within specified timeout.\");\n",
487 |        "        window._bokeh_failed_load = true;\n",
488 |        "      } else if (force !== true) {\n",
489 |        "        var cell = $(document.getElementById(\"96de6e86-f9fd-4d6b-b45d-c87894b084f5\")).parents('.cell').data().cell;\n",
490 |        "        cell.output_area.append_execute_result(NB_LOAD_WARNING)\n",
491 |        "      }\n",
492 |        "  \n",
493 |        "    }\n",
494 |        "  \n",
495 |        "    if (window._bokeh_is_loading === 0) {\n",
496 |        "      console.log(\"Bokeh: BokehJS loaded, going straight to plotting\");\n",
497 |        "      run_inline_js();\n",
498 |        "    } else {\n",
499 |        "      load_libs(js_urls, function() {\n",
500 |        "        console.log(\"Bokeh: BokehJS plotting callback run at\", now());\n",
501 |        "        run_inline_js();\n",
502 |        "      });\n",
503 |        "    }\n",
504 |        "  }(this));\n",
505 |        "</script>"
506 |       ]
507 |      },
508 |      "metadata": {},
509 |      "output_type": "display_data"
510 |     }
511 |    ],
512 |    "source": [
513 |     "tsne_plot(model, data, labels, CLASSES_24, BOOKEH_COLORS, BOOKEH_SHAPES, layer_index=-2,legend = False, show_preds=False)"
514 |    ]
515 |   },
516 |   {
517 |    "cell_type": "code",
518 |    "execution_count": null,
519 |    "metadata": {
520 |     "collapsed": true
521 |    },
522 |    "outputs": [],
523 |    "source": []
524 |   }
525 |  ],
526 |  "metadata": {
527 |   "kernelspec": {
528 |    "display_name": "Python 3",
529 |    "language": "python",
530 |    "name": "python3"
531 |   },
532 |   "language_info": {
533 |    "codemirror_mode": {
534 |     "name": "ipython",
535 |     "version": 3
536 |    },
537 |    "file_extension": ".py",
538 |    "mimetype": "text/x-python",
539 |    "name": "python",
540 |    "nbconvert_exporter": "python",
541 |    "pygments_lexer": "ipython3",
542 |    "version": "3.6.1"
543 |   }
544 |  },
545 |  "nbformat": 4,
546 |  "nbformat_minor": 2
547 | }
548 | 


--------------------------------------------------------------------------------
/tsne_utils.py:
--------------------------------------------------------------------------------
  1 | from keras import backend as K
  2 | from keras.models import Model
  3 | from sklearn.manifold import TSNE
  4 | from sklearn.decomposition import PCA
  5 | 
  6 | from bokeh.plotting import figure, output_notebook, show, ColumnDataSource
  7 | from bokeh.models import HoverTool
  8 | 
  9 | import numpy as np
 10 | 
 11 | def tsne_model(model, data, pca_dim=50, tsne_dim=2, preds = False, layer_index = -2, random_state = None):
 12 |     """
 13 |     Does tsne reduction
 14 |     Parameters:
 15 |     model (keras model)
 16 |     data (np array): input data for the model
 17 |     layer_name: name of the output layer to do tsne on. 'None' will use 2nd to last layer, before final dense layer
 18 |  
 19 |     pca_dim (int): first dimensions are reduced using PCA to this number of dimensions
 20 |     tsne_dim (int): final desired dimension after doing tsne reduction
 21 |     
 22 |     Returns:
 23 |     np.array: Shape (data.shape[0], tsne_reduc); sample points in reduced space
 24 |     """
 25 |     
 26 |     layer_name = model.layers[layer_index].name
 27 |     
 28 |     intermediate_layer_model = Model(inputs=model.input,
 29 |                                  outputs=model.get_layer(layer_name).output)
 30 |     
 31 |     intermediate_output = intermediate_layer_model.predict(data)
 32 | 
 33 |     pca = PCA(n_components= pca_dim, random_state = random_state)    
 34 |     output_pca_reduced = pca.fit_transform(intermediate_output)
 35 |     
 36 |     tsne = TSNE(n_components=tsne_dim, random_state=random_state)
 37 |     intermediates_tsne = tsne.fit_transform(output_pca_reduced)
 38 |     
 39 |     if preds:
 40 |         return intermediates_tsne, model.predict(data)
 41 |     return intermediates_tsne
 42 | 
 43 | 
 44 | def separate_labels(tsne_output, labels, preds):
 45 |     # splits tsne output, labels, and preds into subarrays that have the same labels (for plotting by label)
 46 |     unique_labels = np.unique(labels)
 47 |     
 48 |     tsne_subar = [0] * unique_labels.shape[0]
 49 |     label_subar = [0] * unique_labels.shape[0]
 50 |     preds_subar = [0] * unique_labels.shape[0]
 51 |     indices = [0] * unique_labels.shape[0]
 52 |     for i,label in enumerate(unique_labels):
 53 |         idx = np.where(labels==label)
 54 |         
 55 |         tsne_subar[i] = tsne_output[idx]
 56 |         label_subar[i] = labels[idx]
 57 |         preds_subar[i] = preds[idx]
 58 |         indices[i] = idx[0]
 59 |         
 60 |     return tsne_subar, label_subar, indices, preds_subar
 61 | 
 62 | 
 63 | 
 64 | def tsne_plot(model, data, labels, classes, colors, shapes,layer_index, legend = True, show_preds = True, pca_dim=50, tsne_dim=2):
 65 |     """
 66 |     Does tsne reduction and plotting
 67 |     Parameters:
 68 |     model (keras model)
 69 |     data (np array): input data for the model
 70 |     layer_index: index of the output layer to do tsne on. 
 71 |     pca_dim (int): first dimensions are reduced using PCA to this number of dimensions
 72 |     tsne_dim (int): final desired dimension after doing tsne reduction
 73 |     show_preds (boolean): show prediction probability for each class when hovering over plot
 74 |     legend (boolean): show legend
 75 |     colors (dict label (String)-> Color (String)): Bookeh colors to use for the plot,  https://bokeh.pydata.org/en/latest/docs/reference/colors.html for referece of choices
 76 |     shapes (dict label (String)-> shape (int)): Specify which shape is desired 1 for circle,2 for diamond, 3 for cross (easy to modify to your own desired shapes)
 77 |     classes (string array): if show_preds is true specify what prediction belongs to what label, e.g. if the i-th one hot vector corresponds to label 'dog' then classes[i] should be 'dog'
 78 |     
 79 |     """
 80 |     if legend:
 81 |         legend = 'label'
 82 | 
 83 |     if show_preds:
 84 |         tsne_output, preds = tsne_model(model=model,data=data, preds=show_preds, layer_index=layer_index, pca_dim=pca_dim, tsne_dim=tsne_dim)
 85 |         # round to 3 decimals
 86 |         preds = np.around(preds, decimals=3)
 87 | 
 88 |         # turn each prediction into a list of strings
 89 |         preds = np.asarray([[classes[i] +": " + str(pred[i]) for i in range(len(pred))] for pred in preds])
 90 |     else:
 91 |         tsne_output = tsne_model(model=model,data=data, preds=show_preds, layer_index=layer_index, pca_dim=pca_dim, tsne_dim=tsne_dim)
 92 |         preds = np.zeros(labels.shape)
 93 | 
 94 |     tsne_sub, labels_sub, indices, preds_sub = separate_labels(tsne_output, labels, preds)
 95 |     
 96 |     
 97 |     tooltips = [("p", "(@x, @y)"),("label", "@label"),("index", "@index")]
 98 |     
 99 |     if show_preds:
100 |         tooltips.append(("preds", "@preds"))
101 |     
102 |     hover_tsne = HoverTool(tooltips = tooltips) 
103 |     tools_tsne = [hover_tsne, 'pan', 'wheel_zoom', 'reset','box_zoom','save']
104 |     p = figure(plot_width=700, plot_height=700, tools=tools_tsne)
105 |     output_notebook()
106 | 
107 |     for i in range(len(tsne_sub)):
108 |         tsne_ = tsne_sub[i]
109 |         labels_ = labels_sub[i]
110 |         indices_ = indices[i]
111 |         
112 |         if show_preds:
113 |             preds_ = preds_sub[i]
114 |         else:
115 |             preds_ = np.zeros(indices_.shape)
116 | 
117 |         
118 |         labl = labels_[0]
119 | 
120 |         source_train = ColumnDataSource(
121 |             data=dict(
122 |                 x = tsne_[:,0],
123 |                 y = tsne_[:,1],
124 |                 index = indices_,
125 |                 label = labels_,
126 |                 preds = preds_
127 |             )
128 |         )
129 | 
130 |         shape = shapes[labl]
131 |         if shape == 1:
132 |             p.circle('x', 'y', size=7, fill_color=colors[labl], 
133 |                      alpha=0.9, line_width=0, source=source_train, name="test", legend=legend)
134 |         elif shape == 2:
135 |             p.diamond('x', 'y', size=7, fill_color=colors[labl], 
136 |                      alpha=0.9, line_width=0, source=source_train, name="test", legend=legend)
137 |         elif shape == 3:
138 |             p.cross('x', 'y', size=7, fill_color=colors[labl], 
139 |                      alpha=0.9, line_width=0, source=source_train, name="test", legend=legend)
140 | 
141 |     p.legend.click_policy="hide"
142 | 
143 |     show(p)
144 |     
145 | 


--------------------------------------------------------------------------------