├── README.md ├── _config.yml └── src ├── codes ├── DeepLOB keras FI-2010.ipynb └── DeepLOB keras JNJ.ipynb ├── data └── data.txt ├── images ├── change_pct_adjust.png ├── inception_module_structure.png ├── labelling.png ├── limit_orderbook_example.png ├── mid_adjust.png └── whole_model_structure.png └── outputs ├── FI-2010_Accuracy_Graph.png ├── FI-2010_Loss_Graph.png ├── JNJ_Accuracy_Graph.png ├── JNJ_Loss_Graph.png └── outputs.txt /README.md: -------------------------------------------------------------------------------- 1 | 2 | ## DeepLOB-Model-Implementation-Project 3 | 4 | Welcome to my project page! It's time to dive DEEP :) 5 | 6 | This is a replicate project to develop the model raised up in the paper DeepLOB - Deep Convolutional Neural Networks. 7 | The original paper's authors are Zihao Zhang, Stefan Zohren, Stephen Roberts. The paper is linked here: [Original Paper](https://arxiv.org/pdf/1808.03668.pdf) 8 | 9 | ## Project Information 10 | ### University: Northwestern University 11 | 12 | ### Professor: Prof. Han Liu 13 | 14 | ### Project Member & Contact Information: 15 | 16 | * Yuxiang(Alvin) Chen yuxiangchen2021 at u.northwestern.edu 17 | 18 | ### GitHub Repository: 19 | Here is my [GitHub Repository](https://github.com/yuxiangalvin/DeepLOB-Model-Implementation-Project). 20 | 21 | This repo contains some codes and outputs of my implementation of DeepLOB model. 22 | 23 | ### Project Presentation 24 | Here is my [Project Presentation Video](https://youtu.be/KEP-8hXKk0c) 25 | 26 | ## Motivation: 27 | 28 | Deep Learning's application in Finance has always been one of the most complicated research area for Deep Learning. While reading various papers that focus on Deep Learning methods on Quantitative Finance applications, this paper about [DeepLOB - Deep Convolutional Neural Networks](https://arxiv.org/pdf/1808.03668.pdf) catches my attention. 29 | 30 | Nowadays, most companies in Quantitative Finance field uses limit orderbook data to conduct all kinds of analysis. It provides much more information than traditional one point data. High frequency limit orderbook data is essential for companies which conduct high frequency trading and the importance has been growing at an extremely fast speed. As an individal who is very passionate about machine learning's applications in finance data of Level II or above, I would like to fully understand the DeepLOB model and the authors' intensions behind each design component. At the same time, I would like to further enhance my deep learning application skills. Thus, I conducted this replicate project. 31 | 32 | 33 | ## Model Details 34 | 35 | The model takes in the limit orderbook data of one specific stock as input and conduct a 3-class prediction for the stock's mid-price movement. The three classes are 'Down', 'Hold' and 'Up'. There has been previous researches which focus on limit orderbook data. However, most of them applied static feature extract method that mainly based on domain expertise and conventions. These static methods include Linear Discriminant Analysis (LDA) and Principal Component Analysis (PCA), etc. 36 | 37 | The DeepLOB model intead applies a dynamic feature ectraction approach through applying Deep Learning architecture (CNN + Inception Module). It also uses an additional LSTM to capture additional dependence that is not captured by the CNN + Inception Module. 38 | 39 | ### Inputs 40 | 41 | #### Raw Data 42 | The DeepLOB model takes limit orderbook data as inputs, specifically, at each time point, it takes in a limit orderbook snippet - the lowest 10 ask levels and the highest 10 bid levels. Each level has one price data and one size data. Thus, at each time point, there are 40 numbers. Below is am example of how the orderbook looks like at one time point (10 levels are shown here in this example) 43 | 44 | ![LIMIT ORDERBOOK EXAMPLE](./src/images/limit_orderbook_example.png) 45 | 46 | 47 | The authors of the model use a lookback period of 100 timesteps at each time step. Thus at each time step, the input matrix has a dimension of 100 x 40. 48 | 49 | Thus the input size of the model is N x 100 x 40 x 1 (N is the number of timesteps used as input) 50 | 51 | The paper authors used two different datasets: FI-2010 and London Stock Exchange (LSE) limit orderbook data. 52 | 53 | #### FI-2010 dataset 54 | 55 | * FI-2010 is a public benchmark dataset of HFT LOB data and extracted time series data for five stocks from the Nasdaq Nordic stock market for a time period of 10 consecutive days. 56 | * The timestep distance between two data points are in average < 1 second. 57 | * The dataset is pre-normalized using z-score normalization. 58 | 59 | #### LSE dataset 60 | 61 | * LSE dataset is not a publicly accessible dataset. 62 | * The stocks involved has much higher liqudity than the 5 stocks in FI-2010 dataset 63 | * The timestep distance between two data points are samller and in average 0.192 seocnd 64 | * The dataset is not pre-normalized. 65 | 66 | #### Data Labeling 67 | Following quantities are calculated using corresponding equations & labels are generated. 68 | | mid-price | previous k timesteps avg mid-price | future k timesteps avg mid-price| move pct | label 69 | | -- | ---- | ------- | - | - | 70 | | p | m- | m+ | l | y | 71 | 72 | 73 | ![labelling](./src/images/labelling.png) 74 | 75 | 76 | #### Data Normalization 77 | 78 | * FI-2010 dataset is pre-noralized 79 | * LSE dataset: Each trading day's price and size at each level is normalized using the previous 5 trading days' price and size separately. 80 | 81 | 82 | ### Model Structure 83 | 84 | Here I will use the original pictures used in the original paper with my annotations to present the model structure. 85 | 86 | ![WHOLE MODEL STRUCTURE](./src/images/whole_model_structure.png) 87 | 88 | The model starts with 1 CNN block with 3 sub parts. 89 | 90 | #### CNN Block Design 91 | 92 | There are three points that worths noticing in the CNN block design. 93 | 94 | 1. The design of 1x2 filter and 1x2 stride at the beginning the 1st sub part is used to capture one important nature of the input data. One of the dimentions of the input data is 40 (price and size at 20 levels of order book). Since the data is ordered as price, size alternatively. This design keeps the first element of the horizontal filter only looking at prices and the second element of the filter only looking at sizes. This design takes the nature of the data into account and thus makes the 16 different feature maps generated from 16 different filters more representative. 95 | 96 | 2. The design of 4x1 filter in the 2nd layer of 1st subpart capture local interactions amongst data over four time steps. 97 | 98 | 3. The further layers keep exploring boarder interactions. 99 | 100 | #### Inception Module 101 | 102 | Following the CNN block is an Inception Module. The Inception Module is more powerful than a common CNN block becasue it allows to use multiple types of filter size, instead of being restricted to a single filter size. The specific structure of the Inception Module is shown below in the figure. 103 | 104 | ![INCEPTION MODULE STRUCTURE](./src/images/inception_module_structure.png) 105 | 106 | As the structure figure shows, this specific Inception Module contains three parallel processes. This allows the module to capture dynamic behaviors over multiple timescales. An 1 x 1 Conv layer is used in every path. This idea is form the Network-in-Network approach proposed in a [2014 paper](https://arxiv.org/pdf/1312.4400v3.pdf). Instead of applying a simple convolution to the data, the Network-in-Network method uses a small neural network to capture the non-linear properties of the data. 107 | 108 | #### LSTM & FC 109 | 110 | A LSTM layer with 64 LSTM unities is used after the CNN + Inception Module part in order to capture additioanl time dependencies. 111 | 112 | A fully connected layer is used to map the 64 outputs from LSTM units to size 3 (one hot encoding of the 3 categories) 113 | 114 | 115 | ## My Experiments & Codes 116 | 117 | All the following codes could be found in 'DeepLOB keras FI-2010.ipynb' and 'DeepLOB keras JNJ.ipynb' in codes folder. The two files differ only in data preprocessing process and the experiment process. 118 | 119 | ### Import Libraries 120 | ```python 121 | import pandas as pd 122 | import numpy as np 123 | 124 | from sklearn.model_selection import train_test_split 125 | 126 | from keras.utils import np_utils 127 | from keras.models import Model 128 | from keras.optimizers import Adam 129 | from keras.layers import Input, Conv2D, LeakyReLU, MaxPooling2D, concatenate, LSTM, Reshape, Dense 130 | from keras.callbacks import EarlyStopping 131 | 132 | import pandas_market_calendars as mcal 133 | ``` 134 | 135 | ### Hyper-parameters 136 | ```python 137 | #Input param 138 | lookback_timestep = 100 139 | feature_num = 40 140 | 141 | #Conv param 142 | conv_filter_num = 16 143 | 144 | #Inception module param 145 | inception_num = 32 146 | 147 | #LSTM param 148 | LSTM_num = 64 149 | 150 | #Activation param 151 | leaky_relu_alpha = 0.01 152 | 153 | #Training params 154 | loss = 'categorical_crossentropy' 155 | learning_rate = 0.01 156 | adam_epsilon = 1 157 | optimizer = Adam(lr=learning_rate, epsilon=1) 158 | batch_size = 32 159 | 160 | #Training stopping Criteria 161 | metrics = ['accuracy'] 162 | #stop training when validation accuracy does not improve for 20 epochs 163 | stop_epoch_num = 20 164 | 165 | #max epoch num is not specified in paper, use an arbitrary large number 10000 166 | num_epoch = 10000 167 | ``` 168 | 169 | ### Data Normalization & Labeling 170 | 171 | I conducted experiment on two different datasets: FI-2010 & JNJ (Johnson & Johnson) 2020 January limit orderbook dataset. 172 | 173 | #### FI-2010 174 | * Benchmark dataset of HFT LOB data 175 | * Extracted time series data for five stocks from the Nasdaq Nordic stock market (not very liquid asset) 176 | * Timestep distance in average < 1 second 177 | * Pre-normalized using z-score 178 | * Labelled by paper authors 179 | 180 | The dataset is included in the github repo. 181 | 182 | Since this dataset accessed from authors' provided source is pre-normalized & labelled so no normalization or labelling is needed and the only preprocessing is dimension adjustion. 183 | 184 | ```python 185 | def extract_x_y_data(data, timestamp_per_sample): 186 | data_x = np.array(data[:40, :].T) 187 | data_y = np.array(data[-5:, :].T) 188 | [N, P_x] = data_x.shape 189 | P_y = data_y.shape[1] 190 | 191 | x = np.zeros([(N-timestamp_per_sample+1), timestamp_per_sample, P_x]) 192 | 193 | for i in range(N-timestamp_per_sample+1): 194 | x[i] = data_x[i:(i+timestamp_per_sample), :] 195 | 196 | x = x.reshape(x.shape + (1,)) 197 | 198 | y = data_y[(timestamp_per_sample-1):] 199 | y = y[:,3] - 1 200 | y = np_utils.to_categorical(y, 3) 201 | 202 | return x, y 203 | 204 | train_fi_x, train_fi_y = extract_x_y_data(train_fi, timestamp_per_sample=100) 205 | test_fi_x, test_fi_y = extract_x_y_data(test_fi, timestamp_per_sample=100) 206 | # use a subset of the data for experiment 207 | train_fi_x3, train_fi_y3, test_fi_x3, test_fi_y3 = train_fi_x[:100000,:,:,:], train_fi_y[:100000,:], test_fi_x[:20000,:], test_fi_y[:20000,:] 208 | ``` 209 | 210 | #### JNJ LOB 211 | * About 160000-200000 data points per trading day 212 | * Timestep distance in average about 0.15 second 213 | * Not pre-normalized 214 | * Unlabelled 215 | 216 | This dataset is restricted to class use so it's not included in github repo. Here I will present my complete code example of data pre-processing (normalization, labelling & dimension adjustion) 217 | 218 | ##### Data Read-in & Normalization 219 | 220 | The paper authors conducted their second experiment on London Stock Exchange (LSE) LOB dataset. The JNJ dataset and LSE dataset share similar characteristics in their frequency, stock liquidity, etc. Thus I followed the same method for nomalization as that used by authors for LSE dataset. I used the previous 5 days data to normalize the current day' data. This is applied to every day (excluding the first 5 days in the dataset) 221 | 222 | ```python 223 | # get all trading days in the date range 224 | nyse = mcal.get_calendar('NYSE') 225 | dates = list(nyse.schedule(start_date='2020-01-01', end_date='2020-01-09').index) 226 | dates_str_list = [] 227 | for trading_day in dates: 228 | dates_str_list.append(str(trading_day.date())) 229 | 230 | # read & store daily LOB data in a dictionary 231 | daily_data_dict= {} 232 | for i in range(len(dates_str_list)): 233 | date = dates_str_list[i] 234 | if date not in daily_data_dict.keys(): 235 | date = dates_str_list[i] 236 | daily_data_dict[date] = np.array(pd.read_csv('./data/JNJ_orderbook/JNJ_' + date + '_34200000_57600000_orderbook_10.csv',header = None)) 237 | 238 | # get the previous 5 day mean & standard deviation for each trading day and store in dictionaries. 239 | normalization_mean_dict = {} 240 | normalization_stddev_dict = {} 241 | for i in range(5,len(dates_str_list)): 242 | date = dates_str_list[i] 243 | 244 | if (date not in normalization_mean_dict.keys()) or (date not in normalization_stddev_dict.keys()): 245 | look_back_dates_list = dates_str_list[(i-5):i] 246 | prev_5_day_orderbook_np = None 247 | for look_back_date in look_back_dates_list: 248 | if prev_5_day_orderbook_np is None: 249 | prev_5_day_orderbook_np = daily_data_dict[look_back_date] 250 | else: 251 | prev_5_day_orderbook_np = np.vstack((prev_5_day_orderbook_np, daily_data_dict[look_back_date])) 252 | 253 | 254 | price_mean = prev_5_day_orderbook_np[:,range(0,prev_5_day_orderbook_np.shape[1],2)].mean() 255 | price_std = prev_5_day_orderbook_np[:,range(0,prev_5_day_orderbook_np.shape[1],2)].std() 256 | size_mean = prev_5_day_orderbook_np[:,range(1,prev_5_day_orderbook_np.shape[1],2)].mean() 257 | size_std = prev_5_day_orderbook_np[:,range(1,prev_5_day_orderbook_np.shape[1],2)].std() 258 | 259 | normalization_mean_dict[date] = np.repeat([[price_mean,size_mean]], 20, axis=0).flatten() 260 | normalization_stddev_dict[date] = np.repeat([[price_std,size_std]], 20, axis=0).flatten() 261 | 262 | # normalize each day's data separatly 263 | daily_norm_data_dict = {} 264 | for i in range(5,len(dates_str_list)): 265 | date = dates_str_list[i] 266 | if date not in daily_norm_data_dict.keys(): 267 | daily_norm_data_dict[date] = (daily_data_dict[date] - normalization_mean_dict[date])/ normalization_stddev_dict[date] 268 | ``` 269 | 270 | ##### Labelling 271 | 272 | I applied two adjustions to the author's labelling method. 273 | 274 | * mid price is calculated as the weighted mid price using limit order size at the best ask and bid level instead of the simple mid point. This is a mroe accuracte way to calculate theoretical mid price used by quantitative finance companies and researchers. 275 | 276 | ![mid_adjust](./src/images/mid_adjust.png) 277 | 278 | * The category label is labelled through looking at change percentage from current timestep mid-price to future k timestep average mid-price instead of past k to future k. This adjustion makes sure the model could not see part of the change percentage information from input X. 279 | 280 | ![change_pct_adjust](./src/images/change_pct_adjust.png) 281 | 282 | ```python 283 | # define functions to generate X (appropriate dimension) and y (labelling) 284 | def moving_average(x, k): 285 | return np.convolve(x, np.ones(k), 'valid') / k 286 | 287 | def generate_labels(k, alpha, daily_data_dict): 288 | daily_label_dict = {} 289 | for date in list(daily_data_dict.keys())[5:]: 290 | price_ask = daily_data_dict[date][:,0] 291 | size_ask = daily_data_dict[date][:,1] 292 | price_bid = daily_data_dict[date][:,2] 293 | size_bid = daily_data_dict[date][:,3] 294 | mid_price = (price_ask * size_bid + price_bid * size_ask) / (size_ask + size_bid) 295 | future_k_avg_mid_price = moving_average(mid_price, k)[1:] 296 | change_pct = (future_k_avg_mid_price - mid_price[:-k])/mid_price[:-k] 297 | y_label = (-(change_pct < -alpha).astype(int)) + (change_pct > alpha).astype(int) 298 | 299 | daily_label_dict[date] = y_label.reshape(-1,1) 300 | return daily_label_dict 301 | 302 | def generate_X_y(k, alpha, timestamp_per_sample, daily_norm_data_dict, daily_data_dict): 303 | #k is the number of future timesteps used to generate the label y 304 | data_x = None 305 | for date in daily_norm_data_dict.keys(): 306 | if data_x is None: 307 | data_x = daily_norm_data_dict[date].copy()[:-k,:] 308 | else: 309 | data_x = np.vstack((data_x, daily_norm_data_dict[date][:-k,:])) 310 | print(data_x.shape) 311 | 312 | daily_label_dict = generate_labels(k, alpha, daily_data_dict) 313 | data_y = None 314 | for date in daily_label_dict.keys(): 315 | if data_y is None: 316 | data_y = daily_label_dict[date].copy() 317 | else: 318 | data_y = np.vstack((data_y, daily_label_dict[date])) 319 | 320 | [N, P_x] = data_x.shape 321 | x = np.zeros([(N-timestamp_per_sample+1), timestamp_per_sample, P_x]) 322 | 323 | for i in range(N-timestamp_per_sample+1): 324 | x[i] = data_x[i:(i+timestamp_per_sample), :] 325 | 326 | x = x.reshape(x.shape + (1,)) 327 | y = data_y[(timestamp_per_sample-1):] 328 | y = np_utils.to_categorical(y, 3) 329 | 330 | return x, y 331 | 332 | # generate X and y with k = 8 & alpha = 7e-6 (alpha decided through finding the threshold value that approximately separates data into 3 balanced label categories for the specific k value) 333 | X,y = generate_X_y(k=8, alpha=7e-6, timestamp_per_sample=100, 334 | daily_norm_data_dict= daily_norm_data_dict, 335 | daily_data_dict = daily_data_dict) 336 | # separate into train & validation data (4:1) 337 | X_train, X_test, y_train, y_test = train_test_split(X,y,test_size = 0.2) 338 | ``` 339 | 340 | ### Model Definition 341 | 342 | ```python 343 | def initiate_DeepLOB_model(lookback_timestep, feature_num, conv_filter_num, inception_num, LSTM_num, leaky_relu_alpha, 344 | loss, optimizer, metrics): 345 | 346 | input_tensor = Input(shape=(lookback_timestep, feature_num, 1)) 347 | 348 | # Conv block1 349 | print(input_tensor.shape) 350 | conv_layer1 = Conv2D(conv_filter_num, (1,2), strides=(1, 2))(input_tensor) 351 | print(conv_layer1.shape) 352 | conv_layer1 =LeakyReLU(alpha=leaky_relu_alpha)(conv_layer1) 353 | print(conv_layer1.shape) 354 | conv_layer1 = Conv2D(conv_filter_num, (4,1), padding='same')(conv_layer1) 355 | conv_first1 = LeakyReLU(alpha=leaky_relu_alpha)(conv_layer1) 356 | print(conv_layer1.shape) 357 | conv_layer1 = Conv2D(conv_filter_num, (4,1), padding='same')(conv_layer1) 358 | conv_layer1 = LeakyReLU(alpha=leaky_relu_alpha)(conv_layer1) 359 | print(conv_layer1.shape) 360 | 361 | # Conv block2 362 | conv_layer2 = Conv2D(conv_filter_num, (1,2), strides=(1, 2))(conv_layer1) 363 | conv_layer2 = LeakyReLU(alpha=leaky_relu_alpha)(conv_layer2) 364 | print(conv_layer2.shape) 365 | conv_layer2 = Conv2D(conv_filter_num, (4,1), padding='same')(conv_layer2) 366 | conv_layer2 = LeakyReLU(alpha=leaky_relu_alpha)(conv_layer2) 367 | print(conv_layer2.shape) 368 | conv_layer2 = Conv2D(conv_filter_num, (4,1), padding='same')(conv_layer2) 369 | conv_layer2 = LeakyReLU(alpha=leaky_relu_alpha)(conv_layer2) 370 | print(conv_layer2.shape) 371 | 372 | # Conv block3 373 | conv_layer3 = Conv2D(conv_filter_num, (1,10))(conv_layer2) 374 | conv_layer3 = LeakyReLU(alpha=leaky_relu_alpha)(conv_layer3) 375 | print(conv_layer3.shape) 376 | conv_layer3 = Conv2D(conv_filter_num, (4,1), padding='same')(conv_layer3) 377 | conv_layer3 = LeakyReLU(alpha=leaky_relu_alpha)(conv_layer3) 378 | print(conv_layer3.shape) 379 | conv_layer3 = Conv2D(conv_filter_num, (4,1), padding='same')(conv_layer3) 380 | conv_layer3 = LeakyReLU(alpha=leaky_relu_alpha)(conv_layer3) 381 | print(conv_layer3.shape) 382 | 383 | # Inception module 384 | inception_module1 = Conv2D(inception_num, (1,1), padding='same')(conv_layer3) 385 | inception_module1 = LeakyReLU(alpha=leaky_relu_alpha)(inception_module1) 386 | print(inception_module1.shape) 387 | inception_module1 = Conv2D(inception_num, (3,1), padding='same')(inception_module1) 388 | inception_module1 = LeakyReLU(alpha=leaky_relu_alpha)(inception_module1) 389 | print(inception_module1.shape) 390 | 391 | inception_module2 = Conv2D(inception_num, (1,1), padding='same')(conv_layer3) 392 | inception_module2 = LeakyReLU(alpha=leaky_relu_alpha)(inception_module2) 393 | print(inception_module2.shape) 394 | inception_module2 = Conv2D(inception_num, (5,1), padding='same')(inception_module2) 395 | inception_module2 = LeakyReLU(alpha=leaky_relu_alpha)(inception_module2) 396 | print(inception_module2.shape) 397 | 398 | inception_module3 = MaxPooling2D((3,1), strides=(1,1), padding='same')(conv_layer3) 399 | print(inception_module3.shape) 400 | inception_module3 = Conv2D(inception_num, (1,1), padding='same')(inception_module3) 401 | print(inception_module3.shape) 402 | inception_module3 = LeakyReLU(alpha=leaky_relu_alpha)(inception_module3) 403 | print(inception_module3.shape) 404 | 405 | inception_module_final = concatenate([inception_module1, inception_module2, inception_module3], axis=3) 406 | print(inception_module_final.shape) 407 | inception_module_final = Reshape((inception_module_final.shape[1], inception_module_final.shape[3]))(inception_module_final) 408 | print(inception_module_final.shape) 409 | 410 | # LSTM 411 | LSTM_output = LSTM(LSTM_num)(inception_module_final) 412 | print(LSTM_output.shape) 413 | 414 | # Fully Connected Layer with softmax activation function for output 415 | model_output = Dense(3, activation='softmax')(LSTM_output) 416 | print(model_output.shape) 417 | 418 | DeepLOB_model = Model(inputs=input_tensor, outputs= model_output) 419 | es = EarlyStopping(monitor='val_accuracy', mode='max', verbose=1) 420 | 421 | DeepLOB_model.compile(optimizer=optimizer, loss=loss, metrics=metrics) 422 | 423 | return DeepLOB_model 424 | ``` 425 | 426 | ### Model Initiation, Training & Testing 427 | 428 | As stated above, the data is separated into a training set and a validation set. Since all parameters are provided by the paper clearly, there is no model tuning process that depend on validation results. The validation set is used as the function of the testing set. The code below will print out the training process. Graphs of loss and accuracy through training and other Keras supported metrics could be accessed fro mthe history variable. 429 | 430 | ```python 431 | DeepLOB_model = initiate_DeepLOB_model(lookback_timestep, feature_num, conv_filter_num, inception_num, LSTM_num, leaky_relu_alpha, 432 | loss, optimizer, metrics) 433 | 434 | # definte the training stop criteria (no new max validation accuracy in 20 consecutive epochs) 435 | es = EarlyStopping(monitor='val_accuracy', mode='max', patience = stop_epoch_num, verbose=1) 436 | history = DeepLOB_model.fit(X_train, y_train, epochs=num_epoch, batch_size=batch_size, verbose=2, validation_data=(X_test, y_test), callbacks = [es]) 437 | ``` 438 | 439 | ## Experiments Results 440 | 441 | ### FI-2010 442 | 443 | Here are the loss and accuracy graphs along the training process of the FI-2010 experiment (k = 20) 444 | 445 | ![FI-2010 Loss Graph](./src/outputs/FI-2010_Loss_Graph.png) 446 | 447 | ![FI-2010 Accuracy Graph](./src/outputs/FI-2010_Accuracy_Graph.png) 448 | 449 | According to the graphs, both validation loss and accuracy stops improving after about 60 epochs although training loss and accuracy are still improving and at around 80 epochs the training stops. 450 | 451 | Here is the comparison between authors' reported validation accuracy of their experiment with FI-2010 dataset. 452 | 453 | | Model| Accuracy| 454 | | -- | ---- | 455 | | Author’s Report |78.91%| 456 | | My Experiment| 73.00%| 457 | 458 | The potential reason of this difference could be that I am using only part of the FI-2010 dataset for my experiment so the training data number is not as big as the one the authors used. 459 | 460 | To further assess the performance of my model, I also conducted the experiment on the JNJ stock LOB dataset. 461 | 462 | ### JNJ LOB 463 | 464 | Here are the loss and accuracy graphs along the training process of one specific JNJ LOB experiment (k = 8, alpha = 7e-6) 465 | 466 | 467 | ![JNJ Loss Graph](./src/outputs/JNJ_Loss_Graph.png) 468 | 469 | ![JNJ Accuracy Graph](./src/outputs/JNJ_Accuracy_Graph.png) 470 | 471 | |Model |k | Accuracy |Epochs taken| 472 | | -- | - | ------- | ----- | 473 | |Author’s Report on LSE dataset |20 |70.17% | | 474 | |Author’s Report on LSE dataset | 50 | 63.93% | | 475 | |Author’s Report on LSE dataset | 100|61.52%| | 476 | |Replicate on JNJ orderbook data |8 |70.28% | 184 | 477 | |Replicate on JNJ orderbook data |26 |80.50% |113 | 478 | |Replicate on JNJ orderbook data |80 |77.52% |32 | 479 | 480 | From the result, my experiment result shows high validation accuracy than authors' experiment on LSE dataset. This shows that my replicated model has great performance on the specific JNJ dataset. 481 | 482 | I also notice that as k increases in my experiment, final valdiation accuracy has a rough increasing trend (until certain k value) and the number of epochs taken for training goes down as k increases. However, the valdiation accruacy trend along k is opposite in authors' report. This is an observation that is worth more thinking and research. 483 | 484 | ## Conclusions & Next Steps 485 | 486 | ### Conclusions 487 | 488 | * The model I replicated and implemented could successfully be trained and reach similar performance as the author reported. 489 | * The model does need sufficient amount of data for training to avoid overfitting problem. 490 | * As the k parameter increases, according to my experiment, the number of epochs needed for the training to finish decreases. 491 | * The model performs well still with adjustment on how midprice is calculated and how lables are generated. 492 | * The model has a stable great performance on high frequency dataset. 493 | 494 | ### Considerations & Next Steps 495 | In the next step, I have two plan steps: 496 | 497 | * With more computing power allowed, I plan to conduct experiment on larger limit order book dataset on liquid assets 498 | * Further research on how the value of k affects model performance and find out what tasks the model is most appropriate for. 499 | 500 | An important next-step consideration of putting this model into application from here: 501 | 502 | * It will be very meaningful to think about how the result of the model could be used for real-life trading strategies in high frequency trading environment. 503 | * Whether there is any other adjustment on the labelling method needed for the model to make the conversion to real trading strategies feasible and applicable. 504 | -------------------------------------------------------------------------------- /_config.yml: -------------------------------------------------------------------------------- 1 | theme: jekyll-theme-slate -------------------------------------------------------------------------------- /src/codes/DeepLOB keras JNJ.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "## Replicate - DeepLOB: Deep Convolutional Neural Networks for Limit Order Books\n", 8 | "(Paper authors: Zihao Zhang, Stefan Zohren, Stephen Roberts)\n", 9 | "\n", 10 | "Dataset source: https://drive.google.com/drive/folders/1Xen3aRid9ZZhFqJRgEMyETNazk02cNmv?usp=sharing" 11 | ] 12 | }, 13 | { 14 | "cell_type": "code", 15 | "execution_count": 2, 16 | "metadata": {}, 17 | "outputs": [], 18 | "source": [ 19 | "# import libraries\n", 20 | "import pandas as pd\n", 21 | "import numpy as np\n", 22 | "\n", 23 | "from sklearn.model_selection import train_test_split\n", 24 | "\n", 25 | "from keras.utils import np_utils\n", 26 | "from keras.models import Model\n", 27 | "from keras.optimizers import Adam\n", 28 | "from keras.layers import Input, Conv2D, LeakyReLU, MaxPooling2D, concatenate, LSTM, Reshape, Dense\n", 29 | "from keras.callbacks import EarlyStopping\n", 30 | "\n", 31 | "import pandas_market_calendars as mcal" 32 | ] 33 | }, 34 | { 35 | "cell_type": "markdown", 36 | "metadata": {}, 37 | "source": [ 38 | "#### Data Preprocessing" 39 | ] 40 | }, 41 | { 42 | "cell_type": "code", 43 | "execution_count": 4, 44 | "metadata": {}, 45 | "outputs": [], 46 | "source": [ 47 | "nyse = mcal.get_calendar('NYSE')\n", 48 | "dates = list(nyse.schedule(start_date='2020-01-01', end_date='2020-01-09').index)" 49 | ] 50 | }, 51 | { 52 | "cell_type": "code", 53 | "execution_count": 5, 54 | "metadata": {}, 55 | "outputs": [], 56 | "source": [ 57 | "dates_str_list = []\n", 58 | "for trading_day in dates:\n", 59 | " dates_str_list.append(str(trading_day.date()))" 60 | ] 61 | }, 62 | { 63 | "cell_type": "code", 64 | "execution_count": 6, 65 | "metadata": {}, 66 | "outputs": [], 67 | "source": [ 68 | "daily_data_dict= {}" 69 | ] 70 | }, 71 | { 72 | "cell_type": "code", 73 | "execution_count": 7, 74 | "metadata": {}, 75 | "outputs": [], 76 | "source": [ 77 | "for i in range(len(dates_str_list)):\n", 78 | " date = dates_str_list[i]\n", 79 | " if date not in daily_data_dict.keys():\n", 80 | " date = dates_str_list[i]\n", 81 | " daily_data_dict[date] = np.array(pd.read_csv('./data/JNJ_orderbook/JNJ_' + date + '_34200000_57600000_orderbook_10.csv',header = None))" 82 | ] 83 | }, 84 | { 85 | "cell_type": "code", 86 | "execution_count": 8, 87 | "metadata": {}, 88 | "outputs": [], 89 | "source": [ 90 | "normalization_mean_dict = {}\n", 91 | "normalization_stddev_dict = {}" 92 | ] 93 | }, 94 | { 95 | "cell_type": "code", 96 | "execution_count": 9, 97 | "metadata": {}, 98 | "outputs": [], 99 | "source": [ 100 | "for i in range(5,len(dates_str_list)):\n", 101 | " date = dates_str_list[i]\n", 102 | " \n", 103 | " if (date not in normalization_mean_dict.keys()) or (date not in normalization_stddev_dict.keys()):\n", 104 | " look_back_dates_list = dates_str_list[(i-5):i]\n", 105 | " prev_5_day_orderbook_np = None\n", 106 | " for look_back_date in look_back_dates_list:\n", 107 | " if prev_5_day_orderbook_np is None:\n", 108 | " prev_5_day_orderbook_np = daily_data_dict[look_back_date]\n", 109 | " else:\n", 110 | " prev_5_day_orderbook_np = np.vstack((prev_5_day_orderbook_np, daily_data_dict[look_back_date]))\n", 111 | " \n", 112 | " \n", 113 | " price_mean = prev_5_day_orderbook_np[:,range(0,prev_5_day_orderbook_np.shape[1],2)].mean()\n", 114 | " price_std = prev_5_day_orderbook_np[:,range(0,prev_5_day_orderbook_np.shape[1],2)].std()\n", 115 | " size_mean = prev_5_day_orderbook_np[:,range(1,prev_5_day_orderbook_np.shape[1],2)].mean()\n", 116 | " size_std = prev_5_day_orderbook_np[:,range(1,prev_5_day_orderbook_np.shape[1],2)].std()\n", 117 | " \n", 118 | " normalization_mean_dict[date] = np.repeat([[price_mean,size_mean]], 20, axis=0).flatten()\n", 119 | " normalization_stddev_dict[date] = np.repeat([[price_std,size_std]], 20, axis=0).flatten()" 120 | ] 121 | }, 122 | { 123 | "cell_type": "code", 124 | "execution_count": 10, 125 | "metadata": {}, 126 | "outputs": [], 127 | "source": [ 128 | "daily_norm_data_dict = {}" 129 | ] 130 | }, 131 | { 132 | "cell_type": "code", 133 | "execution_count": 11, 134 | "metadata": {}, 135 | "outputs": [], 136 | "source": [ 137 | "for i in range(5,len(dates_str_list)):\n", 138 | " date = dates_str_list[i]\n", 139 | " if date not in daily_norm_data_dict.keys():\n", 140 | " daily_norm_data_dict[date] = (daily_data_dict[date] - normalization_mean_dict[date])/ normalization_stddev_dict[date]" 141 | ] 142 | }, 143 | { 144 | "cell_type": "code", 145 | "execution_count": 12, 146 | "metadata": {}, 147 | "outputs": [ 148 | { 149 | "data": { 150 | "text/plain": [ 151 | "(159941, 40)" 152 | ] 153 | }, 154 | "execution_count": 12, 155 | "metadata": {}, 156 | "output_type": "execute_result" 157 | } 158 | ], 159 | "source": [ 160 | "daily_norm_data_dict['2020-01-09'].shape" 161 | ] 162 | }, 163 | { 164 | "cell_type": "code", 165 | "execution_count": 13, 166 | "metadata": {}, 167 | "outputs": [], 168 | "source": [ 169 | "def moving_average(x, k):\n", 170 | " return np.convolve(x, np.ones(k), 'valid') / k" 171 | ] 172 | }, 173 | { 174 | "cell_type": "code", 175 | "execution_count": 14, 176 | "metadata": {}, 177 | "outputs": [], 178 | "source": [ 179 | "def generate_labels(k, alpha, daily_data_dict):\n", 180 | " daily_label_dict = {}\n", 181 | " for date in list(daily_data_dict.keys())[5:]:\n", 182 | " price_ask = daily_data_dict[date][:,0]\n", 183 | " size_ask = daily_data_dict[date][:,1]\n", 184 | " price_bid = daily_data_dict[date][:,2]\n", 185 | " size_bid = daily_data_dict[date][:,3]\n", 186 | " mid_price = (price_ask * size_bid + price_bid * size_ask) / (size_ask + size_bid)\n", 187 | " future_k_avg_mid_price = moving_average(mid_price, k)[1:]\n", 188 | " change_pct = (future_k_avg_mid_price - mid_price[:-k])/mid_price[:-k]\n", 189 | " y_label = (-(change_pct < -alpha).astype(int)) + (change_pct > alpha).astype(int)\n", 190 | " \n", 191 | " daily_label_dict[date] = y_label.reshape(-1,1)\n", 192 | " return daily_label_dict" 193 | ] 194 | }, 195 | { 196 | "cell_type": "code", 197 | "execution_count": 15, 198 | "metadata": {}, 199 | "outputs": [], 200 | "source": [ 201 | "k=26\n", 202 | "date = '2020-01-09'\n", 203 | "price_ask = daily_data_dict[date][:,0]\n", 204 | "size_ask = daily_data_dict[date][:,1]\n", 205 | "price_bid = daily_data_dict[date][:,2]\n", 206 | "size_bid = daily_data_dict[date][:,3]\n", 207 | "mid_price = (price_ask * size_bid + price_bid * size_ask) / (size_ask + size_bid)\n", 208 | "future_k_avg_mid_price = moving_average(mid_price, k)[1:]\n", 209 | "change_pct = (future_k_avg_mid_price - mid_price[:-k])/mid_price[:-k]" 210 | ] 211 | }, 212 | { 213 | "cell_type": "code", 214 | "execution_count": 68, 215 | "metadata": {}, 216 | "outputs": [ 217 | { 218 | "data": { 219 | "text/plain": [ 220 | "-1.970212233871668e-05" 221 | ] 222 | }, 223 | "execution_count": 68, 224 | "metadata": {}, 225 | "output_type": "execute_result" 226 | } 227 | ], 228 | "source": [ 229 | "np.percentile(change_pct,33.33)" 230 | ] 231 | }, 232 | { 233 | "cell_type": "code", 234 | "execution_count": 69, 235 | "metadata": {}, 236 | "outputs": [ 237 | { 238 | "data": { 239 | "text/plain": [ 240 | "1.8990615131648153e-05" 241 | ] 242 | }, 243 | "execution_count": 69, 244 | "metadata": {}, 245 | "output_type": "execute_result" 246 | } 247 | ], 248 | "source": [ 249 | "np.percentile(change_pct,66.67)" 250 | ] 251 | }, 252 | { 253 | "cell_type": "code", 254 | "execution_count": 16, 255 | "metadata": {}, 256 | "outputs": [], 257 | "source": [ 258 | "def generate_X_y(k, alpha, timestamp_per_sample, daily_norm_data_dict, daily_data_dict):\n", 259 | " #k is the number of future timesteps used to generate the label y\n", 260 | " data_x = None\n", 261 | " for date in daily_norm_data_dict.keys():\n", 262 | " if data_x is None:\n", 263 | " data_x = daily_norm_data_dict[date].copy()[:-k,:]\n", 264 | " else:\n", 265 | " data_x = np.vstack((data_x, daily_norm_data_dict[date][:-k,:]))\n", 266 | " print(data_x.shape)\n", 267 | " \n", 268 | " daily_label_dict = generate_labels(k, alpha, daily_data_dict)\n", 269 | " data_y = None\n", 270 | " for date in daily_label_dict.keys():\n", 271 | " if data_y is None:\n", 272 | " data_y = daily_label_dict[date].copy()\n", 273 | " else:\n", 274 | " data_y = np.vstack((data_y, daily_label_dict[date]))\n", 275 | " \n", 276 | " [N, P_x] = data_x.shape\n", 277 | "# P_y = data_y.shape[1]\n", 278 | " \n", 279 | " x = np.zeros([(N-timestamp_per_sample+1), timestamp_per_sample, P_x])\n", 280 | " \n", 281 | " for i in range(N-timestamp_per_sample+1):\n", 282 | " x[i] = data_x[i:(i+timestamp_per_sample), :]\n", 283 | " \n", 284 | " x = x.reshape(x.shape + (1,))\n", 285 | " y = data_y[(timestamp_per_sample-1):]\n", 286 | " y = np_utils.to_categorical(y, 3)\n", 287 | " \n", 288 | " return x, y" 289 | ] 290 | }, 291 | { 292 | "cell_type": "code", 293 | "execution_count": 22, 294 | "metadata": {}, 295 | "outputs": [], 296 | "source": [ 297 | "# k = 8\n", 298 | "# alpha = 7e-6\n", 299 | "# price_ask = daily_data_dict['2020-01-09'][:,0]\n", 300 | "# size_ask = daily_data_dict['2020-01-09'][:,1]\n", 301 | "# price_bid = daily_data_dict['2020-01-09'][:,2]\n", 302 | "# size_bid = daily_data_dict['2020-01-09'][:,3]\n", 303 | "# mid_price = (price_ask * size_bid + price_bid * size_ask) / (size_ask + size_bid)\n", 304 | "# future_k_avg_mid_price = moving_average(mid_price, k)[1:]\n", 305 | "# change_pct = (future_k_avg_mid_price - mid_price[:-k])/mid_price[:-k]\n", 306 | "# y_label = (-(change_pct < -alpha).astype(int)) + (change_pct > alpha).astype(int) " 307 | ] 308 | }, 309 | { 310 | "cell_type": "code", 311 | "execution_count": 23, 312 | "metadata": {}, 313 | "outputs": [], 314 | "source": [ 315 | "# np.percentile(change_pct,33.33)" 316 | ] 317 | }, 318 | { 319 | "cell_type": "code", 320 | "execution_count": 24, 321 | "metadata": {}, 322 | "outputs": [], 323 | "source": [ 324 | "# np.percentile(change_pct,66.67)" 325 | ] 326 | }, 327 | { 328 | "cell_type": "code", 329 | "execution_count": 25, 330 | "metadata": {}, 331 | "outputs": [ 332 | { 333 | "name": "stdout", 334 | "output_type": "stream", 335 | "text": [ 336 | "(159933, 40)\n" 337 | ] 338 | } 339 | ], 340 | "source": [ 341 | "X,y = generate_X_y(k=8, alpha=7e-6, timestamp_per_sample=100,\n", 342 | " daily_norm_data_dict= daily_norm_data_dict, \n", 343 | " daily_data_dict = daily_data_dict)" 344 | ] 345 | }, 346 | { 347 | "cell_type": "code", 348 | "execution_count": 26, 349 | "metadata": {}, 350 | "outputs": [], 351 | "source": [ 352 | "X_train, X_test, y_train, y_test = train_test_split(X,y,test_size = 0.2)" 353 | ] 354 | }, 355 | { 356 | "cell_type": "code", 357 | "execution_count": null, 358 | "metadata": {}, 359 | "outputs": [], 360 | "source": [] 361 | }, 362 | { 363 | "cell_type": "code", 364 | "execution_count": 27, 365 | "metadata": {}, 366 | "outputs": [ 367 | { 368 | "data": { 369 | "text/plain": [ 370 | "(127867, 100, 40, 1)" 371 | ] 372 | }, 373 | "execution_count": 27, 374 | "metadata": {}, 375 | "output_type": "execute_result" 376 | } 377 | ], 378 | "source": [ 379 | "X_train.shape" 380 | ] 381 | }, 382 | { 383 | "cell_type": "code", 384 | "execution_count": 29, 385 | "metadata": {}, 386 | "outputs": [ 387 | { 388 | "data": { 389 | "text/plain": [ 390 | "(127867, 3)" 391 | ] 392 | }, 393 | "execution_count": 29, 394 | "metadata": {}, 395 | "output_type": "execute_result" 396 | } 397 | ], 398 | "source": [ 399 | "y_train.shape" 400 | ] 401 | }, 402 | { 403 | "cell_type": "code", 404 | "execution_count": 30, 405 | "metadata": {}, 406 | "outputs": [ 407 | { 408 | "data": { 409 | "text/plain": [ 410 | "(31967, 100, 40, 1)" 411 | ] 412 | }, 413 | "execution_count": 30, 414 | "metadata": {}, 415 | "output_type": "execute_result" 416 | } 417 | ], 418 | "source": [ 419 | "X_test.shape" 420 | ] 421 | }, 422 | { 423 | "cell_type": "code", 424 | "execution_count": 31, 425 | "metadata": {}, 426 | "outputs": [ 427 | { 428 | "data": { 429 | "text/plain": [ 430 | "(31967, 3)" 431 | ] 432 | }, 433 | "execution_count": 31, 434 | "metadata": {}, 435 | "output_type": "execute_result" 436 | } 437 | ], 438 | "source": [ 439 | "y_test.shape" 440 | ] 441 | }, 442 | { 443 | "cell_type": "code", 444 | "execution_count": 72, 445 | "metadata": {}, 446 | "outputs": [], 447 | "source": [ 448 | "# del X\n", 449 | "# del y\n", 450 | "# del X2\n", 451 | "# del y2" 452 | ] 453 | }, 454 | { 455 | "cell_type": "code", 456 | "execution_count": 17, 457 | "metadata": {}, 458 | "outputs": [ 459 | { 460 | "name": "stdout", 461 | "output_type": "stream", 462 | "text": [ 463 | "(159861, 40)\n" 464 | ] 465 | } 466 | ], 467 | "source": [ 468 | "X2,y2 = generate_X_y(k=80, alpha=3.63e-5, timestamp_per_sample=100,\n", 469 | " daily_norm_data_dict= daily_norm_data_dict, \n", 470 | " daily_data_dict = daily_data_dict)" 471 | ] 472 | }, 473 | { 474 | "cell_type": "code", 475 | "execution_count": 18, 476 | "metadata": {}, 477 | "outputs": [], 478 | "source": [ 479 | "X_train2, X_test2, y_train2, y_test2 = train_test_split(X2,y2,test_size = 0.2)" 480 | ] 481 | }, 482 | { 483 | "cell_type": "code", 484 | "execution_count": 19, 485 | "metadata": {}, 486 | "outputs": [ 487 | { 488 | "name": "stdout", 489 | "output_type": "stream", 490 | "text": [ 491 | "(159915, 40)\n" 492 | ] 493 | } 494 | ], 495 | "source": [ 496 | "X3,y3 = generate_X_y(k=26, alpha=1.93e-5, timestamp_per_sample=100,\n", 497 | " daily_norm_data_dict= daily_norm_data_dict, \n", 498 | " daily_data_dict = daily_data_dict)" 499 | ] 500 | }, 501 | { 502 | "cell_type": "code", 503 | "execution_count": 20, 504 | "metadata": {}, 505 | "outputs": [], 506 | "source": [ 507 | "X_train3, X_test3, y_train3, y_test3 = train_test_split(X3,y3,test_size = 0.2)" 508 | ] 509 | }, 510 | { 511 | "cell_type": "markdown", 512 | "metadata": {}, 513 | "source": [ 514 | "#### Hypter Parameters" 515 | ] 516 | }, 517 | { 518 | "cell_type": "code", 519 | "execution_count": 22, 520 | "metadata": {}, 521 | "outputs": [], 522 | "source": [ 523 | "lookback_timestep = 100\n", 524 | "feature_num = 40\n", 525 | "\n", 526 | "#Conv param\n", 527 | "conv_filter_num = 16\n", 528 | "\n", 529 | "#Inception module param\n", 530 | "inception_num = 32\n", 531 | "\n", 532 | "#LSTM param\n", 533 | "LSTM_num = 64\n", 534 | "\n", 535 | "#Activation param\n", 536 | "leaky_relu_alpha = 0.01\n" 537 | ] 538 | }, 539 | { 540 | "cell_type": "markdown", 541 | "metadata": {}, 542 | "source": [ 543 | "#### Training Hyper Params" 544 | ] 545 | }, 546 | { 547 | "cell_type": "code", 548 | "execution_count": 23, 549 | "metadata": {}, 550 | "outputs": [], 551 | "source": [ 552 | "# categorical crossentropy loss\n", 553 | "loss = 'categorical_crossentropy'\n", 554 | "\n", 555 | "# ADAM is used\n", 556 | "learning_rate = 0.01\n", 557 | "adam_epsilon = 1\n", 558 | "optimizer = Adam(lr=learning_rate, epsilon=1)\n", 559 | "\n", 560 | "# accuracy is used for stopping training\n", 561 | "metrics = ['accuracy']\n", 562 | "\n", 563 | "#max epoch num is not specified in paper, use 120 because paper mentions training stops at about 100 epochs\n", 564 | "num_epoch = 10000\n", 565 | "#stop training when validation accuracy does not improve for 20 epochs\n", 566 | "stop_epoch_num = 20\n", 567 | "#mini-batch size 32 from paper\n", 568 | "batch_size = 32\n" 569 | ] 570 | }, 571 | { 572 | "cell_type": "markdown", 573 | "metadata": {}, 574 | "source": [ 575 | "Inception model\n", 576 | "\n", 577 | "From paper: In our case, we split the input into a small set of lowerdimensional representations by using 1 × 1 convolutions, transform the representations by a set of filters, here 3 × 1 and 5 × 1, and then merge the outputs. A max-pooling layer is used inside the Inception Module, with stride 1 and zero padding. “Inception@32” represents one module and indicates all convolutional layers have 32 filters in this module, and the approach is depicted schematically in Figure 4" 578 | ] 579 | }, 580 | { 581 | "cell_type": "markdown", 582 | "metadata": {}, 583 | "source": [ 584 | "#### Model" 585 | ] 586 | }, 587 | { 588 | "cell_type": "code", 589 | "execution_count": 24, 590 | "metadata": {}, 591 | "outputs": [], 592 | "source": [ 593 | "\n", 594 | "def initiate_DeepLOB_model(lookback_timestep, feature_num, conv_filter_num, inception_num, LSTM_num, leaky_relu_alpha,\n", 595 | " loss, optimizer, metrics):\n", 596 | " \n", 597 | " input_tensor = Input(shape=(lookback_timestep, feature_num, 1))\n", 598 | " \n", 599 | " # Conv block1\n", 600 | " print(input_tensor.shape)\n", 601 | " conv_layer1 = Conv2D(conv_filter_num, (1,2), strides=(1, 2))(input_tensor)\n", 602 | " print(conv_layer1.shape)\n", 603 | " conv_layer1 =LeakyReLU(alpha=leaky_relu_alpha)(conv_layer1)\n", 604 | " print(conv_layer1.shape)\n", 605 | " conv_layer1 = Conv2D(conv_filter_num, (4,1), padding='same')(conv_layer1)\n", 606 | " conv_first1 = LeakyReLU(alpha=leaky_relu_alpha)(conv_layer1)\n", 607 | " print(conv_layer1.shape)\n", 608 | " conv_layer1 = Conv2D(conv_filter_num, (4,1), padding='same')(conv_layer1)\n", 609 | " conv_layer1 = LeakyReLU(alpha=leaky_relu_alpha)(conv_layer1)\n", 610 | " print(conv_layer1.shape)\n", 611 | "\n", 612 | " # Conv block2\n", 613 | " conv_layer2 = Conv2D(conv_filter_num, (1,2), strides=(1, 2))(conv_layer1)\n", 614 | " conv_layer2 = LeakyReLU(alpha=leaky_relu_alpha)(conv_layer2)\n", 615 | " print(conv_layer2.shape)\n", 616 | " conv_layer2 = Conv2D(conv_filter_num, (4,1), padding='same')(conv_layer2)\n", 617 | " conv_layer2 = LeakyReLU(alpha=leaky_relu_alpha)(conv_layer2)\n", 618 | " print(conv_layer2.shape)\n", 619 | " conv_layer2 = Conv2D(conv_filter_num, (4,1), padding='same')(conv_layer2)\n", 620 | " conv_layer2 = LeakyReLU(alpha=leaky_relu_alpha)(conv_layer2)\n", 621 | " print(conv_layer2.shape)\n", 622 | "\n", 623 | " # Conv block3\n", 624 | " conv_layer3 = Conv2D(conv_filter_num, (1,10))(conv_layer2)\n", 625 | " conv_layer3 = LeakyReLU(alpha=leaky_relu_alpha)(conv_layer3)\n", 626 | " print(conv_layer3.shape)\n", 627 | " conv_layer3 = Conv2D(conv_filter_num, (4,1), padding='same')(conv_layer3)\n", 628 | " conv_layer3 = LeakyReLU(alpha=leaky_relu_alpha)(conv_layer3)\n", 629 | " print(conv_layer3.shape)\n", 630 | " conv_layer3 = Conv2D(conv_filter_num, (4,1), padding='same')(conv_layer3)\n", 631 | " conv_layer3 = LeakyReLU(alpha=leaky_relu_alpha)(conv_layer3)\n", 632 | " print(conv_layer3.shape)\n", 633 | " \n", 634 | " # Inception module\n", 635 | " inception_module1 = Conv2D(inception_num, (1,1), padding='same')(conv_layer3)\n", 636 | " inception_module1 = LeakyReLU(alpha=leaky_relu_alpha)(inception_module1)\n", 637 | " print(inception_module1.shape)\n", 638 | " inception_module1 = Conv2D(inception_num, (3,1), padding='same')(inception_module1)\n", 639 | " inception_module1 = LeakyReLU(alpha=leaky_relu_alpha)(inception_module1)\n", 640 | " print(inception_module1.shape)\n", 641 | "\n", 642 | " inception_module2 = Conv2D(inception_num, (1,1), padding='same')(conv_layer3)\n", 643 | " inception_module2 = LeakyReLU(alpha=leaky_relu_alpha)(inception_module2)\n", 644 | " print(inception_module2.shape)\n", 645 | " inception_module2 = Conv2D(inception_num, (5,1), padding='same')(inception_module2)\n", 646 | " inception_module2 = LeakyReLU(alpha=leaky_relu_alpha)(inception_module2)\n", 647 | " print(inception_module2.shape)\n", 648 | "\n", 649 | " inception_module3 = MaxPooling2D((3,1), strides=(1,1), padding='same')(conv_layer3)\n", 650 | " print(inception_module3.shape)\n", 651 | " inception_module3 = Conv2D(inception_num, (1,1), padding='same')(inception_module3)\n", 652 | " print(inception_module3.shape)\n", 653 | " inception_module3 = LeakyReLU(alpha=leaky_relu_alpha)(inception_module3)\n", 654 | " print(inception_module3.shape)\n", 655 | " \n", 656 | " inception_module_final = concatenate([inception_module1, inception_module2, inception_module3], axis=3)\n", 657 | " print(inception_module_final.shape)\n", 658 | " inception_module_final = Reshape((inception_module_final.shape[1], inception_module_final.shape[3]))(inception_module_final)\n", 659 | " print(inception_module_final.shape)\n", 660 | "\n", 661 | " # LSTM\n", 662 | " LSTM_output = LSTM(LSTM_num)(inception_module_final)\n", 663 | " print(LSTM_output.shape)\n", 664 | "\n", 665 | " # Fully Connected Layer with softmax activation function for output\n", 666 | " model_output = Dense(3, activation='softmax')(LSTM_output)\n", 667 | " print(model_output.shape)\n", 668 | " \n", 669 | " DeepLOB_model = Model(inputs=input_tensor, outputs= model_output) \n", 670 | " es = EarlyStopping(monitor='val_accuracy', mode='max', verbose=1)\n", 671 | " \n", 672 | " DeepLOB_model.compile(optimizer=optimizer, loss=loss, metrics=metrics)\n", 673 | "\n", 674 | " return DeepLOB_model\n" 675 | ] 676 | }, 677 | { 678 | "cell_type": "code", 679 | "execution_count": 35, 680 | "metadata": {}, 681 | "outputs": [ 682 | { 683 | "name": "stdout", 684 | "output_type": "stream", 685 | "text": [ 686 | "(None, 100, 40, 1)\n", 687 | "(None, 100, 20, 16)\n", 688 | "(None, 100, 20, 16)\n", 689 | "(None, 100, 20, 16)\n", 690 | "(None, 100, 20, 16)\n", 691 | "(None, 100, 10, 16)\n", 692 | "(None, 100, 10, 16)\n", 693 | "(None, 100, 10, 16)\n", 694 | "(None, 100, 1, 16)\n", 695 | "(None, 100, 1, 16)\n", 696 | "(None, 100, 1, 16)\n", 697 | "(None, 100, 1, 32)\n", 698 | "(None, 100, 1, 32)\n", 699 | "(None, 100, 1, 32)\n", 700 | "(None, 100, 1, 32)\n", 701 | "(None, 100, 1, 16)\n", 702 | "(None, 100, 1, 32)\n", 703 | "(None, 100, 1, 32)\n", 704 | "(None, 100, 1, 96)\n", 705 | "(None, 100, 96)\n", 706 | "(None, 64)\n", 707 | "(None, 3)\n" 708 | ] 709 | } 710 | ], 711 | "source": [ 712 | "DeepLOB_model = initiate_DeepLOB_model(lookback_timestep, feature_num, conv_filter_num, inception_num, LSTM_num, leaky_relu_alpha,\n", 713 | " loss, optimizer, metrics)" 714 | ] 715 | }, 716 | { 717 | "cell_type": "code", 718 | "execution_count": 36, 719 | "metadata": {}, 720 | "outputs": [ 721 | { 722 | "name": "stdout", 723 | "output_type": "stream", 724 | "text": [ 725 | "Model: \"model\"\n", 726 | "__________________________________________________________________________________________________\n", 727 | "Layer (type) Output Shape Param # Connected to \n", 728 | "==================================================================================================\n", 729 | "input_1 (InputLayer) [(None, 100, 40, 1)] 0 \n", 730 | "__________________________________________________________________________________________________\n", 731 | "conv2d (Conv2D) (None, 100, 20, 16) 48 input_1[0][0] \n", 732 | "__________________________________________________________________________________________________\n", 733 | "leaky_re_lu (LeakyReLU) (None, 100, 20, 16) 0 conv2d[0][0] \n", 734 | "__________________________________________________________________________________________________\n", 735 | "conv2d_1 (Conv2D) (None, 100, 20, 16) 1040 leaky_re_lu[0][0] \n", 736 | "__________________________________________________________________________________________________\n", 737 | "conv2d_2 (Conv2D) (None, 100, 20, 16) 1040 conv2d_1[0][0] \n", 738 | "__________________________________________________________________________________________________\n", 739 | "leaky_re_lu_2 (LeakyReLU) (None, 100, 20, 16) 0 conv2d_2[0][0] \n", 740 | "__________________________________________________________________________________________________\n", 741 | "conv2d_3 (Conv2D) (None, 100, 10, 16) 528 leaky_re_lu_2[0][0] \n", 742 | "__________________________________________________________________________________________________\n", 743 | "leaky_re_lu_3 (LeakyReLU) (None, 100, 10, 16) 0 conv2d_3[0][0] \n", 744 | "__________________________________________________________________________________________________\n", 745 | "conv2d_4 (Conv2D) (None, 100, 10, 16) 1040 leaky_re_lu_3[0][0] \n", 746 | "__________________________________________________________________________________________________\n", 747 | "leaky_re_lu_4 (LeakyReLU) (None, 100, 10, 16) 0 conv2d_4[0][0] \n", 748 | "__________________________________________________________________________________________________\n", 749 | "conv2d_5 (Conv2D) (None, 100, 10, 16) 1040 leaky_re_lu_4[0][0] \n", 750 | "__________________________________________________________________________________________________\n", 751 | "leaky_re_lu_5 (LeakyReLU) (None, 100, 10, 16) 0 conv2d_5[0][0] \n", 752 | "__________________________________________________________________________________________________\n", 753 | "conv2d_6 (Conv2D) (None, 100, 1, 16) 2576 leaky_re_lu_5[0][0] \n", 754 | "__________________________________________________________________________________________________\n", 755 | "leaky_re_lu_6 (LeakyReLU) (None, 100, 1, 16) 0 conv2d_6[0][0] \n", 756 | "__________________________________________________________________________________________________\n", 757 | "conv2d_7 (Conv2D) (None, 100, 1, 16) 1040 leaky_re_lu_6[0][0] \n", 758 | "__________________________________________________________________________________________________\n", 759 | "leaky_re_lu_7 (LeakyReLU) (None, 100, 1, 16) 0 conv2d_7[0][0] \n", 760 | "__________________________________________________________________________________________________\n", 761 | "conv2d_8 (Conv2D) (None, 100, 1, 16) 1040 leaky_re_lu_7[0][0] \n", 762 | "__________________________________________________________________________________________________\n", 763 | "leaky_re_lu_8 (LeakyReLU) (None, 100, 1, 16) 0 conv2d_8[0][0] \n", 764 | "__________________________________________________________________________________________________\n", 765 | "conv2d_9 (Conv2D) (None, 100, 1, 32) 544 leaky_re_lu_8[0][0] \n", 766 | "__________________________________________________________________________________________________\n", 767 | "conv2d_11 (Conv2D) (None, 100, 1, 32) 544 leaky_re_lu_8[0][0] \n", 768 | "__________________________________________________________________________________________________\n", 769 | "leaky_re_lu_9 (LeakyReLU) (None, 100, 1, 32) 0 conv2d_9[0][0] \n", 770 | "__________________________________________________________________________________________________\n", 771 | "leaky_re_lu_11 (LeakyReLU) (None, 100, 1, 32) 0 conv2d_11[0][0] \n", 772 | "__________________________________________________________________________________________________\n", 773 | "max_pooling2d (MaxPooling2D) (None, 100, 1, 16) 0 leaky_re_lu_8[0][0] \n", 774 | "__________________________________________________________________________________________________\n", 775 | "conv2d_10 (Conv2D) (None, 100, 1, 32) 3104 leaky_re_lu_9[0][0] \n", 776 | "__________________________________________________________________________________________________\n", 777 | "conv2d_12 (Conv2D) (None, 100, 1, 32) 5152 leaky_re_lu_11[0][0] \n", 778 | "__________________________________________________________________________________________________\n", 779 | "conv2d_13 (Conv2D) (None, 100, 1, 32) 544 max_pooling2d[0][0] \n", 780 | "__________________________________________________________________________________________________\n", 781 | "leaky_re_lu_10 (LeakyReLU) (None, 100, 1, 32) 0 conv2d_10[0][0] \n", 782 | "__________________________________________________________________________________________________\n", 783 | "leaky_re_lu_12 (LeakyReLU) (None, 100, 1, 32) 0 conv2d_12[0][0] \n", 784 | "__________________________________________________________________________________________________\n", 785 | "leaky_re_lu_13 (LeakyReLU) (None, 100, 1, 32) 0 conv2d_13[0][0] \n", 786 | "__________________________________________________________________________________________________\n", 787 | "concatenate (Concatenate) (None, 100, 1, 96) 0 leaky_re_lu_10[0][0] \n", 788 | " leaky_re_lu_12[0][0] \n", 789 | " leaky_re_lu_13[0][0] \n", 790 | "__________________________________________________________________________________________________\n", 791 | "reshape (Reshape) (None, 100, 96) 0 concatenate[0][0] \n", 792 | "__________________________________________________________________________________________________\n", 793 | "lstm (LSTM) (None, 64) 41216 reshape[0][0] \n", 794 | "__________________________________________________________________________________________________\n", 795 | "dense (Dense) (None, 3) 195 lstm[0][0] \n", 796 | "==================================================================================================\n", 797 | "Total params: 60,691\n", 798 | "Trainable params: 60,691\n", 799 | "Non-trainable params: 0\n", 800 | "__________________________________________________________________________________________________\n" 801 | ] 802 | } 803 | ], 804 | "source": [ 805 | "DeepLOB_model.summary()" 806 | ] 807 | }, 808 | { 809 | "cell_type": "markdown", 810 | "metadata": {}, 811 | "source": [ 812 | "#### Model Training" 813 | ] 814 | }, 815 | { 816 | "cell_type": "code", 817 | "execution_count": 44, 818 | "metadata": {}, 819 | "outputs": [ 820 | { 821 | "name": "stdout", 822 | "output_type": "stream", 823 | "text": [ 824 | "Epoch 1/10000\n", 825 | "3996/3996 - 315s - loss: 1.0985 - accuracy: 0.3387 - val_loss: 1.0984 - val_accuracy: 0.3381\n", 826 | "Epoch 2/10000\n", 827 | "3996/3996 - 352s - loss: 1.0980 - accuracy: 0.3441 - val_loss: 1.0973 - val_accuracy: 0.3445\n", 828 | "Epoch 3/10000\n", 829 | "3996/3996 - 311s - loss: 1.0945 - accuracy: 0.3667 - val_loss: 1.0889 - val_accuracy: 0.3887\n", 830 | "Epoch 4/10000\n", 831 | "3996/3996 - 297s - loss: 1.0681 - accuracy: 0.4186 - val_loss: 1.0506 - val_accuracy: 0.4420\n", 832 | "Epoch 5/10000\n", 833 | "3996/3996 - 326s - loss: 1.0330 - accuracy: 0.4594 - val_loss: 1.0238 - val_accuracy: 0.4697\n", 834 | "Epoch 6/10000\n", 835 | "3996/3996 - 335s - loss: 1.0199 - accuracy: 0.4744 - val_loss: 1.0192 - val_accuracy: 0.4770\n", 836 | "Epoch 7/10000\n", 837 | "3996/3996 - 369s - loss: 1.0140 - accuracy: 0.4824 - val_loss: 1.0146 - val_accuracy: 0.4837\n", 838 | "Epoch 8/10000\n", 839 | "3996/3996 - 365s - loss: 1.0097 - accuracy: 0.4873 - val_loss: 1.0114 - val_accuracy: 0.4798\n", 840 | "Epoch 9/10000\n", 841 | "3996/3996 - 349s - loss: 1.0058 - accuracy: 0.4915 - val_loss: 1.0036 - val_accuracy: 0.4951\n", 842 | "Epoch 10/10000\n", 843 | "3996/3996 - 349s - loss: 1.0024 - accuracy: 0.4953 - val_loss: 1.0025 - val_accuracy: 0.4967\n", 844 | "Epoch 11/10000\n", 845 | "3996/3996 - 357s - loss: 0.9996 - accuracy: 0.4978 - val_loss: 1.0000 - val_accuracy: 0.4996\n", 846 | "Epoch 12/10000\n", 847 | "3996/3996 - 376s - loss: 0.9969 - accuracy: 0.4999 - val_loss: 0.9991 - val_accuracy: 0.4969\n", 848 | "Epoch 13/10000\n", 849 | "3996/3996 - 380s - loss: 0.9944 - accuracy: 0.5016 - val_loss: 0.9908 - val_accuracy: 0.5084\n", 850 | "Epoch 14/10000\n", 851 | "3996/3996 - 437s - loss: 0.9917 - accuracy: 0.5062 - val_loss: 0.9936 - val_accuracy: 0.5019\n", 852 | "Epoch 15/10000\n", 853 | "3996/3996 - 372s - loss: 0.9896 - accuracy: 0.5075 - val_loss: 0.9914 - val_accuracy: 0.5072\n", 854 | "Epoch 16/10000\n", 855 | "3996/3996 - 373s - loss: 0.9875 - accuracy: 0.5086 - val_loss: 0.9853 - val_accuracy: 0.5108\n", 856 | "Epoch 17/10000\n", 857 | "3996/3996 - 346s - loss: 0.9850 - accuracy: 0.5123 - val_loss: 0.9868 - val_accuracy: 0.5099\n", 858 | "Epoch 18/10000\n", 859 | "3996/3996 - 367s - loss: 0.9833 - accuracy: 0.5135 - val_loss: 0.9854 - val_accuracy: 0.5089\n", 860 | "Epoch 19/10000\n", 861 | "3996/3996 - 362s - loss: 0.9817 - accuracy: 0.5140 - val_loss: 0.9832 - val_accuracy: 0.5110\n", 862 | "Epoch 20/10000\n", 863 | "3996/3996 - 370s - loss: 0.9799 - accuracy: 0.5159 - val_loss: 0.9874 - val_accuracy: 0.5109\n", 864 | "Epoch 21/10000\n", 865 | "3996/3996 - 373s - loss: 0.9782 - accuracy: 0.5187 - val_loss: 0.9830 - val_accuracy: 0.5133\n", 866 | "Epoch 22/10000\n", 867 | "3996/3996 - 980s - loss: 0.9767 - accuracy: 0.5194 - val_loss: 0.9810 - val_accuracy: 0.5178\n", 868 | "Epoch 23/10000\n", 869 | "3996/3996 - 1030s - loss: 0.9753 - accuracy: 0.5213 - val_loss: 0.9789 - val_accuracy: 0.5165\n", 870 | "Epoch 24/10000\n", 871 | "3996/3996 - 730s - loss: 0.9733 - accuracy: 0.5222 - val_loss: 0.9858 - val_accuracy: 0.5107\n", 872 | "Epoch 25/10000\n", 873 | "3996/3996 - 371s - loss: 0.9714 - accuracy: 0.5251 - val_loss: 0.9773 - val_accuracy: 0.5198\n", 874 | "Epoch 26/10000\n", 875 | "3996/3996 - 377s - loss: 0.9698 - accuracy: 0.5260 - val_loss: 0.9732 - val_accuracy: 0.5255\n", 876 | "Epoch 27/10000\n", 877 | "3996/3996 - 362s - loss: 0.9679 - accuracy: 0.5273 - val_loss: 0.9782 - val_accuracy: 0.5165\n", 878 | "Epoch 28/10000\n", 879 | "3996/3996 - 712s - loss: 0.9663 - accuracy: 0.5284 - val_loss: 0.9729 - val_accuracy: 0.5245\n", 880 | "Epoch 29/10000\n", 881 | "3996/3996 - 1104s - loss: 0.9646 - accuracy: 0.5293 - val_loss: 0.9720 - val_accuracy: 0.5266\n", 882 | "Epoch 30/10000\n", 883 | "3996/3996 - 389s - loss: 0.9629 - accuracy: 0.5321 - val_loss: 0.9724 - val_accuracy: 0.5212\n", 884 | "Epoch 31/10000\n", 885 | "3996/3996 - 369s - loss: 0.9606 - accuracy: 0.5334 - val_loss: 0.9736 - val_accuracy: 0.5221\n", 886 | "Epoch 32/10000\n", 887 | "3996/3996 - 377s - loss: 0.9592 - accuracy: 0.5361 - val_loss: 0.9694 - val_accuracy: 0.5254\n", 888 | "Epoch 33/10000\n", 889 | "3996/3996 - 376s - loss: 0.9575 - accuracy: 0.5378 - val_loss: 0.9646 - val_accuracy: 0.5328\n", 890 | "Epoch 34/10000\n", 891 | "3996/3996 - 375s - loss: 0.9552 - accuracy: 0.5387 - val_loss: 0.9648 - val_accuracy: 0.5326\n", 892 | "Epoch 35/10000\n", 893 | "3996/3996 - 372s - loss: 0.9525 - accuracy: 0.5404 - val_loss: 0.9684 - val_accuracy: 0.5258\n", 894 | "Epoch 36/10000\n", 895 | "3996/3996 - 373s - loss: 0.9508 - accuracy: 0.5431 - val_loss: 0.9657 - val_accuracy: 0.5295\n", 896 | "Epoch 37/10000\n", 897 | "3996/3996 - 374s - loss: 0.9489 - accuracy: 0.5445 - val_loss: 0.9628 - val_accuracy: 0.5309\n", 898 | "Epoch 38/10000\n", 899 | "3996/3996 - 374s - loss: 0.9466 - accuracy: 0.5456 - val_loss: 0.9844 - val_accuracy: 0.5122\n", 900 | "Epoch 39/10000\n", 901 | "3996/3996 - 373s - loss: 0.9439 - accuracy: 0.5485 - val_loss: 0.9576 - val_accuracy: 0.5374\n", 902 | "Epoch 40/10000\n", 903 | "3996/3996 - 378s - loss: 0.9423 - accuracy: 0.5488 - val_loss: 0.9596 - val_accuracy: 0.5347\n", 904 | "Epoch 41/10000\n", 905 | "3996/3996 - 381s - loss: 0.9398 - accuracy: 0.5504 - val_loss: 0.9561 - val_accuracy: 0.5386\n", 906 | "Epoch 42/10000\n", 907 | "3996/3996 - 336s - loss: 0.9374 - accuracy: 0.5526 - val_loss: 0.9534 - val_accuracy: 0.5411\n", 908 | "Epoch 43/10000\n", 909 | "3996/3996 - 334s - loss: 0.9347 - accuracy: 0.5545 - val_loss: 0.9530 - val_accuracy: 0.5391\n", 910 | "Epoch 44/10000\n", 911 | "3996/3996 - 351s - loss: 0.9313 - accuracy: 0.5577 - val_loss: 0.9531 - val_accuracy: 0.5381\n", 912 | "Epoch 45/10000\n", 913 | "3996/3996 - 370s - loss: 0.9295 - accuracy: 0.5592 - val_loss: 0.9537 - val_accuracy: 0.5384\n", 914 | "Epoch 46/10000\n", 915 | "3996/3996 - 363s - loss: 0.9267 - accuracy: 0.5605 - val_loss: 0.9496 - val_accuracy: 0.5409\n", 916 | "Epoch 47/10000\n", 917 | "3996/3996 - 377s - loss: 0.9241 - accuracy: 0.5634 - val_loss: 0.9493 - val_accuracy: 0.5469\n", 918 | "Epoch 48/10000\n", 919 | "3996/3996 - 372s - loss: 0.9217 - accuracy: 0.5642 - val_loss: 0.9444 - val_accuracy: 0.5455\n", 920 | "Epoch 49/10000\n", 921 | "3996/3996 - 379s - loss: 0.9187 - accuracy: 0.5665 - val_loss: 0.9449 - val_accuracy: 0.5475\n", 922 | "Epoch 50/10000\n", 923 | "3996/3996 - 379s - loss: 0.9156 - accuracy: 0.5687 - val_loss: 0.9415 - val_accuracy: 0.5488\n", 924 | "Epoch 51/10000\n", 925 | "3996/3996 - 346s - loss: 0.9117 - accuracy: 0.5703 - val_loss: 0.9396 - val_accuracy: 0.5508\n", 926 | "Epoch 52/10000\n", 927 | "3996/3996 - 337s - loss: 0.9086 - accuracy: 0.5730 - val_loss: 0.9372 - val_accuracy: 0.5569\n", 928 | "Epoch 53/10000\n", 929 | "3996/3996 - 344s - loss: 0.9042 - accuracy: 0.5758 - val_loss: 0.9414 - val_accuracy: 0.5500\n", 930 | "Epoch 54/10000\n", 931 | "3996/3996 - 351s - loss: 0.9008 - accuracy: 0.5787 - val_loss: 0.9301 - val_accuracy: 0.5581\n", 932 | "Epoch 55/10000\n", 933 | "3996/3996 - 374s - loss: 0.8971 - accuracy: 0.5830 - val_loss: 0.9339 - val_accuracy: 0.5608\n", 934 | "Epoch 56/10000\n", 935 | "3996/3996 - 370s - loss: 0.8941 - accuracy: 0.5849 - val_loss: 0.9297 - val_accuracy: 0.5583\n", 936 | "Epoch 57/10000\n", 937 | "3996/3996 - 367s - loss: 0.8913 - accuracy: 0.5860 - val_loss: 0.9190 - val_accuracy: 0.5679\n", 938 | "Epoch 58/10000\n", 939 | "3996/3996 - 350s - loss: 0.8869 - accuracy: 0.5885 - val_loss: 0.9223 - val_accuracy: 0.5675\n", 940 | "Epoch 59/10000\n", 941 | "3996/3996 - 358s - loss: 0.8842 - accuracy: 0.5905 - val_loss: 0.9260 - val_accuracy: 0.5630\n", 942 | "Epoch 60/10000\n", 943 | "3996/3996 - 359s - loss: 0.8796 - accuracy: 0.5948 - val_loss: 0.9200 - val_accuracy: 0.5675\n", 944 | "Epoch 61/10000\n", 945 | "3996/3996 - 352s - loss: 0.8772 - accuracy: 0.5962 - val_loss: 0.9208 - val_accuracy: 0.5679\n", 946 | "Epoch 62/10000\n", 947 | "3996/3996 - 356s - loss: 0.8713 - accuracy: 0.5995 - val_loss: 0.9150 - val_accuracy: 0.5724\n", 948 | "Epoch 63/10000\n", 949 | "3996/3996 - 319s - loss: 0.8675 - accuracy: 0.6019 - val_loss: 0.9258 - val_accuracy: 0.5667\n", 950 | "Epoch 64/10000\n", 951 | "3996/3996 - 314s - loss: 0.8637 - accuracy: 0.6059 - val_loss: 0.9051 - val_accuracy: 0.5784\n", 952 | "Epoch 65/10000\n", 953 | "3996/3996 - 315s - loss: 0.8595 - accuracy: 0.6073 - val_loss: 0.8974 - val_accuracy: 0.5831\n", 954 | "Epoch 66/10000\n", 955 | "3996/3996 - 313s - loss: 0.8569 - accuracy: 0.6092 - val_loss: 0.9017 - val_accuracy: 0.5834\n", 956 | "Epoch 67/10000\n", 957 | "3996/3996 - 315s - loss: 0.8525 - accuracy: 0.6124 - val_loss: 0.9265 - val_accuracy: 0.5656\n", 958 | "Epoch 68/10000\n", 959 | "3996/3996 - 326s - loss: 0.8487 - accuracy: 0.6143 - val_loss: 0.9111 - val_accuracy: 0.5813\n", 960 | "Epoch 69/10000\n", 961 | "3996/3996 - 324s - loss: 0.8451 - accuracy: 0.6171 - val_loss: 0.9075 - val_accuracy: 0.5796\n", 962 | "Epoch 70/10000\n", 963 | "3996/3996 - 324s - loss: 0.8398 - accuracy: 0.6206 - val_loss: 0.9241 - val_accuracy: 0.5662\n", 964 | "Epoch 71/10000\n", 965 | "3996/3996 - 326s - loss: 0.8377 - accuracy: 0.6209 - val_loss: 0.8851 - val_accuracy: 0.5907\n", 966 | "Epoch 72/10000\n", 967 | "3996/3996 - 333s - loss: 0.8318 - accuracy: 0.6247 - val_loss: 0.9203 - val_accuracy: 0.5740\n", 968 | "Epoch 73/10000\n", 969 | "3996/3996 - 329s - loss: 0.8287 - accuracy: 0.6268 - val_loss: 0.8775 - val_accuracy: 0.5990\n", 970 | "Epoch 74/10000\n", 971 | "3996/3996 - 326s - loss: 0.8250 - accuracy: 0.6290 - val_loss: 0.8905 - val_accuracy: 0.5882\n", 972 | "Epoch 75/10000\n", 973 | "3996/3996 - 336s - loss: 0.8194 - accuracy: 0.6325 - val_loss: 0.8880 - val_accuracy: 0.5940\n", 974 | "Epoch 76/10000\n", 975 | "3996/3996 - 345s - loss: 0.8140 - accuracy: 0.6358 - val_loss: 0.8862 - val_accuracy: 0.5967\n" 976 | ] 977 | }, 978 | { 979 | "name": "stdout", 980 | "output_type": "stream", 981 | "text": [ 982 | "Epoch 77/10000\n", 983 | "3996/3996 - 332s - loss: 0.8113 - accuracy: 0.6377 - val_loss: 0.8903 - val_accuracy: 0.5929\n", 984 | "Epoch 78/10000\n", 985 | "3996/3996 - 326s - loss: 0.8088 - accuracy: 0.6401 - val_loss: 0.8831 - val_accuracy: 0.5955\n", 986 | "Epoch 79/10000\n", 987 | "3996/3996 - 324s - loss: 0.8020 - accuracy: 0.6434 - val_loss: 0.8900 - val_accuracy: 0.5936\n", 988 | "Epoch 80/10000\n", 989 | "3996/3996 - 326s - loss: 0.8004 - accuracy: 0.6432 - val_loss: 0.8783 - val_accuracy: 0.6022\n", 990 | "Epoch 81/10000\n", 991 | "3996/3996 - 328s - loss: 0.7949 - accuracy: 0.6481 - val_loss: 0.8923 - val_accuracy: 0.5955\n", 992 | "Epoch 82/10000\n", 993 | "3996/3996 - 335s - loss: 0.7921 - accuracy: 0.6502 - val_loss: 0.8768 - val_accuracy: 0.6055\n", 994 | "Epoch 83/10000\n", 995 | "3996/3996 - 329s - loss: 0.7881 - accuracy: 0.6523 - val_loss: 0.8684 - val_accuracy: 0.6067\n", 996 | "Epoch 84/10000\n", 997 | "3996/3996 - 327s - loss: 0.7837 - accuracy: 0.6538 - val_loss: 0.8756 - val_accuracy: 0.6045\n", 998 | "Epoch 85/10000\n", 999 | "3996/3996 - 328s - loss: 0.7792 - accuracy: 0.6568 - val_loss: 0.8888 - val_accuracy: 0.5974\n", 1000 | "Epoch 86/10000\n", 1001 | "3996/3996 - 360s - loss: 0.7759 - accuracy: 0.6576 - val_loss: 0.8590 - val_accuracy: 0.6165\n", 1002 | "Epoch 87/10000\n", 1003 | "3996/3996 - 369s - loss: 0.7686 - accuracy: 0.6621 - val_loss: 0.8476 - val_accuracy: 0.6216\n", 1004 | "Epoch 88/10000\n", 1005 | "3996/3996 - 362s - loss: 0.7693 - accuracy: 0.6625 - val_loss: 0.8548 - val_accuracy: 0.6179\n", 1006 | "Epoch 89/10000\n", 1007 | "3996/3996 - 330s - loss: 0.7641 - accuracy: 0.6667 - val_loss: 0.8588 - val_accuracy: 0.6167\n", 1008 | "Epoch 90/10000\n", 1009 | "3996/3996 - 329s - loss: 0.7629 - accuracy: 0.6677 - val_loss: 0.8656 - val_accuracy: 0.6144\n", 1010 | "Epoch 91/10000\n", 1011 | "3996/3996 - 352s - loss: 0.7552 - accuracy: 0.6710 - val_loss: 0.8663 - val_accuracy: 0.6079\n", 1012 | "Epoch 92/10000\n", 1013 | "3996/3996 - 346s - loss: 0.7526 - accuracy: 0.6715 - val_loss: 0.8723 - val_accuracy: 0.6079\n", 1014 | "Epoch 93/10000\n", 1015 | "3996/3996 - 346s - loss: 0.7500 - accuracy: 0.6727 - val_loss: 0.8648 - val_accuracy: 0.6149\n", 1016 | "Epoch 94/10000\n", 1017 | "3996/3996 - 346s - loss: 0.7467 - accuracy: 0.6745 - val_loss: 0.8720 - val_accuracy: 0.6092\n", 1018 | "Epoch 95/10000\n", 1019 | "3996/3996 - 338s - loss: 0.7417 - accuracy: 0.6778 - val_loss: 0.8753 - val_accuracy: 0.6067\n", 1020 | "Epoch 96/10000\n", 1021 | "3996/3996 - 359s - loss: 0.7348 - accuracy: 0.6819 - val_loss: 0.8387 - val_accuracy: 0.6269\n", 1022 | "Epoch 97/10000\n", 1023 | "3996/3996 - 356s - loss: 0.7348 - accuracy: 0.6814 - val_loss: 0.8338 - val_accuracy: 0.6310\n", 1024 | "Epoch 98/10000\n", 1025 | "3996/3996 - 369s - loss: 0.7307 - accuracy: 0.6846 - val_loss: 0.8503 - val_accuracy: 0.6240\n", 1026 | "Epoch 99/10000\n", 1027 | "3996/3996 - 361s - loss: 0.7257 - accuracy: 0.6873 - val_loss: 0.8352 - val_accuracy: 0.6310\n", 1028 | "Epoch 100/10000\n", 1029 | "3996/3996 - 368s - loss: 0.7189 - accuracy: 0.6916 - val_loss: 0.8307 - val_accuracy: 0.6360\n", 1030 | "Epoch 101/10000\n", 1031 | "3996/3996 - 365s - loss: 0.7198 - accuracy: 0.6910 - val_loss: 0.8419 - val_accuracy: 0.6301\n", 1032 | "Epoch 102/10000\n", 1033 | "3996/3996 - 367s - loss: 0.7180 - accuracy: 0.6923 - val_loss: 0.8247 - val_accuracy: 0.6392\n", 1034 | "Epoch 103/10000\n", 1035 | "3996/3996 - 363s - loss: 0.7169 - accuracy: 0.6922 - val_loss: 0.8255 - val_accuracy: 0.6385\n", 1036 | "Epoch 104/10000\n", 1037 | "3996/3996 - 368s - loss: 0.7095 - accuracy: 0.6961 - val_loss: 0.8357 - val_accuracy: 0.6363\n", 1038 | "Epoch 105/10000\n", 1039 | "3996/3996 - 375s - loss: 0.7057 - accuracy: 0.6977 - val_loss: 0.8180 - val_accuracy: 0.6393\n", 1040 | "Epoch 106/10000\n", 1041 | "3996/3996 - 374s - loss: 0.7019 - accuracy: 0.6992 - val_loss: 0.8318 - val_accuracy: 0.6366\n", 1042 | "Epoch 107/10000\n", 1043 | "3996/3996 - 367s - loss: 0.6977 - accuracy: 0.7022 - val_loss: 0.8155 - val_accuracy: 0.6410\n", 1044 | "Epoch 108/10000\n", 1045 | "3996/3996 - 650s - loss: 0.6996 - accuracy: 0.7023 - val_loss: 0.8185 - val_accuracy: 0.6418\n", 1046 | "Epoch 109/10000\n", 1047 | "3996/3996 - 934s - loss: 0.6926 - accuracy: 0.7047 - val_loss: 0.8382 - val_accuracy: 0.6327\n", 1048 | "Epoch 110/10000\n", 1049 | "3996/3996 - 398s - loss: 0.6916 - accuracy: 0.7054 - val_loss: 0.8058 - val_accuracy: 0.6509\n", 1050 | "Epoch 111/10000\n", 1051 | "3996/3996 - 346s - loss: 0.6854 - accuracy: 0.7093 - val_loss: 0.8309 - val_accuracy: 0.6426\n", 1052 | "Epoch 112/10000\n", 1053 | "3996/3996 - 341s - loss: 0.6829 - accuracy: 0.7108 - val_loss: 0.8440 - val_accuracy: 0.6371\n", 1054 | "Epoch 113/10000\n", 1055 | "3996/3996 - 531s - loss: 0.6824 - accuracy: 0.7097 - val_loss: 0.8164 - val_accuracy: 0.6482\n", 1056 | "Epoch 114/10000\n", 1057 | "3996/3996 - 773s - loss: 0.6782 - accuracy: 0.7123 - val_loss: 0.7949 - val_accuracy: 0.6540\n", 1058 | "Epoch 115/10000\n", 1059 | "3996/3996 - 342s - loss: 0.6741 - accuracy: 0.7141 - val_loss: 0.7992 - val_accuracy: 0.6583\n", 1060 | "Epoch 116/10000\n", 1061 | "3996/3996 - 341s - loss: 0.6734 - accuracy: 0.7154 - val_loss: 0.8138 - val_accuracy: 0.6459\n", 1062 | "Epoch 117/10000\n", 1063 | "3996/3996 - 356s - loss: 0.6686 - accuracy: 0.7151 - val_loss: 0.8253 - val_accuracy: 0.6426\n", 1064 | "Epoch 118/10000\n", 1065 | "3996/3996 - 345s - loss: 0.6655 - accuracy: 0.7193 - val_loss: 0.8151 - val_accuracy: 0.6518\n", 1066 | "Epoch 119/10000\n", 1067 | "3996/3996 - 354s - loss: 0.6628 - accuracy: 0.7207 - val_loss: 0.8169 - val_accuracy: 0.6479\n", 1068 | "Epoch 120/10000\n", 1069 | "3996/3996 - 350s - loss: 0.6630 - accuracy: 0.7204 - val_loss: 0.8152 - val_accuracy: 0.6448\n", 1070 | "Epoch 121/10000\n", 1071 | "3996/3996 - 341s - loss: 0.6608 - accuracy: 0.7217 - val_loss: 0.8189 - val_accuracy: 0.6502\n", 1072 | "Epoch 122/10000\n", 1073 | "3996/3996 - 367s - loss: 0.6536 - accuracy: 0.7256 - val_loss: 0.7989 - val_accuracy: 0.6625\n", 1074 | "Epoch 123/10000\n", 1075 | "3996/3996 - 377s - loss: 0.6517 - accuracy: 0.7272 - val_loss: 0.7929 - val_accuracy: 0.6592\n", 1076 | "Epoch 124/10000\n", 1077 | "3996/3996 - 383s - loss: 0.6549 - accuracy: 0.7234 - val_loss: 0.7846 - val_accuracy: 0.6646\n", 1078 | "Epoch 125/10000\n", 1079 | "3996/3996 - 374s - loss: 0.6481 - accuracy: 0.7271 - val_loss: 0.8435 - val_accuracy: 0.6360\n", 1080 | "Epoch 126/10000\n", 1081 | "3996/3996 - 365s - loss: 0.6445 - accuracy: 0.7299 - val_loss: 0.7787 - val_accuracy: 0.6670\n", 1082 | "Epoch 127/10000\n", 1083 | "3996/3996 - 380s - loss: 0.6434 - accuracy: 0.7293 - val_loss: 0.8245 - val_accuracy: 0.6405\n", 1084 | "Epoch 128/10000\n", 1085 | "3996/3996 - 372s - loss: 0.6424 - accuracy: 0.7302 - val_loss: 0.8148 - val_accuracy: 0.6506\n", 1086 | "Epoch 129/10000\n", 1087 | "3996/3996 - 359s - loss: 0.6373 - accuracy: 0.7329 - val_loss: 0.8133 - val_accuracy: 0.6518\n", 1088 | "Epoch 130/10000\n", 1089 | "3996/3996 - 391s - loss: 0.6388 - accuracy: 0.7333 - val_loss: 0.7965 - val_accuracy: 0.6607\n", 1090 | "Epoch 131/10000\n", 1091 | "3996/3996 - 337s - loss: 0.6409 - accuracy: 0.7318 - val_loss: 0.7892 - val_accuracy: 0.6636\n", 1092 | "Epoch 132/10000\n", 1093 | "3996/3996 - 298s - loss: 0.6341 - accuracy: 0.7349 - val_loss: 0.7922 - val_accuracy: 0.6691\n", 1094 | "Epoch 133/10000\n", 1095 | "3996/3996 - 296s - loss: 0.6304 - accuracy: 0.7364 - val_loss: 0.8010 - val_accuracy: 0.6596\n", 1096 | "Epoch 134/10000\n", 1097 | "3996/3996 - 295s - loss: 0.6253 - accuracy: 0.7378 - val_loss: 0.7688 - val_accuracy: 0.6759\n", 1098 | "Epoch 135/10000\n", 1099 | "3996/3996 - 295s - loss: 0.6235 - accuracy: 0.7392 - val_loss: 0.7741 - val_accuracy: 0.6741\n", 1100 | "Epoch 136/10000\n", 1101 | "3996/3996 - 296s - loss: 0.6233 - accuracy: 0.7398 - val_loss: 0.7619 - val_accuracy: 0.6787\n", 1102 | "Epoch 137/10000\n", 1103 | "3996/3996 - 295s - loss: 0.6236 - accuracy: 0.7411 - val_loss: 0.7964 - val_accuracy: 0.6670\n", 1104 | "Epoch 138/10000\n", 1105 | "3996/3996 - 294s - loss: 0.6181 - accuracy: 0.7436 - val_loss: 0.7771 - val_accuracy: 0.6700\n", 1106 | "Epoch 139/10000\n", 1107 | "3996/3996 - 296s - loss: 0.6205 - accuracy: 0.7418 - val_loss: 0.7737 - val_accuracy: 0.6725\n", 1108 | "Epoch 140/10000\n", 1109 | "3996/3996 - 294s - loss: 0.6113 - accuracy: 0.7463 - val_loss: 0.7817 - val_accuracy: 0.6699\n", 1110 | "Epoch 141/10000\n", 1111 | "3996/3996 - 294s - loss: 0.6134 - accuracy: 0.7448 - val_loss: 0.7852 - val_accuracy: 0.6739\n", 1112 | "Epoch 142/10000\n", 1113 | "3996/3996 - 296s - loss: 0.6133 - accuracy: 0.7453 - val_loss: 0.7526 - val_accuracy: 0.6829\n", 1114 | "Epoch 143/10000\n", 1115 | "3996/3996 - 294s - loss: 0.6107 - accuracy: 0.7473 - val_loss: 0.7639 - val_accuracy: 0.6795\n", 1116 | "Epoch 144/10000\n", 1117 | "3996/3996 - 293s - loss: 0.6071 - accuracy: 0.7478 - val_loss: 0.8044 - val_accuracy: 0.6629\n", 1118 | "Epoch 145/10000\n", 1119 | "3996/3996 - 292s - loss: 0.6074 - accuracy: 0.7485 - val_loss: 0.7947 - val_accuracy: 0.6671\n", 1120 | "Epoch 146/10000\n", 1121 | "3996/3996 - 292s - loss: 0.6040 - accuracy: 0.7498 - val_loss: 0.7879 - val_accuracy: 0.6692\n", 1122 | "Epoch 147/10000\n", 1123 | "3996/3996 - 291s - loss: 0.6029 - accuracy: 0.7483 - val_loss: 0.7564 - val_accuracy: 0.6802\n", 1124 | "Epoch 148/10000\n", 1125 | "3996/3996 - 296s - loss: 0.5978 - accuracy: 0.7523 - val_loss: 0.7688 - val_accuracy: 0.6790\n", 1126 | "Epoch 149/10000\n", 1127 | "3996/3996 - 307s - loss: 0.5982 - accuracy: 0.7528 - val_loss: 0.8013 - val_accuracy: 0.6685\n", 1128 | "Epoch 150/10000\n", 1129 | "3996/3996 - 302s - loss: 0.5954 - accuracy: 0.7535 - val_loss: 0.7713 - val_accuracy: 0.6767\n", 1130 | "Epoch 151/10000\n", 1131 | "3996/3996 - 304s - loss: 0.5922 - accuracy: 0.7553 - val_loss: 0.8350 - val_accuracy: 0.6505\n", 1132 | "Epoch 152/10000\n", 1133 | "3996/3996 - 303s - loss: 0.5954 - accuracy: 0.7530 - val_loss: 0.7580 - val_accuracy: 0.6862\n" 1134 | ] 1135 | }, 1136 | { 1137 | "name": "stdout", 1138 | "output_type": "stream", 1139 | "text": [ 1140 | "Epoch 153/10000\n", 1141 | "3996/3996 - 302s - loss: 0.5890 - accuracy: 0.7577 - val_loss: 0.8001 - val_accuracy: 0.6663\n", 1142 | "Epoch 154/10000\n", 1143 | "3996/3996 - 305s - loss: 0.5948 - accuracy: 0.7552 - val_loss: 0.7463 - val_accuracy: 0.6948\n", 1144 | "Epoch 155/10000\n", 1145 | "3996/3996 - 300s - loss: 0.5823 - accuracy: 0.7597 - val_loss: 0.7629 - val_accuracy: 0.6862\n", 1146 | "Epoch 156/10000\n", 1147 | "3996/3996 - 311s - loss: 0.5837 - accuracy: 0.7586 - val_loss: 0.7551 - val_accuracy: 0.6874\n", 1148 | "Epoch 157/10000\n", 1149 | "3996/3996 - 303s - loss: 0.5816 - accuracy: 0.7607 - val_loss: 0.7496 - val_accuracy: 0.6889\n", 1150 | "Epoch 158/10000\n", 1151 | "3996/3996 - 301s - loss: 0.5822 - accuracy: 0.7596 - val_loss: 0.7935 - val_accuracy: 0.6719\n", 1152 | "Epoch 159/10000\n", 1153 | "3996/3996 - 301s - loss: 0.5805 - accuracy: 0.7617 - val_loss: 0.7535 - val_accuracy: 0.6895\n", 1154 | "Epoch 160/10000\n", 1155 | "3996/3996 - 304s - loss: 0.5824 - accuracy: 0.7598 - val_loss: 0.7802 - val_accuracy: 0.6774\n", 1156 | "Epoch 161/10000\n", 1157 | "3996/3996 - 302s - loss: 0.5807 - accuracy: 0.7617 - val_loss: 0.7396 - val_accuracy: 0.6966\n", 1158 | "Epoch 162/10000\n", 1159 | "3996/3996 - 302s - loss: 0.5773 - accuracy: 0.7640 - val_loss: 0.7762 - val_accuracy: 0.6845\n", 1160 | "Epoch 163/10000\n", 1161 | "3996/3996 - 304s - loss: 0.5797 - accuracy: 0.7618 - val_loss: 0.7568 - val_accuracy: 0.6865\n", 1162 | "Epoch 164/10000\n", 1163 | "3996/3996 - 302s - loss: 0.5748 - accuracy: 0.7638 - val_loss: 0.7427 - val_accuracy: 0.6922\n", 1164 | "Epoch 165/10000\n", 1165 | "3996/3996 - 303s - loss: 0.5730 - accuracy: 0.7661 - val_loss: 0.8040 - val_accuracy: 0.6636\n", 1166 | "Epoch 166/10000\n", 1167 | "3996/3996 - 305s - loss: 0.5719 - accuracy: 0.7658 - val_loss: 0.7598 - val_accuracy: 0.6847\n", 1168 | "Epoch 167/10000\n", 1169 | "3996/3996 - 300s - loss: 0.5714 - accuracy: 0.7655 - val_loss: 0.7764 - val_accuracy: 0.6830\n", 1170 | "Epoch 168/10000\n", 1171 | "3996/3996 - 300s - loss: 0.5674 - accuracy: 0.7674 - val_loss: 0.7636 - val_accuracy: 0.6893\n", 1172 | "Epoch 169/10000\n", 1173 | "3996/3996 - 301s - loss: 0.5669 - accuracy: 0.7675 - val_loss: 0.7469 - val_accuracy: 0.6883\n", 1174 | "Epoch 170/10000\n", 1175 | "3996/3996 - 301s - loss: 0.5678 - accuracy: 0.7668 - val_loss: 0.7383 - val_accuracy: 0.6959\n", 1176 | "Epoch 171/10000\n", 1177 | "3996/3996 - 301s - loss: 0.5676 - accuracy: 0.7667 - val_loss: 0.7811 - val_accuracy: 0.6800\n", 1178 | "Epoch 172/10000\n", 1179 | "3996/3996 - 314s - loss: 0.5619 - accuracy: 0.7689 - val_loss: 0.7729 - val_accuracy: 0.6802\n", 1180 | "Epoch 173/10000\n", 1181 | "3996/3996 - 300s - loss: 0.5640 - accuracy: 0.7675 - val_loss: 0.7647 - val_accuracy: 0.6820\n", 1182 | "Epoch 174/10000\n", 1183 | "3996/3996 - 300s - loss: 0.5641 - accuracy: 0.7686 - val_loss: 0.7511 - val_accuracy: 0.6888\n", 1184 | "Epoch 175/10000\n", 1185 | "3996/3996 - 302s - loss: 0.5584 - accuracy: 0.7723 - val_loss: 0.7693 - val_accuracy: 0.6891\n", 1186 | "Epoch 176/10000\n", 1187 | "3996/3996 - 301s - loss: 0.5558 - accuracy: 0.7738 - val_loss: 0.7415 - val_accuracy: 0.6973\n", 1188 | "Epoch 177/10000\n", 1189 | "3996/3996 - 302s - loss: 0.5614 - accuracy: 0.7688 - val_loss: 0.7588 - val_accuracy: 0.6860\n", 1190 | "Epoch 178/10000\n", 1191 | "3996/3996 - 304s - loss: 0.5570 - accuracy: 0.7725 - val_loss: 0.7639 - val_accuracy: 0.6835\n", 1192 | "Epoch 179/10000\n", 1193 | "3996/3996 - 301s - loss: 0.5544 - accuracy: 0.7737 - val_loss: 0.7625 - val_accuracy: 0.6904\n", 1194 | "Epoch 180/10000\n", 1195 | "3996/3996 - 301s - loss: 0.5513 - accuracy: 0.7743 - val_loss: 0.7522 - val_accuracy: 0.6930\n", 1196 | "Epoch 181/10000\n", 1197 | "3996/3996 - 301s - loss: 0.5537 - accuracy: 0.7728 - val_loss: 0.7709 - val_accuracy: 0.6795\n", 1198 | "Epoch 182/10000\n", 1199 | "3996/3996 - 302s - loss: 0.5534 - accuracy: 0.7730 - val_loss: 0.7766 - val_accuracy: 0.6824\n", 1200 | "Epoch 183/10000\n", 1201 | "3996/3996 - 302s - loss: 0.5483 - accuracy: 0.7750 - val_loss: 0.7395 - val_accuracy: 0.6983\n", 1202 | "Epoch 184/10000\n", 1203 | "3996/3996 - 303s - loss: 0.5479 - accuracy: 0.7757 - val_loss: 0.7293 - val_accuracy: 0.7028\n", 1204 | "Epoch 185/10000\n", 1205 | "3996/3996 - 302s - loss: 0.5534 - accuracy: 0.7713 - val_loss: 0.7630 - val_accuracy: 0.6858\n", 1206 | "Epoch 186/10000\n", 1207 | "3996/3996 - 302s - loss: 0.5459 - accuracy: 0.7761 - val_loss: 0.7857 - val_accuracy: 0.6754\n", 1208 | "Epoch 187/10000\n", 1209 | "3996/3996 - 304s - loss: 0.5440 - accuracy: 0.7777 - val_loss: 0.7370 - val_accuracy: 0.6957\n", 1210 | "Epoch 188/10000\n", 1211 | "3996/3996 - 302s - loss: 0.5427 - accuracy: 0.7786 - val_loss: 0.7328 - val_accuracy: 0.6981\n", 1212 | "Epoch 189/10000\n", 1213 | "3996/3996 - 303s - loss: 0.5437 - accuracy: 0.7786 - val_loss: 0.7496 - val_accuracy: 0.6943\n", 1214 | "Epoch 190/10000\n", 1215 | "3996/3996 - 305s - loss: 0.5488 - accuracy: 0.7744 - val_loss: 0.8212 - val_accuracy: 0.6571\n", 1216 | "Epoch 191/10000\n", 1217 | "3996/3996 - 301s - loss: 0.5437 - accuracy: 0.7785 - val_loss: 0.7432 - val_accuracy: 0.6967\n", 1218 | "Epoch 192/10000\n", 1219 | "3996/3996 - 301s - loss: 0.5422 - accuracy: 0.7792 - val_loss: 0.7561 - val_accuracy: 0.6871\n", 1220 | "Epoch 193/10000\n", 1221 | "3996/3996 - 301s - loss: 0.5445 - accuracy: 0.7774 - val_loss: 0.7719 - val_accuracy: 0.6812\n", 1222 | "Epoch 194/10000\n", 1223 | "3996/3996 - 301s - loss: 0.5415 - accuracy: 0.7792 - val_loss: 0.7571 - val_accuracy: 0.6920\n", 1224 | "Epoch 195/10000\n", 1225 | "3996/3996 - 302s - loss: 0.5420 - accuracy: 0.7792 - val_loss: 0.7616 - val_accuracy: 0.6878\n", 1226 | "Epoch 196/10000\n", 1227 | "3996/3996 - 302s - loss: 0.5370 - accuracy: 0.7812 - val_loss: 0.7793 - val_accuracy: 0.6827\n", 1228 | "Epoch 197/10000\n", 1229 | "3996/3996 - 299s - loss: 0.5360 - accuracy: 0.7818 - val_loss: 0.7812 - val_accuracy: 0.6817\n", 1230 | "Epoch 198/10000\n", 1231 | "3996/3996 - 300s - loss: 0.5404 - accuracy: 0.7807 - val_loss: 0.7861 - val_accuracy: 0.6795\n", 1232 | "Epoch 199/10000\n", 1233 | "3996/3996 - 302s - loss: 0.5337 - accuracy: 0.7814 - val_loss: 0.7762 - val_accuracy: 0.6849\n", 1234 | "Epoch 200/10000\n", 1235 | "3996/3996 - 300s - loss: 0.5328 - accuracy: 0.7828 - val_loss: 0.7544 - val_accuracy: 0.6946\n", 1236 | "Epoch 201/10000\n", 1237 | "3996/3996 - 301s - loss: 0.5363 - accuracy: 0.7816 - val_loss: 0.7555 - val_accuracy: 0.6932\n", 1238 | "Epoch 202/10000\n", 1239 | "3996/3996 - 303s - loss: 0.5277 - accuracy: 0.7852 - val_loss: 0.7491 - val_accuracy: 0.6902\n", 1240 | "Epoch 203/10000\n", 1241 | "3996/3996 - 300s - loss: 0.5290 - accuracy: 0.7844 - val_loss: 0.7398 - val_accuracy: 0.6996\n", 1242 | "Epoch 204/10000\n", 1243 | "3996/3996 - 301s - loss: 0.5271 - accuracy: 0.7854 - val_loss: 0.7325 - val_accuracy: 0.6997\n", 1244 | "Epoch 00204: early stopping\n" 1245 | ] 1246 | }, 1247 | { 1248 | "data": { 1249 | "text/plain": [ 1250 | "" 1251 | ] 1252 | }, 1253 | "execution_count": 44, 1254 | "metadata": {}, 1255 | "output_type": "execute_result" 1256 | } 1257 | ], 1258 | "source": [ 1259 | "es = EarlyStopping(monitor='val_accuracy', mode='max', patience = stop_epoch_num, verbose=1)\n", 1260 | "DeepLOB_model_2.fit(X_train, y_train, epochs=10000, batch_size=batch_size, verbose=2, validation_data=(X_test, y_test), callbacks = [es])" 1261 | ] 1262 | }, 1263 | { 1264 | "cell_type": "code", 1265 | "execution_count": 75, 1266 | "metadata": {}, 1267 | "outputs": [ 1268 | { 1269 | "name": "stdout", 1270 | "output_type": "stream", 1271 | "text": [ 1272 | "(None, 100, 40, 1)\n", 1273 | "(None, 100, 20, 16)\n", 1274 | "(None, 100, 20, 16)\n", 1275 | "(None, 100, 20, 16)\n", 1276 | "(None, 100, 20, 16)\n", 1277 | "(None, 100, 10, 16)\n", 1278 | "(None, 100, 10, 16)\n", 1279 | "(None, 100, 10, 16)\n", 1280 | "(None, 100, 1, 16)\n", 1281 | "(None, 100, 1, 16)\n", 1282 | "(None, 100, 1, 16)\n", 1283 | "(None, 100, 1, 32)\n", 1284 | "(None, 100, 1, 32)\n", 1285 | "(None, 100, 1, 32)\n", 1286 | "(None, 100, 1, 32)\n", 1287 | "(None, 100, 1, 16)\n", 1288 | "(None, 100, 1, 32)\n", 1289 | "(None, 100, 1, 32)\n", 1290 | "(None, 100, 1, 96)\n", 1291 | "(None, 100, 96)\n", 1292 | "(None, 64)\n", 1293 | "(None, 3)\n" 1294 | ] 1295 | } 1296 | ], 1297 | "source": [ 1298 | "DeepLOB_model_3_k26 = initiate_DeepLOB_model(lookback_timestep, feature_num, conv_filter_num, inception_num, LSTM_num, leaky_relu_alpha,\n", 1299 | " loss, optimizer, metrics)" 1300 | ] 1301 | }, 1302 | { 1303 | "cell_type": "code", 1304 | "execution_count": 28, 1305 | "metadata": {}, 1306 | "outputs": [ 1307 | { 1308 | "name": "stdout", 1309 | "output_type": "stream", 1310 | "text": [ 1311 | "Epoch 1/120\n", 1312 | "3996/3996 - 296s - loss: 1.0978 - accuracy: 0.3454 - val_loss: 1.0965 - val_accuracy: 0.3413\n", 1313 | "Epoch 2/120\n", 1314 | "3996/3996 - 298s - loss: 1.0924 - accuracy: 0.3705 - val_loss: 1.0972 - val_accuracy: 0.3639\n", 1315 | "Epoch 3/120\n", 1316 | "3996/3996 - 302s - loss: 1.0727 - accuracy: 0.4107 - val_loss: 1.0507 - val_accuracy: 0.4428\n", 1317 | "Epoch 4/120\n", 1318 | "3996/3996 - 302s - loss: 1.0364 - accuracy: 0.4601 - val_loss: 1.0221 - val_accuracy: 0.4733\n", 1319 | "Epoch 5/120\n", 1320 | "3996/3996 - 301s - loss: 1.0136 - accuracy: 0.4812 - val_loss: 1.0104 - val_accuracy: 0.4813\n", 1321 | "Epoch 6/120\n", 1322 | "3996/3996 - 302s - loss: 1.0052 - accuracy: 0.4898 - val_loss: 1.0027 - val_accuracy: 0.4869\n", 1323 | "Epoch 7/120\n", 1324 | "3996/3996 - 305s - loss: 0.9996 - accuracy: 0.4926 - val_loss: 1.0000 - val_accuracy: 0.4907\n", 1325 | "Epoch 8/120\n", 1326 | "3996/3996 - 302s - loss: 0.9948 - accuracy: 0.4959 - val_loss: 1.0002 - val_accuracy: 0.4890\n", 1327 | "Epoch 9/120\n", 1328 | "3996/3996 - 305s - loss: 0.9902 - accuracy: 0.4990 - val_loss: 0.9917 - val_accuracy: 0.4989\n", 1329 | "Epoch 10/120\n", 1330 | "3996/3996 - 302s - loss: 0.9861 - accuracy: 0.5016 - val_loss: 0.9876 - val_accuracy: 0.5003\n", 1331 | "Epoch 11/120\n", 1332 | "3996/3996 - 302s - loss: 0.9824 - accuracy: 0.5050 - val_loss: 0.9839 - val_accuracy: 0.5027\n", 1333 | "Epoch 12/120\n", 1334 | "3996/3996 - 303s - loss: 0.9784 - accuracy: 0.5104 - val_loss: 1.0057 - val_accuracy: 0.4883\n", 1335 | "Epoch 13/120\n", 1336 | "3996/3996 - 306s - loss: 0.9743 - accuracy: 0.5128 - val_loss: 0.9772 - val_accuracy: 0.5123\n", 1337 | "Epoch 14/120\n", 1338 | "3996/3996 - 305s - loss: 0.9699 - accuracy: 0.5173 - val_loss: 0.9783 - val_accuracy: 0.5099\n", 1339 | "Epoch 15/120\n", 1340 | "3996/3996 - 300s - loss: 0.9652 - accuracy: 0.5224 - val_loss: 0.9751 - val_accuracy: 0.5178\n", 1341 | "Epoch 16/120\n", 1342 | "3996/3996 - 300s - loss: 0.9606 - accuracy: 0.5266 - val_loss: 0.9683 - val_accuracy: 0.5197\n", 1343 | "Epoch 17/120\n", 1344 | "3996/3996 - 301s - loss: 0.9546 - accuracy: 0.5315 - val_loss: 0.9567 - val_accuracy: 0.5316\n", 1345 | "Epoch 18/120\n", 1346 | "3996/3996 - 300s - loss: 0.9493 - accuracy: 0.5352 - val_loss: 0.9622 - val_accuracy: 0.5276\n", 1347 | "Epoch 19/120\n", 1348 | "3996/3996 - 303s - loss: 0.9432 - accuracy: 0.5395 - val_loss: 0.9525 - val_accuracy: 0.5367\n", 1349 | "Epoch 20/120\n", 1350 | "3996/3996 - 301s - loss: 0.9383 - accuracy: 0.5434 - val_loss: 0.9580 - val_accuracy: 0.5288\n", 1351 | "Epoch 21/120\n", 1352 | "3996/3996 - 304s - loss: 0.9314 - accuracy: 0.5492 - val_loss: 0.9447 - val_accuracy: 0.5348\n", 1353 | "Epoch 22/120\n", 1354 | "3996/3996 - 302s - loss: 0.9242 - accuracy: 0.5547 - val_loss: 0.9351 - val_accuracy: 0.5469\n", 1355 | "Epoch 23/120\n", 1356 | "3996/3996 - 302s - loss: 0.9170 - accuracy: 0.5602 - val_loss: 0.9311 - val_accuracy: 0.5504\n", 1357 | "Epoch 24/120\n", 1358 | "3996/3996 - 303s - loss: 0.9093 - accuracy: 0.5656 - val_loss: 0.9155 - val_accuracy: 0.5659\n", 1359 | "Epoch 25/120\n", 1360 | "3996/3996 - 303s - loss: 0.9013 - accuracy: 0.5740 - val_loss: 0.9419 - val_accuracy: 0.5443\n", 1361 | "Epoch 26/120\n", 1362 | "3996/3996 - 304s - loss: 0.8930 - accuracy: 0.5784 - val_loss: 0.9027 - val_accuracy: 0.5717\n", 1363 | "Epoch 27/120\n", 1364 | "3996/3996 - 303s - loss: 0.8857 - accuracy: 0.5836 - val_loss: 0.9086 - val_accuracy: 0.5681\n", 1365 | "Epoch 28/120\n", 1366 | "3996/3996 - 304s - loss: 0.8777 - accuracy: 0.5880 - val_loss: 0.8970 - val_accuracy: 0.5795\n", 1367 | "Epoch 29/120\n", 1368 | "3996/3996 - 303s - loss: 0.8685 - accuracy: 0.5958 - val_loss: 0.8799 - val_accuracy: 0.5903\n", 1369 | "Epoch 30/120\n", 1370 | "3996/3996 - 304s - loss: 0.8577 - accuracy: 0.6032 - val_loss: 0.8976 - val_accuracy: 0.5781\n", 1371 | "Epoch 31/120\n", 1372 | "3996/3996 - 304s - loss: 0.8472 - accuracy: 0.6098 - val_loss: 0.8748 - val_accuracy: 0.5931\n", 1373 | "Epoch 32/120\n", 1374 | "3996/3996 - 303s - loss: 0.8419 - accuracy: 0.6132 - val_loss: 0.8519 - val_accuracy: 0.6092\n", 1375 | "Epoch 33/120\n", 1376 | "3996/3996 - 304s - loss: 0.8328 - accuracy: 0.6175 - val_loss: 0.8586 - val_accuracy: 0.6006\n", 1377 | "Epoch 34/120\n", 1378 | "3996/3996 - 303s - loss: 0.8224 - accuracy: 0.6223 - val_loss: 0.8460 - val_accuracy: 0.6147\n", 1379 | "Epoch 35/120\n", 1380 | "3996/3996 - 306s - loss: 0.8097 - accuracy: 0.6313 - val_loss: 0.8216 - val_accuracy: 0.6258\n", 1381 | "Epoch 36/120\n", 1382 | "3996/3996 - 315s - loss: 0.8032 - accuracy: 0.6347 - val_loss: 0.8282 - val_accuracy: 0.6237\n", 1383 | "Epoch 37/120\n", 1384 | "3996/3996 - 405s - loss: 0.7928 - accuracy: 0.6428 - val_loss: 0.8006 - val_accuracy: 0.6345\n", 1385 | "Epoch 38/120\n", 1386 | "3996/3996 - 317s - loss: 0.7891 - accuracy: 0.6447 - val_loss: 0.8052 - val_accuracy: 0.6377\n", 1387 | "Epoch 39/120\n", 1388 | "3996/3996 - 332s - loss: 0.7735 - accuracy: 0.6524 - val_loss: 0.8009 - val_accuracy: 0.6391\n", 1389 | "Epoch 40/120\n", 1390 | "3996/3996 - 331s - loss: 0.7665 - accuracy: 0.6564 - val_loss: 0.7973 - val_accuracy: 0.6382\n", 1391 | "Epoch 41/120\n", 1392 | "3996/3996 - 327s - loss: 0.7590 - accuracy: 0.6608 - val_loss: 0.7694 - val_accuracy: 0.6555\n", 1393 | "Epoch 42/120\n", 1394 | "3996/3996 - 372s - loss: 0.7454 - accuracy: 0.6692 - val_loss: 0.8091 - val_accuracy: 0.6365\n", 1395 | "Epoch 43/120\n", 1396 | "3996/3996 - 344s - loss: 0.7376 - accuracy: 0.6727 - val_loss: 0.7716 - val_accuracy: 0.6570\n", 1397 | "Epoch 44/120\n", 1398 | "3996/3996 - 343s - loss: 0.7289 - accuracy: 0.6776 - val_loss: 0.7640 - val_accuracy: 0.6569\n", 1399 | "Epoch 45/120\n", 1400 | "3996/3996 - 339s - loss: 0.7184 - accuracy: 0.6834 - val_loss: 0.7369 - val_accuracy: 0.6771\n", 1401 | "Epoch 46/120\n", 1402 | "3996/3996 - 341s - loss: 0.7088 - accuracy: 0.6888 - val_loss: 0.7277 - val_accuracy: 0.6820\n", 1403 | "Epoch 47/120\n", 1404 | "3996/3996 - 351s - loss: 0.7038 - accuracy: 0.6904 - val_loss: 0.7439 - val_accuracy: 0.6738\n", 1405 | "Epoch 48/120\n", 1406 | "3996/3996 - 348s - loss: 0.6938 - accuracy: 0.6974 - val_loss: 0.7606 - val_accuracy: 0.6601\n", 1407 | "Epoch 49/120\n", 1408 | "3996/3996 - 350s - loss: 0.6829 - accuracy: 0.7027 - val_loss: 0.7244 - val_accuracy: 0.6820\n", 1409 | "Epoch 50/120\n", 1410 | "3996/3996 - 345s - loss: 0.6780 - accuracy: 0.7068 - val_loss: 0.7418 - val_accuracy: 0.6758\n", 1411 | "Epoch 51/120\n", 1412 | "3996/3996 - 321s - loss: 0.6645 - accuracy: 0.7142 - val_loss: 0.7371 - val_accuracy: 0.6731\n", 1413 | "Epoch 52/120\n", 1414 | "3996/3996 - 310s - loss: 0.6622 - accuracy: 0.7135 - val_loss: 0.7252 - val_accuracy: 0.6829\n", 1415 | "Epoch 53/120\n", 1416 | "3996/3996 - 349s - loss: 0.6467 - accuracy: 0.7222 - val_loss: 0.7092 - val_accuracy: 0.6956\n", 1417 | "Epoch 54/120\n", 1418 | "3996/3996 - 311s - loss: 0.6432 - accuracy: 0.7231 - val_loss: 0.6741 - val_accuracy: 0.7091\n", 1419 | "Epoch 55/120\n", 1420 | "3996/3996 - 313s - loss: 0.6318 - accuracy: 0.7295 - val_loss: 0.7065 - val_accuracy: 0.6957\n", 1421 | "Epoch 56/120\n", 1422 | "3996/3996 - 329s - loss: 0.6264 - accuracy: 0.7330 - val_loss: 0.6810 - val_accuracy: 0.7065\n", 1423 | "Epoch 57/120\n", 1424 | "3996/3996 - 642s - loss: 0.6186 - accuracy: 0.7368 - val_loss: 0.6660 - val_accuracy: 0.7152\n", 1425 | "Epoch 58/120\n", 1426 | "3996/3996 - 652s - loss: 0.6126 - accuracy: 0.7406 - val_loss: 0.6736 - val_accuracy: 0.7103\n", 1427 | "Epoch 59/120\n", 1428 | "3996/3996 - 658s - loss: 0.6036 - accuracy: 0.7440 - val_loss: 0.6744 - val_accuracy: 0.7095\n", 1429 | "Epoch 60/120\n", 1430 | "3996/3996 - 658s - loss: 0.6052 - accuracy: 0.7445 - val_loss: 0.6394 - val_accuracy: 0.7279\n", 1431 | "Epoch 61/120\n", 1432 | "3996/3996 - 637s - loss: 0.5917 - accuracy: 0.7513 - val_loss: 0.6361 - val_accuracy: 0.7318\n", 1433 | "Epoch 62/120\n", 1434 | "3996/3996 - 682s - loss: 0.5909 - accuracy: 0.7502 - val_loss: 0.6594 - val_accuracy: 0.7191\n", 1435 | "Epoch 63/120\n", 1436 | "3996/3996 - 421s - loss: 0.5779 - accuracy: 0.7573 - val_loss: 0.6301 - val_accuracy: 0.7335\n", 1437 | "Epoch 64/120\n", 1438 | "3996/3996 - 315s - loss: 0.5794 - accuracy: 0.7554 - val_loss: 0.6501 - val_accuracy: 0.7251\n", 1439 | "Epoch 65/120\n", 1440 | "3996/3996 - 331s - loss: 0.5703 - accuracy: 0.7615 - val_loss: 0.6276 - val_accuracy: 0.7348\n", 1441 | "Epoch 66/120\n", 1442 | "3996/3996 - 332s - loss: 0.5634 - accuracy: 0.7650 - val_loss: 0.6153 - val_accuracy: 0.7437\n", 1443 | "Epoch 67/120\n", 1444 | "3996/3996 - 327s - loss: 0.5575 - accuracy: 0.7682 - val_loss: 0.6345 - val_accuracy: 0.7339\n", 1445 | "Epoch 68/120\n", 1446 | "3996/3996 - 333s - loss: 0.5523 - accuracy: 0.7706 - val_loss: 0.6229 - val_accuracy: 0.7401\n", 1447 | "Epoch 69/120\n", 1448 | "3996/3996 - 332s - loss: 0.5432 - accuracy: 0.7748 - val_loss: 0.6362 - val_accuracy: 0.7330\n", 1449 | "Epoch 70/120\n", 1450 | "3996/3996 - 328s - loss: 0.5373 - accuracy: 0.7772 - val_loss: 0.6184 - val_accuracy: 0.7409\n", 1451 | "Epoch 71/120\n", 1452 | "3996/3996 - 332s - loss: 0.5334 - accuracy: 0.7783 - val_loss: 0.6046 - val_accuracy: 0.7487\n", 1453 | "Epoch 72/120\n", 1454 | "3996/3996 - 332s - loss: 0.5328 - accuracy: 0.7791 - val_loss: 0.6041 - val_accuracy: 0.7496\n", 1455 | "Epoch 73/120\n", 1456 | "3996/3996 - 331s - loss: 0.5318 - accuracy: 0.7806 - val_loss: 0.6132 - val_accuracy: 0.7462\n", 1457 | "Epoch 74/120\n", 1458 | "3996/3996 - 334s - loss: 0.5185 - accuracy: 0.7855 - val_loss: 0.6307 - val_accuracy: 0.7379\n", 1459 | "Epoch 75/120\n", 1460 | "3996/3996 - 329s - loss: 0.5186 - accuracy: 0.7863 - val_loss: 0.6500 - val_accuracy: 0.7319\n", 1461 | "Epoch 76/120\n", 1462 | "3996/3996 - 479s - loss: 0.5126 - accuracy: 0.7891 - val_loss: 0.6083 - val_accuracy: 0.7465\n", 1463 | "Epoch 77/120\n", 1464 | "3996/3996 - 612s - loss: 0.5079 - accuracy: 0.7906 - val_loss: 0.5543 - val_accuracy: 0.7716\n", 1465 | "Epoch 78/120\n", 1466 | "3996/3996 - 318s - loss: 0.5008 - accuracy: 0.7946 - val_loss: 0.6200 - val_accuracy: 0.7423\n" 1467 | ] 1468 | }, 1469 | { 1470 | "name": "stdout", 1471 | "output_type": "stream", 1472 | "text": [ 1473 | "Epoch 79/120\n", 1474 | "3996/3996 - 318s - loss: 0.4981 - accuracy: 0.7964 - val_loss: 0.5825 - val_accuracy: 0.7582\n", 1475 | "Epoch 80/120\n", 1476 | "3996/3996 - 322s - loss: 0.4984 - accuracy: 0.7969 - val_loss: 0.5771 - val_accuracy: 0.7599\n", 1477 | "Epoch 81/120\n", 1478 | "3996/3996 - 325s - loss: 0.4904 - accuracy: 0.8009 - val_loss: 0.6058 - val_accuracy: 0.7531\n", 1479 | "Epoch 82/120\n", 1480 | "3996/3996 - 326s - loss: 0.4955 - accuracy: 0.7987 - val_loss: 0.5614 - val_accuracy: 0.7717\n", 1481 | "Epoch 83/120\n", 1482 | "3996/3996 - 325s - loss: 0.4877 - accuracy: 0.8012 - val_loss: 0.6697 - val_accuracy: 0.7244\n", 1483 | "Epoch 84/120\n", 1484 | "3996/3996 - 320s - loss: 0.4864 - accuracy: 0.8005 - val_loss: 0.5677 - val_accuracy: 0.7680\n", 1485 | "Epoch 85/120\n", 1486 | "3996/3996 - 328s - loss: 0.4782 - accuracy: 0.8041 - val_loss: 0.6067 - val_accuracy: 0.7538\n", 1487 | "Epoch 86/120\n", 1488 | "3996/3996 - 325s - loss: 0.4744 - accuracy: 0.8076 - val_loss: 0.5478 - val_accuracy: 0.7763\n", 1489 | "Epoch 87/120\n", 1490 | "3996/3996 - 326s - loss: 0.4703 - accuracy: 0.8094 - val_loss: 0.5855 - val_accuracy: 0.7600\n", 1491 | "Epoch 88/120\n", 1492 | "3996/3996 - 324s - loss: 0.4705 - accuracy: 0.8084 - val_loss: 0.5504 - val_accuracy: 0.7784\n", 1493 | "Epoch 89/120\n", 1494 | "3996/3996 - 328s - loss: 0.4700 - accuracy: 0.8099 - val_loss: 0.5618 - val_accuracy: 0.7736\n", 1495 | "Epoch 90/120\n", 1496 | "3996/3996 - 322s - loss: 0.4620 - accuracy: 0.8126 - val_loss: 0.5932 - val_accuracy: 0.7603\n", 1497 | "Epoch 91/120\n", 1498 | "3996/3996 - 325s - loss: 0.4609 - accuracy: 0.8145 - val_loss: 0.5642 - val_accuracy: 0.7711\n", 1499 | "Epoch 92/120\n", 1500 | "3996/3996 - 328s - loss: 0.4582 - accuracy: 0.8149 - val_loss: 0.5199 - val_accuracy: 0.7922\n", 1501 | "Epoch 93/120\n", 1502 | "3996/3996 - 324s - loss: 0.4524 - accuracy: 0.8173 - val_loss: 0.5380 - val_accuracy: 0.7832\n", 1503 | "Epoch 94/120\n", 1504 | "3996/3996 - 316s - loss: 0.4563 - accuracy: 0.8151 - val_loss: 0.5418 - val_accuracy: 0.7820\n", 1505 | "Epoch 95/120\n", 1506 | "3996/3996 - 431s - loss: 0.4504 - accuracy: 0.8185 - val_loss: 0.5554 - val_accuracy: 0.7758\n", 1507 | "Epoch 96/120\n", 1508 | "3996/3996 - 308s - loss: 0.4467 - accuracy: 0.8201 - val_loss: 0.5491 - val_accuracy: 0.7772\n", 1509 | "Epoch 97/120\n", 1510 | "3996/3996 - 313s - loss: 0.4482 - accuracy: 0.8175 - val_loss: 0.5690 - val_accuracy: 0.7677\n", 1511 | "Epoch 98/120\n", 1512 | "3996/3996 - 314s - loss: 0.4395 - accuracy: 0.8225 - val_loss: 0.5470 - val_accuracy: 0.7802\n", 1513 | "Epoch 99/120\n", 1514 | "3996/3996 - 336s - loss: 0.4401 - accuracy: 0.8232 - val_loss: 0.6160 - val_accuracy: 0.7514\n", 1515 | "Epoch 100/120\n", 1516 | "3996/3996 - 339s - loss: 0.4378 - accuracy: 0.8234 - val_loss: 0.5371 - val_accuracy: 0.7836\n", 1517 | "Epoch 101/120\n", 1518 | "3996/3996 - 326s - loss: 0.4332 - accuracy: 0.8257 - val_loss: 0.5337 - val_accuracy: 0.7858\n", 1519 | "Epoch 102/120\n", 1520 | "3996/3996 - 329s - loss: 0.4329 - accuracy: 0.8248 - val_loss: 0.5228 - val_accuracy: 0.7943\n", 1521 | "Epoch 103/120\n", 1522 | "3996/3996 - 329s - loss: 0.4292 - accuracy: 0.8276 - val_loss: 0.5899 - val_accuracy: 0.7651\n", 1523 | "Epoch 104/120\n", 1524 | "3996/3996 - 330s - loss: 0.4271 - accuracy: 0.8286 - val_loss: 0.5310 - val_accuracy: 0.7872\n", 1525 | "Epoch 105/120\n", 1526 | "3996/3996 - 325s - loss: 0.4229 - accuracy: 0.8297 - val_loss: 0.5513 - val_accuracy: 0.7737\n", 1527 | "Epoch 106/120\n", 1528 | "3996/3996 - 493s - loss: 0.4250 - accuracy: 0.8280 - val_loss: 0.4998 - val_accuracy: 0.8027\n", 1529 | "Epoch 107/120\n", 1530 | "3996/3996 - 699s - loss: 0.4182 - accuracy: 0.8329 - val_loss: 0.5186 - val_accuracy: 0.7943\n", 1531 | "Epoch 108/120\n", 1532 | "3996/3996 - 704s - loss: 0.4222 - accuracy: 0.8299 - val_loss: 0.5504 - val_accuracy: 0.7797\n", 1533 | "Epoch 109/120\n", 1534 | "3996/3996 - 681s - loss: 0.4157 - accuracy: 0.8330 - val_loss: 0.5093 - val_accuracy: 0.7981\n", 1535 | "Epoch 110/120\n", 1536 | "3996/3996 - 600s - loss: 0.4221 - accuracy: 0.8316 - val_loss: 0.5086 - val_accuracy: 0.7994\n", 1537 | "Epoch 111/120\n", 1538 | "3996/3996 - 311s - loss: 0.4219 - accuracy: 0.8307 - val_loss: 0.5979 - val_accuracy: 0.7594\n", 1539 | "Epoch 112/120\n", 1540 | "3996/3996 - 329s - loss: 0.4133 - accuracy: 0.8340 - val_loss: 0.5461 - val_accuracy: 0.7846\n", 1541 | "Epoch 113/120\n", 1542 | "3996/3996 - 348s - loss: 0.4067 - accuracy: 0.8376 - val_loss: 0.4962 - val_accuracy: 0.8050\n", 1543 | "Epoch 114/120\n", 1544 | "3996/3996 - 356s - loss: 0.4070 - accuracy: 0.8374 - val_loss: 0.5603 - val_accuracy: 0.7788\n", 1545 | "Epoch 115/120\n", 1546 | "3996/3996 - 335s - loss: 0.4007 - accuracy: 0.8391 - val_loss: 0.5273 - val_accuracy: 0.7940\n", 1547 | "Epoch 116/120\n", 1548 | "3996/3996 - 329s - loss: 0.4062 - accuracy: 0.8379 - val_loss: 0.5494 - val_accuracy: 0.7832\n", 1549 | "Epoch 117/120\n", 1550 | "3996/3996 - 334s - loss: 0.4068 - accuracy: 0.8379 - val_loss: 0.5390 - val_accuracy: 0.7883\n", 1551 | "Epoch 118/120\n", 1552 | "3996/3996 - 333s - loss: 0.4023 - accuracy: 0.8398 - val_loss: 0.5197 - val_accuracy: 0.7940\n", 1553 | "Epoch 119/120\n", 1554 | "3996/3996 - 326s - loss: 0.3970 - accuracy: 0.8419 - val_loss: 0.5347 - val_accuracy: 0.7884\n", 1555 | "Epoch 120/120\n", 1556 | "3996/3996 - 331s - loss: 0.3996 - accuracy: 0.8405 - val_loss: 0.5514 - val_accuracy: 0.7754\n" 1557 | ] 1558 | } 1559 | ], 1560 | "source": [ 1561 | "es = EarlyStopping(monitor='val_accuracy', mode='max', patience = stop_epoch_num, verbose=1)\n", 1562 | "history_k26 = DeepLOB_model_3_k26.fit(X_train3, y_train3, epochs=num_epoch, batch_size=batch_size, verbose=2, validation_data=(X_test3, y_test3), callbacks = [es])" 1563 | ] 1564 | } 1565 | ], 1566 | "metadata": { 1567 | "kernelspec": { 1568 | "display_name": "Python 3", 1569 | "language": "python", 1570 | "name": "python3" 1571 | }, 1572 | "language_info": { 1573 | "codemirror_mode": { 1574 | "name": "ipython", 1575 | "version": 3 1576 | }, 1577 | "file_extension": ".py", 1578 | "mimetype": "text/x-python", 1579 | "name": "python", 1580 | "nbconvert_exporter": "python", 1581 | "pygments_lexer": "ipython3", 1582 | "version": "3.8.5" 1583 | } 1584 | }, 1585 | "nbformat": 4, 1586 | "nbformat_minor": 2 1587 | } 1588 | -------------------------------------------------------------------------------- /src/data/data.txt: -------------------------------------------------------------------------------- 1 | All data files are >25MB so they are not uploaded. 2 | -------------------------------------------------------------------------------- /src/images/change_pct_adjust.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/yuxiangalvin/DeepLOB-Model-Implementation-Project/70855aaef6e177a32ccd3eb85e2696fb90ab4f57/src/images/change_pct_adjust.png -------------------------------------------------------------------------------- /src/images/inception_module_structure.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/yuxiangalvin/DeepLOB-Model-Implementation-Project/70855aaef6e177a32ccd3eb85e2696fb90ab4f57/src/images/inception_module_structure.png -------------------------------------------------------------------------------- /src/images/labelling.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/yuxiangalvin/DeepLOB-Model-Implementation-Project/70855aaef6e177a32ccd3eb85e2696fb90ab4f57/src/images/labelling.png -------------------------------------------------------------------------------- /src/images/limit_orderbook_example.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/yuxiangalvin/DeepLOB-Model-Implementation-Project/70855aaef6e177a32ccd3eb85e2696fb90ab4f57/src/images/limit_orderbook_example.png -------------------------------------------------------------------------------- /src/images/mid_adjust.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/yuxiangalvin/DeepLOB-Model-Implementation-Project/70855aaef6e177a32ccd3eb85e2696fb90ab4f57/src/images/mid_adjust.png -------------------------------------------------------------------------------- /src/images/whole_model_structure.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/yuxiangalvin/DeepLOB-Model-Implementation-Project/70855aaef6e177a32ccd3eb85e2696fb90ab4f57/src/images/whole_model_structure.png -------------------------------------------------------------------------------- /src/outputs/FI-2010_Accuracy_Graph.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/yuxiangalvin/DeepLOB-Model-Implementation-Project/70855aaef6e177a32ccd3eb85e2696fb90ab4f57/src/outputs/FI-2010_Accuracy_Graph.png -------------------------------------------------------------------------------- /src/outputs/FI-2010_Loss_Graph.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/yuxiangalvin/DeepLOB-Model-Implementation-Project/70855aaef6e177a32ccd3eb85e2696fb90ab4f57/src/outputs/FI-2010_Loss_Graph.png -------------------------------------------------------------------------------- /src/outputs/JNJ_Accuracy_Graph.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/yuxiangalvin/DeepLOB-Model-Implementation-Project/70855aaef6e177a32ccd3eb85e2696fb90ab4f57/src/outputs/JNJ_Accuracy_Graph.png -------------------------------------------------------------------------------- /src/outputs/JNJ_Loss_Graph.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/yuxiangalvin/DeepLOB-Model-Implementation-Project/70855aaef6e177a32ccd3eb85e2696fb90ab4f57/src/outputs/JNJ_Loss_Graph.png -------------------------------------------------------------------------------- /src/outputs/outputs.txt: -------------------------------------------------------------------------------- 1 | 2 | --------------------------------------------------------------------------------